3 posts
In self-hosted LLMs, bill and latency come from the serving layer. Manifold throughput via PagedAttention, continuous batching, speculative decoding, and quantization.
In LLMs the real cost is inference. I explain how I cut cost across the model, system and application layers with caching, routing, quantization and batching.
Large language models have become one of the most influential technologies in modern AI. Yet they are often explained too superficially, as if they were merely “text prediction engines trained on huge amounts of data.” While that description is not entirely wrong, it is far from sufficient. Without understanding transformer architecture, tokenization, self-attention, representation learning, and inference dynamics, it is impossible to understand how LLMs actually behave. This guide provides a systematic and technically grounded explanation of how large language models work, from tokens and embeddings to transformer blocks, attention, training, inference, and sampling.