Causal Language Modeling

Causal Language Modeling is the core training paradigm of large generative language models. At each step, the model predicts the next token based only on previous tokens. This seemingly simple objective can, with enough data and scale, yield strong generation, reasoning, and adaptability. It forms the basis of text generation, code generation, and general-purpose language intelligence systems.