Glossary Library

Technical GlossaryNatural Language Processing

Continued Pretraining

The process of further training a pretrained language model on new data to improve general or domain-specific knowledge.

Continued pretraining is a strong strategy for adapting a model to new domains without the cost of training from scratch. A general model can be further pretrained on legal, healthcare, finance, or enterprise-specific terminology. This plays an important role in domain adaptation and enterprise LLM design.

You Might Also Like

Explore these concepts to continue your artificial intelligence journey.

Glossary Cover

yapay-zeka-temelleri

Generative AI

A class of AI systems capable of generating new content such as text, images, audio, video, or code.

Glossary Cover

Decoder-Only Transformer

A modern large-language-model architecture that generates autoregressively by predicting the next token.

Glossary Cover

Causal Attention

An autoregressive attention structure that allows a token to attend only to positions at or before itself.