Glossary Library

Technical GlossaryGenerative AI and LLM

QLoRA

An approach that performs LoRA adaptation on a quantized base model to enable fine-tuning at lower hardware cost.

QLoRA is a major practical innovation that makes it possible to adapt large language models with more limited GPU memory. The base model is kept at lower precision while LoRA layers are trained. This significantly reduces training cost while preserving task performance in many settings. It is especially valuable for individual developers and enterprise prototyping.

You Might Also Like

Explore these concepts to continue your artificial intelligence journey.

Glossary Cover

yapay-zeka-temelleri

Generative AI

A class of AI systems capable of generating new content such as text, images, audio, video, or code.

Glossary Cover

Decoder-Only Transformer

A modern large-language-model architecture that generates autoregressively by predicting the next token.

Glossary Cover

Causal Attention

An autoregressive attention structure that allows a token to attend only to positions at or before itself.