Multi-Head Attention

Multi-head attention enables the model to learn multiple relationship patterns simultaneously instead of relying on a single attention map. Some heads may focus on local context, others on long-range dependencies, and still others on different semantic structures. This multiplicity significantly increases the representational power of Transformer models. It has become a standard architectural component in modern language and multimodal systems.