Attention

A mechanism that enables a model to learn which parts of the input deserve more focus during prediction.

The attention mechanism makes it possible to focus more strongly on the most informative parts of the input instead of treating everything equally. This idea allows models to use capacity more efficiently, especially in long sequences and complex representation problems. It has had a profound impact across language processing, visual attention, and multimodal learning. Much of modern AI is built around the idea of attention.