← Back to Glossary
Self-Attention Mechanism
A calculation that lets an LLM weigh how relevant every word in a sequence is to every other word.
Why it exists
Words get meaning from context. Self-attention allows an LLM to link 'it' to 'the dog' or 'river' to 'bank' across a long sentence.
How it works
Computes Query, Key, and Value vectors for each token. Dot-product between Query and Key yields attention weights, which scale Value vectors into context-aware outputs.
Related terms
Where you'll use this
roadmap