← Back to Glossary
Context Rot (Lost in the Middle)
The degraded ability of an LLM to recall information located in the middle of long context windows compared to the top or bottom.
Why it exists
Transformer attention mechanisms naturally focus heavier weights on the start (system prompt) and end (recent tokens) of long context sequences.
How it works
Facts placed at token position 50,000 in a 100,000 token prompt are missed significantly more often than facts at position 100 or 99,000.
Related terms
Where you'll use this
roadmap