Abstract
Long-context language models do not use the full window uniformly — performance degrades for information placed in the middle of long inputs (Liu et al., 2023).
Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2023). Lost in the Middle: How Language Models Use Long Contexts. arXiv. DOI
Key findings
- U-shaped use: — Models retrieve best from the start and end of long contexts; middle content is easier to miss.
- Window ≠ memory: — A larger context budget does not guarantee reliable use of everything you paste.
Application
When a chat thread “knew” last week and now invents with confidence — open this cite before you trust folklore over an attached project file.