Why is a long context not reliable long-term memory?

The bottleneck

Accepting more tokens establishes an input capacity, not reliable use of every fact. Lost in the Middle found positional sensitivity in its tested models and retrieval/QA tasks, with information in the middle sometimes used less effectively.

The mechanism

RoPE rotates queries and keys so their dot products encode relative position. Extending windows also requires training-length coverage, positional extrapolation or scaling and validation. It does not solve all retrieval interference. Measure capacity, retrieval and reasoning separately.

dense attention work ∝ T²; standard KV payload ∝ T

Examples and evidence

Teaching protocol: place the same evidence at the beginning, middle and end, adding similar distractors. Compare citation accuracy and answer quality while holding question and length fixed. Historical results are not a verdict on every current model.

New constraints

Long inputs increase prefill work and KV memory. Compression, retrieval and summaries may omit crucial details. Systems need evidence localization, citation checks and task-level evaluation rather than selection by maximum window size alone.