Why does AI miss information in the middle of a long prompt?
Large language models read the beginning and the end of a prompt far more reliably than the middle. Researchers call this the lost-in-the-middle effect. Paste a 40-page report, ask about a clause on page 22, and the model may answer from page 2 and page 40 instead, even though page 22 is sitting right there in its context window.
AI is bad at reading evenly. That is the honest limitation this article starts from. The context window fills up, attention concentrates at the two edges, and the fact you needed most quietly loses the competition for the model's focus. You did not write a bad prompt. You put the important part in the wrong place.
Definition: lost-in-the-middle is the tendency of a language model to use information placed near the start or the end of its input more reliably than the same information placed in the middle, producing a U-shaped accuracy curve across positions.
The core reference is the paper Lost in the Middle: How Language Models Use Long Contexts by Nelson F. Liu and colleagues, published in TACL in 2024. In their multi-document question-answering tests, accuracy was highest when the answer sat in the first or last document and dropped sharply when the same answer sat in the middle of the stack.
This matters more in September 2026 than it did two years ago, because context windows just got bigger again and more people are filling them.
Does a bigger context window fix the problem?
No. A larger context window lets you paste more, but it does not make the model attend to every token equally. Bigger windows mostly add capacity for distraction. The practical question is not how many tokens a model accepts, but how many it can use accurately at once, and that number is smaller than the headline figure.
The headline figures keep climbing. On 14 September 2026, ChatGPT raised the total context window for manually selected Thinking mode to 256K tokens (128K input plus 128K output), up from 196K, as documented by AI researcher Tibor Blaho. Gemini's Pro-tier models and the top Claude tiers advertise windows in the millions of tokens.
The usable figure lags the advertised one. Chroma's context rot study tested 18 models, including GPT-4.1, Claude 4 and Gemini 2.5, and found that none of them used long inputs uniformly; reliability fell as input length grew. A separate 2025 paper on the maximum effective context window found that tested models fell short of their advertised maximum by as much as 99 percent on some tasks.
Google Research's Found in the Middle work explains the mechanism: tokens at the start and end of the input receive more attention regardless of how relevant they are. The middle is simply farther from both the task framing and the final question.
For a practitioner, the lesson is blunt. The 256K upgrade is welcome, but it does not change where you should put the clause you need answered.
Where should you put the most important information in a prompt?
Put durable rules and the source material's most critical facts at the top, put the actual question and output requirements at the bottom, and never let the single fact that decides the answer sit in the middle third of a long paste. If the key passage must stay inside the document, quote it again next to the question.
Think of a long prompt as three zones with different reading quality.
--- Top zone (strong recall): role, standing rules, the definition or figure everything depends on, and a one-line statement of what you will ask at the end.
--- Middle zone (weak recall): the bulk of the pasted material. Anything that lives only here is at risk.
--- Bottom zone (strong recall): the question itself, output format, constraints, and a repeated quote of the decisive passage.
The most useful single trick comes from Anthropic's own long-context guidance for Claude 2.1. Adding the line "Here is the most relevant sentence in the context:" and asking the model to pull that sentence out before answering raised retrieval accuracy on a long-document test from 27 percent to 98 percent, according to Anthropic's published write-up. The model version is old; the underlying habit of forcing an explicit extraction step still works on 2026 models in day-to-day use.
A second trick is redundancy at the edges. If a definition on page 22 decides the answer, quote it verbatim in the bottom zone right before your question. You are not cheating; you are compensating for a known attention bias.
How do you structure a long-document prompt so nothing gets lost?
Use labelled sections in a fixed order: rules, then the key facts, then the full document, then the extraction step, then the question and format. Labels such as RULES, KEY FACTS, DOCUMENT and TASK act as signposts, and the extraction step forces the model to locate the decisive passage before it writes anything.
Here is a complete template you can paste today. It is written for ChatGPT, Claude or Gemini and works in a plain chat window; no API access needed.
Try this prompt:
RULES
You are a careful analyst. Answer only from the DOCUMENT below. If the answer is not in the document, say "Not found in document." Do not summarise the whole document unless asked.
KEY FACTS TO KEEP IN MIND
1. The contract renewal date is 31 March 2027.
2. Termination requires 90 days written notice.
3. Late-payment interest is 1.5% per month.
DOCUMENT
[paste the full document here]
EXTRACTION STEP
Before answering, quote verbatim the one to three sentences from the DOCUMENT most relevant to the TASK, with their section numbers.
TASK
Question: If we send notice on 15 January 2027, will we exit before the renewal date, and what penalty applies?
Output format: (1) Quoted sentences, (2) One-paragraph answer, (3) A one-line confidence rating with the reason.
Three details make this template work. The KEY FACTS block sits in the top zone, so the model anchors on the right numbers before it wades into the paste. The EXTRACTION STEP turns a retrieval problem into a copying task, which models are good at. The TASK sits at the very bottom, closest to generation.
If you already use a structured contract for outputs, this slots in neatly; see UD's guide to output contracts for consistent AI results for the format side of the equation.
How does this apply to real work like reports, meetings and data?
Apply the same three-zone structure whenever your paste exceeds roughly ten pages or one hour of transcript: pin the facts that matter at the top, add an extraction step, and ask at the bottom. It works for board reports, meeting transcripts, spreadsheets exported as text, and multi-email threads alike.
Client meeting prep. You paste a 90-minute transcript and ask what the client committed to. Commitments cluster in the last 15 minutes, which is fine, but the client's budget ceiling was mentioned at minute 40. Put "Budget ceiling mentioned around minute 40: HK$1.2 million" in KEY FACTS, or the model will build a plan around the wrong number.
Quarterly report review. A 60-page PDF with the only revenue restatement buried in a footnote on page 31. Quote the footnote in the top zone and again before the question. Without that, the model will happily reconcile numbers that were superseded.
Data pasted as text. A 2,000-row CSV pasted into chat has a very long middle. Ask for a per-column summary first, then ask your question with the relevant rows re-quoted. Summaries move detail from the weak zone to the strong zone.
Long chat sessions. Instructions you gave 40 messages ago are now mid-window. Restate the standing rules every 15 to 20 turns, or start a fresh chat with the rules at the top. Drift in long conversations is usually lost-in-the-middle wearing a different hat.
What mistakes make the lost-in-the-middle problem worse?
The most common mistakes are pasting more than the task needs, mixing rules and data in one block, asking several questions at once, and trusting a confident answer without checking which passage it came from. Each one increases competition inside the context window and hides the moment the model skipped your key fact.
--- Pasting everything "just in case". More text means more distractors. Chroma's data shows reliability falls as length grows even when the extra text is relevant. Trim to what the question needs.
--- Near-duplicate passages. Two similar definitions in one paste is the classic trap; the model may pick the stale one because it sat nearer an edge. Delete superseded versions before pasting.
--- Rules buried after the document. If your formatting rules come after 30 pages of paste, they are in the middle from the model's point of view. Rules go at the top, format goes at the bottom, never in between.
--- Skipping the extraction step to save time. The quoted-sentence step costs a few seconds and is the only visible proof that the model looked at the right place. Keep it.
--- Confusing this with hallucination. Hallucination invents facts. Lost-in-the-middle ignores a fact that is present. The fixes differ: grounding helps hallucination, placement and extraction help this.
One caveat in the other direction: for short prompts under a page or two, none of this matters. Models handle a few thousand tokens evenly. Reserve the full three-zone structure for long inputs.
How can you test your own AI tool for the effect in ten minutes?
Take one fact the model should get right, hide it in three positions inside the same long document, near the start, in the middle and near the end, then ask the identical question three times. If the middle run is wrong or vaguer, you have measured the effect on your own tool and your own documents.
A ten-minute version you can run now:
--- Pick a long document you know well, around 20 pages.
--- Insert one invented but plausible sentence, such as "The project code name is Harbour Lantern", at page 2. Ask "What is the project code name?" Note the answer.
--- Start a fresh chat, move the sentence to page 10, ask again. Then page 19.
--- Repeat the page 10 run once more with the EXTRACTION STEP added and the question at the very bottom.
Most people see the middle run fail or hedge, and the extraction run recover. That is your evidence, on your model, on your content, with no benchmark paper required. The habit that follows is simple: whenever you paste long, place deliberately and extract before you ask.
The practitioners pulling ahead this year are not the ones with the largest context windows. They are the ones who know AI does not read the way people do, and who structure their inputs so its cold spots never hold the decisive fact. We know AI's cold edges. We know your real challenges. 28 years with UD, turning technology into a partnership with warmth.
Reviewed by the UD AI team.
See how your AI-assisted work stacks up against an AI teammate
You now know how to make a model read what matters. The next step is building that into a repeatable workflow, and seeing where an AI colleague genuinely beats or trails a human one. UD's free AI Battle Staff puts human roles and AI staff head to head across 20 real business arenas, and we'll walk you through every step of turning the results into a working setup for your team.
