It says a million tokens, so why did it miss the clause in the middle?
An advertised context window is how much text the model will accept, not how much it will use. Attention thins out in the middle of a prompt, tokens are not pages, and tools truncate quietly. Retrieve less and cite more.
Capacity is not capability
The context window is the maximum number of tokens the model will accept in one request, counting your instruction, the retrieved material and anything the conversation carries forward. It does not tell you how well the model will use what you put in it. Attention across a long input is not uniform. Models are trained on text where the important thing is usually near the beginning or the end, and instruction tuning puts the question and the rules at the end, which strengthens that bias. The practical result, widely reproducible and easy to see on your own bundle, is that a model asked to find a fact in a very long prompt often performs best when the fact sits at the start or the finish, and worst somewhere in the middle. The advertised window tells you nothing about where that soft spot begins. It depends on the model, the task, the amount of similar-looking material around the answer, and how many separate facts you have asked it to track at once. So when a firm tells us a long-context model missed a clause in the middle of a bundle, the answer is usually not that the model is broken. It is that the model was handed a very large amount of material and asked to behave as though all of it mattered equally, and it did not.
Tokens are not pages, and tools truncate quietly
A token is a fragment of a word — for English prose, a rough rule of thumb is three tokens to about four words, but the ratio moves with the material. Numbers, statute references, dates, defined terms, page headers, numbering and the ragged text that comes out of optical character recognition on a scanned bundle all consume tokens faster than clean prose. A bundle that looks like it fits comfortably can be twice the size you estimated. Worse, the truncation is usually silent. If the assembled request exceeds the window, the tool drops material, and the model does not announce that pages 200 to 400 were never there. It answers confidently about what it saw. You then have an output that looks complete, a bundle that appears to be in scope, and no evidence in the review trail that a third of the reading never happened. Ask the tool, or the person who configured it, how many tokens were actually sent for your matter and what happens when the limit is reached. If nobody can answer, that is the finding, and it belongs in your risk register rather than in a hope that the material is small enough.
Why retrieval usually beats the whole bundle
The alternative to a very long prompt is a short one containing only the passages that matter. Retrieve first — by search, by document type, by date range, by cross-reference — then instruct the model to answer only from those passages and to quote them. This improves two things at once. Accuracy goes up, because the model is not asked to hold a hundred competing facts in view while answering a question about one. Verifiability goes up, because a short prompt means the model can quote the passage, and you can find the passage in seconds. A model that cites clause 14.2 of a schedule can be checked; a model that has read everything and asserts a conclusion cannot. Retrieval has its own failure modes, and they are worth knowing before you build. Chunking splits documents at fixed lengths, so a clause can be separated from the defined term it depends on or from the schedule it cross-refers to. Search matches words, so a limitation clause may be retrieved for a liability question while the answer actually sits in the definition of a term used somewhere else. And retrieval tends to over-return: ten passages where two are relevant gives the model eight opportunities to be distracted. Fewer, better passages is the goal.
Measuring effective context on your own matters
You can produce a defensible number for your own documents in a morning. Take one matter you know well and one answer you can verify — the notice period, a break date, the cap on liability. Build prompt variants where the answer sits at the beginning, in the middle and at the end, surrounded by a realistic amount of other material from the same bundle. Ask for the answer with a verbatim quote and a location. Then add a variant where answering needs two passages that are far apart, because multi-hop questions fail earlier than single-fact ones, and a variant with near-identical distractors, which is what a real bundle supplies. Run each variant two or three times. What you are looking for is not a headline figure but a shape: where quality starts to fall away as the prompt grows, and how much of the fall is about length rather than about the material being genuinely ambiguous. Repeat the exercise when you change model version or quantisation. Then write the result down in two sentences — this model, on bundles of this kind, is reliable up to about this much retrieved material — and use it to set the retrieval defaults rather than trusting the marketed window.
What goes in the file note
The record should say which documents the tool actually saw, not which documents were in the folder. Those are different facts and only one of them is relevant to whether the output can be relied on. The distinction bites hardest on assertions of absence. If a note says the bundle contains no restriction on assignment, that statement is an implicit claim about the completeness of what was read. If the model saw six retrieved passages and the tool searched only the body of each document, the assertion is not supported and should not appear in the file as though it were. Write it as the narrower thing it is: no restriction on assignment was identified in the passages searched. That is a finding a reader can act on, and it is honest about its own limits. Two smaller habits are worth building. Quote locations from the document, not from the model — a model-supplied page number is a suggestion until you have checked it. And where a summary was produced by condensing, keep the intermediate summary with the matter while it is live, so a later reader can see what survived the condensing and what was dropped along the way.
What to watch
- A tool that truncates a long bundle without saying so and answers confidently about the pages it never read
- Asserting the bundle contains no restriction on assignment when the model only saw six retrieved passages
- Fixed-length chunking that separates a clause from its defined terms or from the schedule it cross-refers to
- Trusting a long-context claim measured on tidy prose rather than on a scanned, numbered, cross-referenced bundle
- Blaming the model for missing a clause when the real fault is retrieval that returned eight passages of which two were relevant
- Copying a page number out of the model's answer instead of out of the document
Stop treating the bundle as the unit of work. Set retrieval defaults so the model receives a small, reviewed set of passages rather than everything available, and make the retrieved set visible to the fee-earner before the answer is generated. Ask your tool provider, in writing, what happens when the request limit is reached. Then run the position test — same answer, placed at the start, middle and end of a realistic prompt — on one matter you know, and write the result into the configuration register so it governs how much material goes into a request. Require assertions of absence in file notes to be framed as what was searched, not what exists.