Module 01 · Lesson 8 of 13Free
Context windows: what fits and what gets lost
- Level
- Beginner
- Topic
- AI engineering fundamentals
- Reading time
- 7 min
When you finish this page you can
- List what counts toward a model call’s context window, thinking and the answer included.
- Work out whether a request fits and what it costs per call and per day, including Claude Haiku 5.5’s price step at 100,000 tokens.
- Choose what to put in a request by weighing its cost against the risk of burying the detail that matters.
- Say what the API does when a request is too big, and why a long chat grows toward the limit.
Builds on 2 lessons
Start with lesson 315 minVideo lesson · coming soon
The summary missed the pattern
Deskora is a help desk app. A shop’s support team uses it to answer customer emails from one shared inbox. Each customer problem becomes a ticket: one customer’s conversation with the support team. Deskora writes a short summary at the top of long tickets.
Priya, a support agent at a kitchenware shop, opens a ticket from Chidi. The summary says his parcel is late and he wants a refund. It misses that this is his fourth late parcel in three months: the model saw only this ticket.
Tomás, the Deskora engineer who builds the AI features, wants the call to also include Chidi’s last five tickets and the shop’s refund policy. Will that fit in one call, and what will every summary cost?
Everything in one call shares one window
The context window is all the text a model can use to write one response, including the response itself 1. Everything in the request counts: the system prompt, every message and any document you paste in. What comes back counts too: the model’s thinking and its visible answer.
As of October 2026, Claude Haiku 5.5, Sonnet 5.5, Opus 5.5 and Fable 5.1 all have a 1M-token context window. One call can produce up to 128K output tokens 1. Tokens shows how text becomes tokens.
Tomás’s planned request for a typical ticket, with sizes estimated for this lesson’s scenario:
| Block | Tokens (estimate) | Counts as |
|---|---|---|
| System prompt | 150 | Input |
| This ticket (about 20 messages) | 4,000 | Input |
| Chidi’s last five tickets, about 4,000 each | 20,000 | Input |
| Refund policy | 2,000 | Input |
| Thinking (assumed) | 300 | Output |
| Summary | 80 | Output |
| Total | 26,530 |
How much of Claude Haiku 5.5’s 1M-token window does this request fill?Show answer
Diagram not drawn yet: deskora-summary-context-window.svg
Top: one long bar for Claude Haiku 5.5’s 1M-token context window. A thin slice at the left, 26,530 tokens or 2.7%, is filled; the rest, about 973,000 tokens, is empty. A marker at 100,000 tokens reads “Haiku 5.5 price step”. Below: the filled slice enlarged into six blocks, left to right. Input: system prompt 150, this ticket 4,000, five past tickets 20,000, refund policy 2,000. Output: thinking 300, summary 80.
It fits easily, and you pay for all of it
A common belief is that a big window means you can stop choosing what to send. But the window only decides whether a call is allowed. Deskora pays for every token on every summary.
As of October 2026, Claude Haiku 5.5 costs $0.10 per million input tokens and $0.50 per million output tokens, for prompts up to 100,000 tokens 2. Thinking is billed at the output price. One summary is one call; retries are left out.
The formula
cost = input tokens × input price + (thinking + summary tokens) × output price
Prices are per million tokens, so divide by 1,000,000.
Typical ticket, with the extras
26,150 × $0.10 ÷ 1,000,000 + 380 × $0.50 ÷ 1,000,000 = $0.002615 + $0.00019 = about $0.0028
About 4.6 times today’s $0.0006.
Long ticket and long history
92,150 × $0.10 ÷ 1,000,000 + 900 × $0.50 ÷ 1,000,000 = $0.009215 + $0.00045 = about $0.0097
Five past tickets as long as this 15,000-token one. Assumed: 800 thinking tokens, a 100-token summary.
One day
30,000 × $0.002805 + 10,000 × $0.009665 = $84.15 + $96.65 = about $181
Assumed: 40,000 summaries a day, one in four long. Today: about $38 a day.
The thinking amounts are assumptions. usage.output_tokens_details.thinking_tokens reports the real ones.
The long case sits near a price step. On Claude Haiku 5.5, a prompt over 100,000 tokens pays $0.50 and $2.50 per million for the whole request 2. With 8,000 more tokens of history, the same summary costs about $0.052, five times more.
Prompt caching would make the repeated system prompt and policy cheaper; Estimating the cost and latency of a feature adds it and the other cost levers.
More text can bury the detail
Cost is not the only problem. Anthropic’s documentation says accuracy and recall degrade as the token count grows, and calls this context rot 1. Attention explains why.
For Chidi, the useful fact is small: four late parcels in three months. Five full tickets hide it among 20,000 tokens of greetings and tracking numbers. The docs don’t say how much accuracy drops at this size; only tests on Deskora’s own tickets can show that.
What should the summary call include from Chidi’s past tickets?
Nothing: only this ticket, as today.
Cheapest, about $0.0006 a summary. Priya treats Chidi as a first complaint.
The five full tickets and the whole policy.
Every detail is there, at about $181 a day. The useful fact sits among 20,000 tokens of old messages.
One line per past ticket (subject, status, date closed), plus the whole policy.
About $0.00084 a typical summary, $47 a day. The pattern sits in five short lines. Details only inside old messages are lost.
Why this choice
The pattern matters, not the wording. One line per ticket, about 60 tokens each (an estimate), carries it in 300 tokens instead of 20,000. Anthropic advises sending the smallest set of tokens that carries the signal you need 3. The policy stays whole: it is short, and refund rules depend on exact wording.
When the choice changes. If Priya needs what an agent promised in an old ticket, one line is not enough. Then the question is which old messages to send; Long context or RAG compares sending everything with retrieving parts. Either way, test summaries on real tickets first, as Evals shows.
When the window does run out
Deskora’s summaries never get near 1M tokens, but other requests can. The docs describe two cases 1:
| The request | What the API does |
|---|---|
| The input alone is bigger than the window | Rejects it on every model: a 400 invalid_request_error, “prompt is too long” |
The input fits, but input plus max_tokens is bigger than the window |
On Claude 4.5 models and newer, accepts it. If the output reaches the end of the window, it stops with stop_reason: "model_context_window_exceeded" |
Deskora’s check that stop_reason is end_turn already catches the second case.
Chats grow toward the limit because each follow-up resends the whole conversation, as Calling a model API showed. On Haiku 5.5, earlier answers’ thinking stays in it and counts too 1. If each of Priya’s follow-ups adds about 300 tokens (an estimate), she needs over 3,000 questions to fill the window. The cost grows long before that. An agent that adds long tool results at every step fills a window far faster; Context engineering covers what to keep.
Priya still waits for the whole summary before she sees a word of it.
Common mistakes
- Sending everything because it fits. The window is a limit, not a budget to fill. Every token is paid for on every call.
Check your understanding
Tomás doubles the refund policy to 4,000 tokens in the typical request. Does it fit, and what changes?Show answer
A request to Claude Haiku 5.5 has 990,000 input tokens and max_tokens set to 20,000. What happens?Show answer
The long case is 92,150 input tokens. Why does adding one more long past ticket cost far more than its own 15,000 tokens?Show answer
Summary
- Tomás’s bigger request fits easily, at under 3% of the window, but costs about $181 a day instead of $38.
- The context window holds everything in one call, thinking and the answer included. As of October 2026, current Claude models have 1M tokens and up to 128K output.
- Every token you add is paid for on every call; on Haiku 5.5, a prompt over 100,000 tokens costs five times as much.
- Accuracy and recall degrade as context grows, so send the smallest input that carries the detail.
- Too much input gets a 400; output that hits the window’s end stops with
model_context_window_exceeded. Chats grow because every call resends the history.