Context Window Management for Long-Running Agents

📉Why long-horizon agents degrade

Three mechanisms compound. First, attention dilution: as the window fills, the model ability to pick out the instruction that matters from the mass of history degrades, with content in the middle of long contexts attended to measurably less than content at the start and end. This is not speculation — lost-in-the-middle effects are documented across model families, and while newer long-context models have narrowed the gap, as of writing none has eliminated it. Second, noise accumulation: an agent transcript is mostly tool output, and tool output is mostly irrelevant to later steps. Each step pays attention over all of it, so signal density drops monotonically as the run lengthens.

Third, and least discussed: self-reinforcement. The agent reads its own prior reasoning as evidence. An early wrong hypothesis, once written into the transcript, is re-attended to on every subsequent step, and models are more persuaded by fluent text in context than by contradictory tool output that arrives later. Long horizons turn small early errors into entrenched positions.

The practical symptom pattern: the agent does fine for the first stretch of the run, then begins repeating actions it already took, forgetting constraints stated early ("do not modify the billing module" gets silently violated at step forty), and producing summaries of progress that contradict the actual tool results. Teams misread this as a model quality problem. It is a context hygiene problem, and it responds to engineering, not to hoping the next model is better.

The mental model to hold: the context window is working memory under load. Everything you put in it competes with everything else for attention, and every token you leave in it is paid for — in quality as well as dollars — on every subsequent step.

Context window, defined in the glossary→

🗜️Compaction and summarization strategies

Compaction — replacing older transcript with a summary — is the standard answer and the standard disappointment, because naive compaction loses exactly the details the later steps need. The failure mode is familiar to anyone who has watched it: the summary says "explored the auth module and fixed the issue" and the agent, no longer able to see what was tried, tries it again. Summaries written for human readers are lossy in the wrong places for agents.

The fix is to summarize for action, not for narrative. A working compaction schema carries: the current goal and its status; decisions made and the reasoning behind them (so they are not re-litigated); a negative log — approaches tried and rejected, with why; open questions and blockers; and pointers to durable state (file paths, ticket IDs, row keys) so details can be re-fetched rather than remembered. The negative log is the highest-value field and the one generic summarizers always drop.

Trigger policy matters as much as schema. Time-based or "when we hit 80 percent of the window" triggers compact at arbitrary points, often mid-subtask. Event-based triggers — compact at subtask boundaries, after a milestone commit, when a phase closes — produce summaries that align with the task structure and are far more recoverable. Hybrid triggers (compact at the next boundary once a size threshold passes) are the production default we recommend.

Keep the originals. Compaction should move the raw transcript to cheap storage, not delete it. When the summary proves lossy — it will — the agent (or a debugging human) can re-fetch the original segment by pointer. A compaction system without a re-hydration path is an information shredder with a nice interface.

🧩Subagent context isolation

The single most effective context-management technique is not compaction — it is never putting the bulk in the window in the first place. Subagent isolation delegates context-heavy subtasks to a fresh agent with a clean window: "read these thirty files and report the three things relevant to the migration" costs the parent agent a short brief and a short report instead of thirty files of transcript. The parent context stays small, dense, and focused on the plan.

This is why the orchestrator-workers pattern and context management are the same conversation. The lead context holds the task state; worker contexts are disposable scratch space. Search, exploration, log analysis, document review — anything read-heavy with a small output — is a subagent-shaped task. The ratio to watch is read-to-report: a subtask that consumes 50,000 tokens and returns 500 is a 100-to-1 context compression, and stacking a few of those is how multi-hour runs stay coherent.

The discipline is in the interface. The brief must carry everything the subagent needs — it cannot see the parent history — which forces the useful exercise of making assumptions explicit. The report must be structured and complete, because the parent will act on it without the underlying evidence. And the parent must treat the report as a claim to be checked at integration points, not as ground truth: subagents hallucinate too, and isolation means the parent has no way to notice except by verifying at the seams.

A note on depth: one level of isolation is robust; nested subagents work but each level multiplies the brief-quality problem. If you find yourself three levels deep, the task wants a deterministic decomposition, not a deeper tree.

🗄️Structured state outside the window

The strongest version of context management is refusing to use the context as storage at all. Anything the agent must remember reliably — the task plan, the list of files already modified, decisions and their rationale, counters, intermediate results — belongs in structured state outside the window: a state object the orchestrator maintains, a scratchpad file, a database row. The context then holds only what the current step needs, assembled fresh each step from that state.

This inverts the usual architecture. Instead of an ever-growing transcript that the model passively re-reads, each step is a constructed prompt: system instructions, a compact rendering of current state (plan with completed items checked off), the immediate working set, and the most recent tool results. The transcript becomes a log for humans and debugging, not the agent working memory. Frameworks with checkpointed state encourage this; a raw conversation loop fights it.

The benefits compound. Cost drops because you stop re-sending history. Reliability rises because state is exact — "files modified: 7 of 12" rendered from a list is not something the model can misremember the way it misremembers its own summary. Recovery becomes possible: a crashed run resumes from the state object, not from re-reading a 200KB transcript and hoping. And evals get easier, because state diffs between steps are inspectable artifacts.

The cost is upfront design: someone must decide what the state schema is for each agent type, and keep the renderers (state-to-prompt and output-to-state) honest. That schema design is the actual engineering of long-horizon agents. The prompt is the easy part.

🪙Token budgeting per step

Treat context as a budgeted resource with line items, the way you would treat memory in an embedded system. A workable budget for a long-horizon agent step: system instructions and policies (fixed, keep them lean — bloated system prompts are silent context debt), a compact state rendering, the working set for this step, and reserved headroom for tool results and the model output. When any line item overruns, something gets evicted by policy, not by truncation roulette.

Tool results are where budgets go to die. A single unbounded file read or API response can exceed everything else in the step combined. Production agents wrap every tool with output limits: truncated to a budget with a pointer to the full content, head/tail sampling for logs, schema-filtered fields for API responses. The model should receive the result it needs, not the result the tool happened to produce — and "the result it needs" is a design decision you make in the tool wrapper, once, rather than hoping the model copes, forever.

Model choice interacts with the budget in a way teams miss: bigger advertised windows do not relax the budget, because the degradation curve, not the hard limit, is the binding constraint. A step that is reliable at 40K tokens of context does not become reliable at 300K because the model technically accepts it. Budget to the reliable region of the curve — which you measure, per model, on your tasks — not to the number on the pricing page.

Budget accounting also gives you the cheapest reliability signal available: context size per step over a run. A run whose context grows monotonically and never sheds is a run that will degrade; watching that curve in production tells you which tools and subtasks need wrappers or isolation before users notice.

🌡️Measuring context rot instead of discovering it

Context rot is measurable, and if you are not measuring it you are discovering it in production. The core instrument is a length-stratified eval: run the same task suite with artificially varied context lengths (pad with realistic filler — past tool output, earlier subtasks) and plot task success against context size. Every model you use has a curve; they differ more than vendor pages suggest, and the curve on your task distribution is the only one that matters. Re-run it when you change models, prompts, or tool wrappers — as of writing, model updates can and do shift it.

In production, the proxies are cheap to log: context size per step, re-fetch rate (how often the agent re-reads something it already saw — a direct signal that information fell out of effective attention), repeated-action rate, and constraint-violation rate against early instructions. A rising re-fetch rate is context rot announcing itself before quality metrics move.

Two traps in measurement. First, average context size hides the failure: rot is a tail phenomenon, so track the p95 and max, not the mean. Second, synthetic padding underestimates real rot, because real accumulated context is not neutral filler — it is semantically related to the task and competes for attention harder. Use transcripts from real earlier runs as padding once you have them.

Close the loop: the degradation curve should drive the budgets and triggers from the previous sections. "Compact at the next boundary past X tokens" where X came from your own curve is engineering. X from a blog post (including this one) is a starting guess.

🧱The honest limits

Everything above mitigates; nothing abolishes. Summarization is lossy, and on long enough horizons the losses compound — hierarchical compaction over a very long run drifts the way a story retold drifts. Subagent isolation multiplies token spend on re-establishing context and introduces integration risk at every report seam. External state schemas capture what you designed them to capture, and novel situations produce state the schema has no field for, which then lives nowhere.

There is also a hard trade between context hygiene and serendipity. Agents occasionally succeed by noticing an irrelevant-looking detail from twenty steps ago — exactly the detail a good compaction policy evicts. Tighten the window and you trade away a real, if unquantifiable, source of capability. The right response is task-dependent: exploration-heavy research agents tolerate rot differently than a transactional ops agent, and one policy will not serve both.

Model progress keeps moving the frontier — long-context attention quality has improved generation over generation, and as of writing it is reasonable to expect continued improvement, though you should verify the current state before betting a roadmap on it. What does not change with model progress is the economics: attention over more tokens costs more money and more latency per step, forever. Context discipline is cost discipline even in the limit where quality stops being the binding constraint.

The end-state architecture we converge on for long-horizon production agents: small parent context holding structured state, subagents for read-heavy work, event-triggered structured compaction with re-fetch pointers, per-step token budgets with wrapped tools, and a length-stratified eval suite that re-runs on every model or prompt change. Unglamorous, measurable, and — unlike a bigger window — actually under your control.

RAG vs long-context: the adjacent trade-off→What long-running agents cost per task→Talk to us about long-horizon agent builds→