Context Engineering for Long-Horizon Agents

TL;DR

Compaction fixes one of the four ways a long-running agent's context degrades, and makes a second one worse. Treating context engineering as a single lever, usually summarization, is why teams patch one failure and reintroduce another.

Context poisoning, distraction, confusion, and clash are different mechanisms. Compaction is the fix for distraction. Run it on poisoning and the wrong fact survives into the summary with its provenance stripped, which is worse than leaving it alone. One 2026 study found a single summarization event flips a correct answer to a wrong one in roughly 40% of the transitions it touches, even though it helps on net.

None of this argues against long context or claims a hard cliff at some token count. Utilization degrades gradually, well before any window fills, and the fix depends on which failure you actually have.

We think the response is mechanical: diagnose the failure mode first, clear tool results before anything else, and keep the task statement, the authorization record, and the ledger of actions already taken outside anything a summarizer can touch.

Overview

An incident-response agent gets paged at 02:14. It reads the alert, queries three log sources, pulls metrics for the affected service, checks the last four deploys, opens the runbook, correlates a latency spike with a config change, drafts a hypothesis, asks for approval, rolls back the deploy, and then keeps watching for twenty minutes to confirm recovery. By turn sixty its context window holds the original page, four thousand lines of log output, two metrics dumps, a runbook, an approval record, and a rollback confirmation. Somewhere in there is the one fact that matters: the rollback already happened, so do not roll back again.

Nothing above is a fill-level problem. Every one of those items fits easily inside a modern window. What breaks is which ones the model actually weighs when it decides the next action, and in what order they arrived. An agent correct for forty turns can start producing confident, well-formatted, wrong answers at turn seventy with tokens to spare, and the four distinct ways that happens each need a different repair.

This is a different subject from the existing post on this site about building agent memory without letting it drift or corrupt itself. That one is about persistent memory integrity: what an agent stores across sessions and how to keep it from poisoning itself over weeks. This one is about what occupies the window during a single run and how that degrades. They interact, but the fixes are different, and conflating them is why teams add a memory store and are surprised when long-run behavior does not improve.

The Attention Budget

Anthropic's framing is the clearest available and worth quoting directly. Context engineering is "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference," and the resource being managed is attention: "Like humans, who have limited working memory capacity, LLMs have an 'attention budget' that they draw on when parsing large volumes of context. Every new token introduced depletes this budget by some amount."

There is a mechanical reason. Transformers let "every token to attend to every other token across the entire context," which "results in n² pairwise relationships for n tokens," so "as its context length increases, a model's ability to capture these pairwise relationships gets stretched thin." There is also a training-distribution reason: models "have less experience with, and fewer specialized parameters for, context-wide dependencies," because short sequences dominate pretraining data. The result is "a performance gradient rather than a hard cliff."

The empirical support for that gradient comes from three directions.

Position matters, and the middle is worst. Liu et al.'s Lost in the Middle (arXiv:2307.03172, TACL) established the U-shaped curve: performance "is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades when models must access relevant information in the middle of long contexts, even for explicitly long-context models." Concretely, with 20 documents, GPT-3.5-Turbo scored 75.8% when the answer was in the first position, 53.8% in the tenth, and 63.2% in the twentieth. A 22-point trough. Their blunter finding is that "extended-context models are not necessarily better at using their input context."

Long-context models do not fix it. NoLiMa (arXiv:2502.05167) removed the lexical shortcut that needle-in-a-haystack tests accidentally provide, requiring models to "infer latent associations" rather than match words. Across 13 models claiming at least 128K context: "At 32K, for instance, 11 models drop below 50% of their strong short-length baselines. Even GPT-4o, one of the top-performing exceptions, experiences a reduction from an almost-perfect baseline of 99.3% to 69.7%." Databricks measured the same shape on retrieval-augmented generation and found Llama-3.1-405B performance beginning to decrease after 32K tokens and GPT-4-0125-preview after 64K.

"Context rot" is the general name. Chroma's report tested 18 models across 194,480 LLM calls and concluded that "models do not use their context uniformly; instead, their performance grows increasingly unreliable as input length grows." Their most counterintuitive result deserves attention from anyone building retrieval: "models perform better on shuffled haystacks than on logically structured ones." Preserving a coherent narrative in the surrounding text made recall worse, consistently across all 18 models. Their conclusion is the sentence to carry: "Whether relevant information is present in a model's context is not all that matters; what matters more is how that information is presented."

Note what this evidence does not say. It does not say long contexts are useless, or that there is a cliff at some token count. It says utilization degrades gradually and non-uniformly, that the degradation is invisible on lexical-match benchmarks, and that the same information placed differently produces different answers.

Four Failure Modes, Four Different Fixes

We think Drew Breunig's taxonomy is the most useful decomposition in circulation, and the reason it matters is that teams routinely apply one fix (compaction) to all four problems and only solve one of them. His four:

• Context poisoning: "when a hallucination or other error makes it into the context, where it is repeatedly referenced." The incident agent misidentifies the service owner at turn nine and every subsequent turn treats that as established fact.
• Context distraction: "when a context grows so long that the model over-focuses on the context, neglecting what it learned during training." The Gemini 2.5 technical report describes the symptom: beyond roughly 100K tokens the agent "showed a tendency toward favoring repeating actions from its vast history rather than synthesizing novel plans." The report itself calls this observation anecdotal, which is the right level of confidence to hold it at, but it matches what practitioners report: the agent starts re-running the query it already ran.
• Context confusion: "when superfluous content in the context is used by the model to generate a low-quality response." The dominant instance is too many tool definitions.
• Context clash: "when you accrue new information and tools in your context that conflicts with other information in the context." Two log sources disagree about the deploy time; both are in the window; the agent reasons over both.

Figure 1 — Four failure modes and the levers that actually address each. Compaction is the fix for distraction; applied to poisoning it makes things worse, because the wrong fact survives into the summary with its provenance stripped.

On confusion, the evidence is real but conditional, and precision matters because the popular version of the claim is overstated. The cleanest controlled measurement is MCP-Zero (arXiv:2506.01056): holding model, tasks and method fixed and changing only the size of the available tool pool, Claude 3.5's schema-injection accuracy fell from 97.60 to 69.23 on single-turn tasks and from 100.00 to 60.22 on multi-turn when moving from a curated domain subset to a pool roughly forty times larger, with average tokens rising from 312 to 6,308. Vendor guidance agrees on the shape: Anthropic's documentation states that "Claude's ability to pick the right tool degrades once you exceed 30–50 available tools," and OpenAI suggests aiming for "fewer than 20 functions available at the start of a turn at any one time, though this is just a soft suggestion."

But the effect is not universal. Toolshed (arXiv:2410.14594) ran a 1-to-128 tool ablation and found the tool count "is not statistically significant in predicting the Simple Agent's accuracy," with accuracy holding at 97–100% throughout; the degradation they measured was in the retriever, not the model. LiveMCPBench (arXiv:2508.01780) agrees at larger scale (retrieval errors account for nearly half of all failures) and its own ablation cuts against naive minimalism: dropping from five retrieved tools to one moved success from 78.95% to 64.21%.

The honest synthesis: tool confusion bites hardest with small or quantized models, pool sizes in the hundreds, and semantically overlapping names. Anthropic notes that "the most common failures are wrong tool selection and incorrect parameters, especially when tools have similar names like notification-send-user vs. notification-send-channel." It is largely absent with a capable model, a disjoint tool set, and a few dozen tools. Measure it on your own tool set rather than assuming either direction; we would not trust either finding to transfer to a different model or a different pool.

On clash, the strongest evidence is about turn structure rather than volume. LLMs Get Lost In Multi-Turn Conversation (arXiv:2505.06120) took single-turn task instructions and sharded them across multiple turns without changing the information content. Across six generation tasks and 200,000+ simulated conversations, every model tested degraded, "with an average drop of 39%." The decomposition is the useful part: "Model aptitude degrades in a non-significant way… with an average drop of 16%. On the other hand, unreliability skyrockets with an average increase of 112%." o3 retained 98.1% of its single-turn performance when the shards were merely concatenated but only 64.1% when they arrived across turns. Identical information, different distribution, thirty-four points of difference.

The Levers

Compaction and context editing. Compaction is "taking a conversation nearing the context window limit, summarizing its contents, and reinitiating a new context window with the summary." Anthropic reports concrete numbers for the productized form: on "an internal evaluation set for agentic search," "combining the memory tool with context editing improved performance by 39% over baseline. Context editing alone delivered a 29% improvement." And separately, "in a 100-turn web search evaluation, context editing enabled agents to complete workflows that would otherwise fail due to context exhaustion — while reducing token consumption by 84%." Those are two different unpublished evaluations, and neither results sentence names the model, though the announcement is a Claude Sonnet 4.5 launch post. Treat them as vendor-reported magnitudes rather than measurements you can reproduce.

Tool result clearing is the sub-case we'd adopt first. Anthropic calls it "one of the safest lightest touch forms of compaction," and the reason is that a tool result you already extracted the answer from is pure overhead. The four thousand lines of log output can be replaced by the three lines that mattered, with a pointer to where the rest lives.

Subagent isolation. Rather than one window accumulating everything, a lead agent delegates investigation to subagents that each work in their own window and return only conclusions. Anthropic's context-engineering post puts numbers on the interface: "Each subagent might explore extensively, using tens of thousands of tokens or more, but returns only a condensed, distilled summary of its work (often 1,000-2,000 tokens)." Their separate write-up of the multi-agent research system reports a configuration "with Claude Opus 4 as the lead agent and Claude Sonnet 4 subagents outperformed single-agent Claude Opus 4 by 90.2% on our internal research eval." Two caveats from that post keep this honest. The cost is steep: "multi-agent systems use about 15× more tokens than chats," and "for economic viability, multi-agent systems require tasks where the value of the task is high enough to pay for the increased performance." And the mechanism may be simpler than it looks. Anthropic reports that on BrowseComp, "token usage by itself explains 80% of the variance."

lead lead -> metrics subagent (40k local) --1-2k summary--> lead lead -> deploys subagent (30k local) --1-2k summary--> lead The exploration cost moves off the lead's window; coordination fidelity is what you pay. -->

Figure 2 — Single-agent accumulation versus subagent isolation. The lead's window stays roughly flat because the expensive exploration happens in windows that are discarded.

Externalize state instead of holding it. Anthropic's "structured note-taking" is an agent "regularly writ[ing] notes persisted to memory outside of the context window," pulled back in when needed. For the incident agent the natural artifact is a running investigation file: hypothesis, evidence for and against, actions taken, outstanding questions. That file is durable across compaction events, diffable, and reviewable by the on-call human. That makes it an observability artifact as well as a context-management one.

Retrieval instead of preloading, and progressive tool disclosure. Rather than putting every tool definition and every possibly-relevant document in the window at turn one, let the agent discover what it needs. Anthropic reports that a five-server MCP setup "can consume ~55k tokens in definitions before Claude does any work" and that they have "seen tool definitions consume 134K tokens before optimization," with a tool-search approach "typically reduc[ing] this by over 85 percent, loading only the 3–5 tools Claude needs for a given request." Their vendor-reported accuracy effect is substantial. Opus 4 moved from 49% to 74% and Opus 4.5 from 79.5% to 88.1% on internal MCP evaluations with tool search enabled. Their code-execution-with-MCP variant, where tools are presented as files the agent reads on demand, is described as reducing "token usage from 150,000 tokens to 2,000 tokens — a time and cost saving of 98.7%"; that figure is an illustrative worked example in the post rather than a measured benchmark, and should be cited as such.

The 2026 Frontier

The interesting shift this year is that compaction stopped being a heuristic bolted onto inference and became something models are trained to do, and something that can be validated.

Trained compaction. Scaling Long-Horizon LLM Agent via Context-Folding (arXiv:2510.11967) gives the agent explicit branch and return actions: it forks into a subtask, then "fold[s] it upon completion, collapsing the intermediate steps while retaining a concise summary of the outcome," with the folding behavior trained by reinforcement learning rather than scheduled. With a 32K active budget and at most ten branches it reached pass@1 of 62.0% and 58.0% on its two benchmarks, "surpassing baselines that require a massive 327K context window," while cutting the main trajectory to about 8K tokens. The framing is the contribution: "context folding can be viewed as a learnable summarization mechanism aligned with sub-task boundaries."

CompactionRL (arXiv:2607.05378) puts compaction inside the RL loop, so the summary is an action optimized under the task reward. GLM-4.5-Air reached 66.8% pass@1 on SWE-bench Verified and 24.5% on Terminal-Bench 2.0, gains of 7.0 and 3.1 points, and the authors report deploying it in the RL pipeline for training GLM-5.2. Two of their results are more interesting than the headline. Compaction at an 80K window across four rounds scored 66.8, beating a 160K long-context reference at 64.5. Compaction can be better than having the room, not merely a substitute for it. And it does not generalize for free: "CompactionRL does not consistently improve single-window performance, which is expected because disabling compaction creates a mismatch with its training setting."

Compaction validation. Slipstream (arXiv:2605.08580) identifies the structural problem nobody had addressed: "compaction is therefore inherently a prediction about future relevance made without access to future behavior," and "compaction faithfulness has no synchronous ground truth." Their fix is to run the compactor asynchronously while the agent keeps going on uncompacted context, then judge the candidate summary against that independently generated continuation before adopting it. This improves accuracy by up to 8.8 percentage points while reducing end-to-end latency by up to 39.7%. Two ablations matter for anyone implementing it: the gain comes from validation rather than asynchrony, and the judge does not need to be strong: "trajectory-grounded validation does not require a judge as capable as the agent."

Self-compacting agents. Self-Compacting Language Model Agents (arXiv:2606.23525) asks whether an agent can decide when to compact, with no training: a compaction tool plus a rubric of fire and suppress conditions. It matches or beats fixed-interval summarization at 30–70% lower per-question cost, and gains +8.5, +9.2 and +5.3 points over a no-compaction baseline across three models on BrowseComp-Plus. The finding worth internalizing is negative: "although unprompted models cannot reliably tell when their own context is rotting, a lightweight rubric closes this gap." Removing the rubric collapses performance to roughly the level of a naive fixed schedule. Knowing that you should compact is not the same capability as knowing what to keep.

Tradeoffs, and What Never to Compact

Every lever above has a cost, and the 2026 papers are unusually good at quantifying them.

Compaction is lossy, and the loss is measurable. SelfCompact instrumented what happens to correctness across compaction events, tracking answer transitions across the twelve summarization calls of one fixed-interval run, and found that while summarization produces a net gain, "40.4% of all transitions cause degradations": 1,009 cases went from correct to wrong against 1,486 going the other way. An oracle policy that simply suppresses compaction whenever the current answer is already correct outscores a fixed schedule by 11.5 points. Slipstream localizes the mechanism: "omission accounts for roughly 90% of failures," rather than fabrication. Compaction mostly does not lie; it forgets.

Repeated compaction compounds, and almost nobody measures it. A 2026 rate–distortion survey (arXiv:2607.08032) makes the point directly: "while compression is measured carefully on single-turn long context, the repeated compaction that agents actually perform is almost never measured." Their reference experiment contrasts a reversible operator, which "holds recall near 0.95 at every compaction frequency," against an irreversible one that "runs far below it, between 0.33 and 0.56, and is weakest at the highest compaction frequency, because each summary throws away facts the next summary can no longer see and the loss compounds." Their conclusion is the one to design against: "the two look identical under single-turn evaluation and diverge only when compaction repeats."

Compaction buys tokens with latency. ACON (arXiv:2510.00615, ICML 2026) optimizes the compression guideline itself and cuts peak token usage by 26–54%, but its own measurements show the tradeoff plainly: on AppWorld it cut API cost while median wall-clock time per task rose from 73.2 seconds to 87.7 with history compression and 101.9 with observation compression, because the compressor is an extra model call on the critical path. If your agent has a latency budget, that call has to fit inside it.

Subagent isolation costs coordination fidelity and tokens. A 1,000-token summary is a lossy channel between the subagent that did the work and the lead that acts on it. Nuance that did not make the summary is unrecoverable without re-running the subagent, and the 15× token multiplier is real. The right test is whether the subtask has a clean interface: "which deploys touched this service in the last six hours" does; "figure out what is wrong" does not.

Aggressive pruning breaks error recovery. If the agent's second hypothesis fails and the evidence that motivated the first was pruned three compactions ago, it cannot backtrack; it can only start over. This is the specific interaction between context engineering and the compounding-reliability arithmetic covered separately on this site: pruning raises per-step reliability while removing the ability to recover from a failed step.

We think the most consequential loss is a governance one, and there is now a preprint on it. Governance Decay (arXiv:2606.22528) measured what happens to a policy constraint across compaction over 1,323 episodes. With the policy in full context no model ever violated it; a single compaction step raised the pooled violation rate to 30%, and to 59% on the worst models. The decisive variable is survival: "when the constraint survives the summary violation is 0%, when dropped 38%." Tightening the summary budget from 300 words to 15 dropped constraint survival from 88% to 23%, and violations rose from 0% to 78% across four successive compaction rounds. It is a single-author preprint and should be read as suggestive rather than settled, but the mechanism is mechanical and the author's framing is hard to argue with: compacting a policy "can be worse than never stating it, because the summary normalizes the pending task while discarding the rule."

That last point generalizes into a design rule. Anything whose loss is unrecoverable should not be entrusted to a summarizer. Anything whose loss merely costs a re-query (a log page, a metrics dump, a document you can fetch again) is exactly what compaction is for. Sort your context by that criterion before choosing a compaction strategy, and most of the hard decisions make themselves.

Conclusion

Long-context models did not solve context management; they raised the ceiling while leaving the gradient intact. Utilization degrades as sequences grow, the middle of a long window is the worst place for a critical fact, and the same information distributed across turns rather than delivered at once costs real accuracy. Those are measured effects, not folklore.

The practical program, in our view, is to treat the window as a budget with named line items. Diagnose which of the four failures you actually have, because compaction fixes distraction and makes poisoning worse. Clear tool results early. It is the cheapest safe win. Isolate exploration in subagents where the subtask has a clean interface, and accept the token bill. Externalize durable state to a file a human can read. Disclose tools progressively instead of preloading them. And pin the task statement, the authorization context, and the record of irreversible actions somewhere no summarizer can reach.

Sources

• Effective context engineering for AI agents — Anthropic, Sep 2025; the attention-budget framing, context rot, compaction, structured note-taking, and the 1,000–2,000-token subagent summary.
• Managing context on the Claude Developer Platform — Anthropic, Sep 2025; context editing at 29%, 39% with the memory tool, and 84% token reduction on a 100-turn web-search evaluation.
• How we built our multi-agent research system — Anthropic, Jun 2025; the 90.2% internal-eval improvement, the 15× token cost, and token usage explaining 80% of BrowseComp variance.
• Context Rot: How Increasing Input Tokens Impacts LLM Performance — Chroma Research; 18 models, 194,480 LLM calls, and the shuffled-haystack result.
• NoLiMa: Long-Context Evaluation Beyond Literal Matching and Long Context RAG Performance of LLMs — degradation without lexical shortcuts (11 of 13 models below half their baseline at 32K; GPT-4o 99.3% → 69.7%) and the RAG-side onset points.
• Lost in the Middle: How Language Models Use Long Contexts — Liu et al., TACL; the U-shaped position curve and the 20-document GPT-3.5-Turbo numbers.
• How Long Contexts Fail — Drew Breunig; the poisoning / distraction / confusion / clash taxonomy.
• LLMs Get Lost In Multi-Turn Conversation — a 39% average drop from sharding identical instructions across turns, decomposed into 16% aptitude and 112% unreliability.
• MCP-Zero: Active Tool Discovery for Autonomous LLM Agents — the controlled tool-pool ablation: 97.60 → 69.23 single-turn and 100.00 → 60.22 multi-turn.
• Toolshed and LiveMCPBench — the counter-evidence: no significant effect from 1 to 128 tools, and retrieval rather than the model as the dominant failure source.
• Advanced tool use, Tool Search Tool and OpenAI's function calling guide — vendor guidance on tool-count thresholds and definition-token reduction.
• Code execution with MCP — Anthropic; progressive disclosure and the 150,000-to-2,000-token worked example.
• Scaling Long-Horizon LLM Agent via Context-Folding and CompactionRL — trained compaction: branch/return folding under FoldGRPO, and compaction as an RL action reaching 66.8% on SWE-bench Verified.
• Slipstream, Self-Compacting Language Model Agents and ACON — compaction validation, compaction timing, and compression-guideline optimization, with their measured costs.
• What to Keep, What to Forget: A Rate–Distortion View of Memory Compaction and Governance Decay — repeated compaction as the unmeasured axis, and constraint survival falling from 88% to 23% as the summary budget tightens.