Agent Tool Output Compression: 5 Proven Tactics (2026)

TL;DR: Agent tool output compression reduces the size of data returned by tools before it enters an LLM agent’s context window. Raw tool outputs are the single biggest driver of token costs and reasoning degradation in long-running agents, because every uncompressed result stays in context and gets re-sent on every subsequent turn. Compression approaches range from query-aware extraction to rule-based filtering to simple placeholder offloading, each with different tradeoffs between fidelity and savings.

Key Takeaway: What is Agent Tool Output Compression?

Agent Tool Output Compression is a context engineering technique that strips noise, duplicates, or non-essential data from tool execution results before they enter an LLM’s conversation history.

• The Core Problem: LLM agents re-read history on every turn. Uncompressed tool outputs compound token costs exponentially and cause context rot (reasoning accuracy drops up to 90%+ in severe cases).

• The Solution: Applying techniques like query-aware extraction, deduplication references, and reversible offloading achieves 50%–80%+ average context reduction while maintaining high agent fidelity.

What Is Agent Tool Output Compression?

Agent tool output compression is the practice of shrinking the data a tool returns before that data enters (or re-enters) an LLM agent’s context window. The goal is straightforward: keep the information the agent actually needs to act on, and discard everything else.

A concrete example makes this clear. An agent calls a web search tool. The tool returns 15,000 tokens of raw HTML, navigation elements, ads, and repeated boilerplate. The agent only needs two paragraphs from that page to answer the user’s question. Without compression, all 15,000 tokens sit in context for the rest of the session. With compression, only the relevant 2,000 tokens enter the conversation.

Agent tool output compression is a specific application of the broader discipline of context compression. What makes tool outputs special is their size, their frequency, and the fact that agents generate them continuously throughout a session.

Get started with Compresr’s quick-start guide to see tool output compression in action.

Why Tool Outputs Are the Biggest Token Problem in Agents

The Append-Only Trap

Most agent frameworks use an append-only message history. When an agent calls a tool, the full result becomes a message in the conversation. That message travels with every subsequent LLM call for the rest of the session. A single 15,000-token search result doesn’t cost you 15,000 tokens once. It costs you 15,000 tokens on every turn that follows.

One practitioner building an SRE agent described finding a single tool output from a Python file search that consumed 40,000 tokens. About 35,000 of those tokens were just "type": "file" and "language": "python" repeated 2,000 times. That’s 35,000 tokens of pure noise, re-sent on every subsequent API call.

Re-Read Cost Multiplication

GPT-4o and Claude Sonnet charge per input token on every request, not just for new tokens. A conversation that has accumulated 100,000 tokens of history costs 100,000 input tokens on every subsequent call, regardless of how little new content you add.

Practitioners on community forums report that a single build log output can persist across 30+ subsequent turns. If that log is 10,000 tokens, that’s 300,000+ tokens billed just for carrying one stale result forward. This is why agent tool output compression has such outsized ROI compared to other optimization techniques.

For a deeper look at where tokens accumulate in agent workflows, see agent tool call token costs.

Production Data Confirms the Problem

The numbers from production systems are striking. Across 13 million GitHub Copilot sessions, sessions that invoke context compaction account for 44% of all served tokens, and the median compaction removes 73% of the context. AI coding agents spend most of their input tokens on tool output: build logs, test runs, git status, the same file read after every edit.

Context Rot: When Bloat Kills Quality

The cost problem is bad enough, but there’s a quality problem too. Context rot is the gradual degradation of agent output quality as the context window fills up during a long session. Tool outputs are the primary driver.

Research shows that when tool outputs grow large, frontier model performance deteriorates dramatically, with error rates climbing from 7% to roughly 91% in extreme cases. A coding agent implementing a single feature might make 40 or 50 tool calls. The raw output from one test suite run alone can be thousands of tokens. Multiply that across dozens of calls and the agent’s reasoning quality collapses under the weight of its own history.

Common Approaches to Compressing Agent Tool Outputs

There are five distinct approaches to agent tool output compression, each suited to different situations.

Query-Aware Extractive Compression

This approach keeps only the spans of a tool output that are relevant to the agent’s current task or query. A compression model scores which parts of the output matter and discards the rest.

For example, if an agent searches a codebase for error handling patterns, query-aware compression keeps the relevant code blocks and discards the hundreds of unrelated file listings that came back with them. This is the approach that preserves the most useful detail per token, because it understands what the agent is trying to do.

One IBM research team demonstrated compressing a materials science workflow from 20,822,181 tokens down to 1,234 tokens using memory pointers combined with query-aware extraction. That’s not a typo.

Compresr’s LangChain middleware uses this approach, compressing each tool return against the user’s question before it enters the agent’s state. You can integrate it with LangChain or LangGraph directly.

Learn more about how query-specific compression works under the hood.

Rule-Based Deterministic Filtering

This approach strips known noise patterns (timestamps, blank lines, repeated JSON fields, emoji, ANSI color codes) without any ML model involved. It’s fast, predictable, and works well for structured outputs where the noise patterns are consistent.

The open-source tool sqz takes this approach, using per-command formatters that keep what an agent acts on (the failing test, the assertion, the file and line number) and drop what it doesn’t. Anything the model has already seen in context comes back as a short reference instead of full content.

Real session stats from sqz users show meaningful results: across 3,003 compressions, 178,442 tokens saved, with a 24.7% average reduction and up to 92% compression on outputs the model already has in context.

Placeholder and Offloading (Reversible Compression)

Instead of actually compressing the content, this approach replaces large tool outputs with compact placeholders in active context while storing the full output externally. If the agent needs the original data again, it can retrieve it on demand.

The concept is simple: if a ToolMessage exceeds a threshold (say 2,000 tokens), it gets written to disk or a key-value store and replaced with a path reference plus a short preview. The agent sees enough to know what the result contained without paying full token costs on every turn.

This is the only fully reversible approach, making it ideal when you can’t risk losing any detail but want to stop paying for stale context.

LLM-Based Summarization

A cheaper or smaller LLM summarizes tool outputs before the main agent model sees them. This works well for conversational agents or handoff scenarios where the gist matters more than exact details.

The risk is real, though. Summarization is lossy by definition, and common strategies like truncation and lossy summarization may discard useful information or introduce hallucination risk. For agents that need to act on specific file paths, function names, or error codes, a hallucinated detail in the summary can send the agent down the wrong path entirely.

For a comparison of lossy and lossless approaches, see extractive vs. abstractive compression.

Tool-Result Clearing (TRC)

The simplest approach: replace older tool outputs with a short placeholder like [output cleared] while keeping the surrounding agent reasoning and recent tool results intact. Unlike truncation, TRC selectively removes a particular source of context growth rather than chopping off an entire prefix of the conversation.

This works best for stale results the agent no longer needs. A file listing from 20 turns ago, for example, is unlikely to be useful when the agent has since edited three files and moved on to testing.

Comparison of Approaches

The compression ratio you achieve depends heavily on which approach you pick and the nature of your tool outputs. JSON-heavy outputs with repeated fields compress dramatically. Dense numerical data compresses poorly.

Compare prompt compression tools to see how different solutions stack up.

When Compression Helps vs. When to Skip

Compress Aggressively When…

Sessions are long and tool-heavy. If your agent makes more than 10 tool calls per session, compression savings compound quickly. Each compressed result saves tokens not just once but on every subsequent turn.

Tool outputs are verbose and repetitive. Build logs, test output, large JSON responses, and web page scrapes are prime targets. Practitioners report 80 to 99% compression on build and test logs. One developer documented compressing 315KB down to 5.4KB using context-mode sandbox compression.

You’re hitting context window limits. Even with 200K-token windows, long agent sessions fill up fast. Compression buys you headroom to keep going without losing early context.

Costs are scaling with usage. If you’re running agents in production and your token bill is climbing, tool output compression is typically the highest-ROI optimization available. For guidance on calculating the payoff, see net savings from context compression.

Skip or Go Light When…

Outputs are already short. If a tool returns fewer than 500 tokens of dense, relevant information, the overhead of compression (API call latency, processing time) may outweigh the savings.

Fidelity requirements are extreme. For tool-calling agents, every detail is potentially actionable. A missing file path means the wrong file gets read. Research suggests that for an agent with 30 available tools and a 5% acceptable error rate, the fidelity threshold must be at least 0.88. That means you can’t afford to lose more than 12% of the meaningful information.

The agent session is short. A three-turn agent interaction doesn’t accumulate enough context for compression to matter. The payoff comes from preventing re-read multiplication over many turns.

How Agent Tool Output Compression Connects to Other Concepts

Agent tool output compression doesn’t exist in isolation. It fits into a broader family of context engineering techniques:

Context compression is the parent concept. It covers compressing any part of the LLM’s input: system prompts, chat history, retrieved documents, and tool outputs. Tool output compression is the most impactful subcategory for agents because tool results are usually the largest and most frequent additions to context. Read more about context compression as a general practice.

Context rot is the downstream problem that tool output compression prevents. As uncompressed results pile up in context, the agent’s reasoning quality degrades. Compression keeps the context lean and the agent focused.

Prompt compression is the broader category that includes compressing system prompts, few-shot examples, and conversation history in addition to tool outputs. For most agent architectures, tool outputs dominate the token budget, so compressing them first gives the biggest return.

Query-specific compression guides compression based on what the agent needs to accomplish right now, rather than applying a generic reduction. This is what separates high-fidelity agent tool output compression from naive truncation or blind summarization.

Technical Architecture: How Tool Interception Works

To implement tool output compression, intercept the result at the execution layer before appending it to the agent’s message state:

[ Agent Call ] ---> [ Tool Execution ] ---> [ Compression Hook / Middleware ] ---> [ Minimal Context Output ] ---> [ LLM Context ]

Implementation Pattern Example

When configuring middleware in your agent framework, follow these operational steps:

1. Intercept the raw output: Capture the payload immediately after the tool returns execution data.

2. Evaluate length: Check if total tokens exceed your target threshold (e.g., 1,000 tokens).

3. Apply reduction: Strip noise via regex rules or extract high-value spans relative to the query.

4. Append to state: Pass only the compressed ToolMessage back into the LLM conversation state.

Getting Started

If you’re building agents that call tools (and most agents do), compressing tool outputs is one of the fastest ways to cut costs, reduce latency, and improve reasoning quality. The approach you choose depends on your fidelity requirements and the types of tools your agent uses.

Try Compresr free with $10 in credits (no credit card required) to see how query-aware compression handles your tool outputs at $0.10 per million tokens compressed.

Frequently Asked Questions

How is agent tool output compression different from regular prompt compression?

Prompt compression covers everything in the LLM’s input: system prompts, chat history, examples, and tool results. Agent tool output compression focuses specifically on the data returned by tools, which in most agent architectures accounts for the majority of input tokens. The distinction matters because tool outputs have unique characteristics (structured data, repetitive fields, varying relevance) that benefit from specialized compression approaches.

Does compressing tool outputs hurt agent accuracy?

It depends on the approach and the compression ratio. Research shows that light query-aware compression can actually improve accuracy by removing noise that distracts the model. Aggressive compression or naive truncation, on the other hand, can cause agents to pick wrong actions. For agents with many tools, maintaining a fidelity threshold of at least 0.88 is recommended.

Which agent frameworks support tool output compression?

Most major frameworks can accommodate compression. Compresr offers first-party integrations for LangChain, LangGraph, LlamaIndex, and LiteLLM. For other frameworks, you can compress tool outputs at the application layer before inserting them into the agent’s message history. The key is intercepting the tool result before it enters the conversation.

How much can I actually save with agent tool output compression?

Savings vary by use case. Production data from GitHub Copilot shows median compaction removing 73% of context. Individual practitioners report reductions from 24% (average across mixed outputs) to 99.9% (extreme cases like large JSON responses). A reasonable expectation for most agent workloads is 50 to 80% reduction in tool output tokens.

Should I compress every tool output?

No. Very short outputs (under 500 tokens) often don’t justify the compression overhead. Dense, highly relevant outputs where every token matters may also be better left uncompressed. The best practice is to set a minimum token threshold and only compress outputs that exceed it.

What’s the difference between tool-result clearing and actual compression?

Tool-result clearing simply deletes old tool outputs from context and replaces them with a short marker. It saves tokens but destroys information permanently. Actual compression (extractive or summarization-based) retains the important content in fewer tokens. Clearing is appropriate for truly stale results; compression is better when the information might still be useful.

Can built-in context management in tools like Claude Code replace dedicated compression?

Partially. Claude Code and similar tools do perform automatic context compaction. But practitioners report that built-in tools often can’t intercept all tool output paths. For example, native Read, Grep, and Glob tools may bypass compression hooks entirely. Dedicated compression tooling fills those gaps and gives you control over how aggressively outputs are compressed.

Does agent tool output compression add latency?

It adds a small amount of processing time per tool call, but it typically reduces overall session latency because every subsequent LLM call processes fewer input tokens. The prefill time for LLMs scales with input length, so shorter contexts mean faster responses. The net effect is almost always a latency reduction for sessions longer than a few turns.