# https://compresr.ai/docs/introduction

> Human-readable page: https://compresr.ai/docs/introduction

Compresr is a context-compression API for LLM developers. Send the long text you'd pass to your model with the `query` you want answered; Compresr keeps the spans that carry the answer, drops the rest, and returns a shorter context you forward to your LLM. The result: fewer input tokens, lower cost, a longer effective context window, and faster inference. It sits in front of whatever LLM stack you already use and replaces nothing.

## Use cases

Anywhere you send long text with a question about it:

- **RAG** — compress retrieved chunks before they reach the model, so you only pay for tokens that carry the information.
- **Conversations** — compress growing chat history so long sessions stay inside the context window.
- **Tool outputs (agents)** — compress noisy tool results — web search hits, API responses, file dumps — before they re-enter the prompt. Pass the tool call's intent as the `query` so Compresr keeps only what the agent asked for.
- **Document understanding** — ask a question against long, dense documents — legal contracts, medical records, financial filings — and keep only the spans that answer it.

## Models and parameters

Three query-specific compression models are available: **`latte_v1`** (battle-tested, stable), **`latte_v2`** (beta — faster, served at 2× the base rate limits, and adds a `dynamic` mode that picks the compression ratio per input), and **`latte_v2.1`** (beta — the newest checkpoint on the `latte_v2` stack: same speed, same price, same parameters, trained on a broader mix). `latte_v2` is a drop-in replacement for `latte_v1`.

Every call takes three required fields — the `context` to compress, the `query` to score it against, and a `compression_model_name`. The full parameter matrix, defaults, and ratio semantics live in [Models & parameters](/docs/api-reference/models).

## Start with

### Pick a language

The shape of the call is identical across all three; only the syntax differs.

- [Python](/docs/sdks/overview) - `pip install compresr`, then call `client.compress(...)`.
- [TypeScript](/docs/sdks/overview) - `npm install @compresr/sdk`, then call `client.compress({...})`.
- [cURL](/docs/sdks/overview) - one `POST` request, no install required.

### Framework integrations

First-party integrations ship in both SDKs.

- [LangChain](/docs/framework-integration/langchain): three middlewares (tool output, history summarization, prompt budget) + RAG document compressor + single-tool wrapper (HOF or decorator), for `create_agent` and `ContextualCompressionRetriever`.
- [LangGraph](/docs/framework-integration/langgraph): re-exports the three middlewares plus adds `make_compresr_node`, `CompresrCheckpointSerializer`, `CompresrStore`, and `compresr_handoff_tool` for state graphs, at-rest compression, and supervisor → sub-agent transfers.
- [LlamaIndex](/docs/framework-integration/llamaindex): `CompresrNodePostprocessor` for query engines, `wrap_tool_with_compresr` for `FunctionTool`s, and `CompresrMemoryBlock` for the Memory API.
- [LiteLLM](/docs/framework-integration/litellm): Python-only `pre_call` guardrail that auto-compresses tool/function messages before they go upstream, working against every LiteLLM provider.
- [LLM provider recipes](/docs/framework-integration/llm-providers): manual pattern called directly against OpenAI, Anthropic, Gemini, or local Ollama.

### Related reading

- [Quick start](/docs/quick-start) - the same 30-second example in Python, TypeScript, and cURL.
- [Authentication](/docs/authentication) - how `cmp_` keys are issued, rotated, and revoked.
- [API reference](/docs/api-reference/conventions) - every endpoint, parameter, and response field.
- [On-prem deployment](/docs/deployment/on-prem) - run the compression engine inside your own network.


---

# https://compresr.ai/docs/quick-start

> Human-readable page: https://compresr.ai/docs/quick-start

1. **Install the SDK**
   Install the client for your language. The cURL path needs nothing -
   it ships with your OS.

2. **Get an API key**
   Create a key in the [dashboard](/dashboard/tokens), copy
   its `cmp_...` value, and export it as
   `COMPRESR_API_KEY`. See
   [Authentication](/docs/authentication) for the full key
   model and security guidance.

3. **Send a compression request**
   Pass the long `context` you would otherwise send to your
   LLM, plus the `query` you want it to answer.

   - `query`: what the LLM needs to answer. Compression
   keeps the tokens relevant to it, so be specific — good:
   `"What was the project's Q3 churn rate?"`,
   bad: `"churn"` (no intent, degrades to
   generic compression). More in
   [Query-specific compression](/docs/guides/query-specific).

   - `target_compression_ratio`: how hard to compress
   (here `0.75` removes ~75% of tokens). Full semantics:
   [Models & parameters](/docs/api-reference/models#target_compression_ratio).

   ```python
   import os
   from compresr import CompressionClient

   client = CompressionClient(api_key=os.environ["COMPRESR_API_KEY"])

   result = client.compress(
       context=(
           "The James Webb Space Telescope (JWST) is a space telescope designed "
           "primarily to conduct infrared astronomy. As the most powerful telescope "
           "ever launched, its 6.5-metre primary mirror is composed of 18 gold-coated "
           "hexagonal beryllium segments.\n\n"
           "It orbits the Sun near the Sun-Earth L2 Lagrange point, about "
           "1.5 million kilometres from Earth.\n\n"
           "A tennis-court-sized sunshield keeps the instruments below 50 K so the "
           "infrared sensors can detect faint heat signatures from distant galaxies.\n\n"
           "The observatory launched on December 25, 2021 aboard an Ariane 5 rocket "
           "from the Guiana Space Centre.\n\n"
           "It is operated jointly by NASA, ESA, and the Canadian Space Agency, with primary scientific operations conducted from the Space Telescope Science Institute in Baltimore."
       ),
       query="What is the diameter of JWST's primary mirror?",
       compression_model_name="{{DEFAULT_MODEL}}",
       target_compression_ratio=0.75,  # or dynamic=True to auto-pick the ratio
   )

   print(result.data.compressed_context)
   print(
       f"{result.data.original_tokens} -> {result.data.compressed_tokens} tokens "
       f"({result.data.original_tokens / result.data.compressed_tokens:.2f}x)"
   )
   ```

   **TypeScript**

   ```typescript
   import { CompressionClient } from '@compresr/sdk';

   const client = new CompressionClient({
     apiKey: process.env.COMPRESR_API_KEY!,
   });

   const result = await client.compress({
     context:
       "The James Webb Space Telescope (JWST) is a space telescope designed " +
       "primarily to conduct infrared astronomy. As the most powerful telescope " +
       "ever launched, its 6.5-metre primary mirror is composed of 18 gold-coated " +
       "hexagonal beryllium segments.\n\n" +
       "It orbits the Sun near the Sun-Earth L2 Lagrange point, about " +
       "1.5 million kilometres from Earth.\n\n" +
       "A tennis-court-sized sunshield keeps the instruments below 50 K so the " +
       "infrared sensors can detect faint heat signatures from distant galaxies.\n\n" +
       "The observatory launched on December 25, 2021 aboard an Ariane 5 rocket " +
       "from the Guiana Space Centre.\n\n" +
       "It is operated jointly by NASA, ESA, and the Canadian Space Agency, with primary scientific operations conducted from the Space Telescope Science Institute in Baltimore.",
     query: "What is the diameter of JWST's primary mirror?",
     compressionModelName: '{{DEFAULT_MODEL}}',
     targetCompressionRatio: 0.75, // or dynamic: true to auto-pick the ratio
   });

   const { original_tokens, compressed_tokens, compressed_context } = result.data;
   console.log(compressed_context);
   console.log(
     `${original_tokens} -> ${compressed_tokens} tokens ` +
       `(${(original_tokens / compressed_tokens).toFixed(2)}x)`,
   );
   ```

   **cURL**

   ```bash
   curl -X POST https://api.compresr.ai/api/compress/question-specific/ \
     -H "X-API-Key: $COMPRESR_API_KEY" \
     -H "Content-Type: application/json" \
     -d '{
       "context": "The James Webb Space Telescope (JWST) is a space telescope designed primarily to conduct infrared astronomy. As the most powerful telescope ever launched, its 6.5-metre primary mirror is composed of 18 gold-coated hexagonal beryllium segments.\n\nIt orbits the Sun near the Sun-Earth L2 Lagrange point, about 1.5 million kilometres from Earth.\n\nA tennis-court-sized sunshield keeps the instruments below 50 K so the infrared sensors can detect faint heat signatures from distant galaxies.\n\nThe observatory launched on December 25, 2021 aboard an Ariane 5 rocket from the Guiana Space Centre.\n\nIt is operated jointly by NASA, ESA, and the Canadian Space Agency, with primary scientific operations conducted from the Space Telescope Science Institute in Baltimore.",
       "query": "What is the diameter of JWST'"'"'s primary mirror?",
       "compression_model_name": "{{DEFAULT_MODEL}}",
       "target_compression_ratio": 0.75
     }'
   ```

4. **Inspect the result**
   This is the actual response from the live API for the call above
   (numbers will vary slightly run-to-run as `duration_ms`
   depends on load):

   ```text
   The James Webb Space Telescope (JWST) is a space telescope designed primarily to conduct infrared astronomy. As the most powerful telescope ever launched, its 6.5-metre primary mirror is composed of 18 gold-coated hexagonal beryllium segments.[106 tokens dropped]
   160 -> 58 tokens (2.76x)
   ```

   **TypeScript**

   ```text
   The James Webb Space Telescope (JWST) is a space telescope designed primarily to conduct infrared astronomy. As the most powerful telescope ever launched, its 6.5-metre primary mirror is composed of 18 gold-coated hexagonal beryllium segments.[106 tokens dropped]
   160 -> 58 tokens (2.76x)
   ```

   **cURL**

   ```json
   {
     "success": true,
     "message": null,
     "data": {
       "original_tokens": 160,
       "compressed_tokens": 58,
       "target_compression_ratio": 0.75,
       "actual_compression_ratio": 0.6375,
       "tokens_saved": 102,
       "duration_ms": 68,
       "compressed_context": "The James Webb Space Telescope (JWST) is a space telescope designed primarily to conduct infrared astronomy. As the most powerful telescope ever launched, its 6.5-metre primary mirror is composed of 18 gold-coated hexagonal beryllium segments.[106 tokens dropped]"
     }
   }
   ```

   - `compressed_context`: the shortened text. Forward
   this to your LLM exactly as you would the original input.
   `[N tokens dropped]` markers show where spans were
   cut; pass
   `disable_placeholders=true` if you want a clean
   concatenation without them.

   - `actual_compression_ratio`: fraction of input
   tokens _removed_ (here `0.6375` = ~64%
   removed). It is **not** an Nx factor.

   - `target_compression_ratio`: the value you asked
   for, echoed back.

   - `duration_ms`: server-side compression time. Network
   round-trip is on top of this.

From here, tune the parameters or wire Compresr into your stack:

- [SDK reference](/docs/sdks/overview) - Python, TypeScript, and raw REST: full method reference, async variants, streaming, batching.
- [Models](/docs/api-reference/models) - tune `target_compression_ratio` and other latte-only options.
- [Agent client](/docs/sdks/overview#agent-client) - drop-in for `anthropic.Anthropic()` / `openai.OpenAI()` with automatic tool-output compression.
- [Web search](/docs/guides/web-search) - add Tavily or Brave to your agent loop in one line.
- [LangChain integration](/docs/framework-integration/langchain): first-party middleware for tool outputs, history, and outbound prompts, plus a `BaseDocumentCompressor` for RAG.
- [LangGraph integration](/docs/framework-integration/langgraph): state-graph node, lossy checkpoint serializer, store wrapper, and multi-agent handoff tool.
- [LlamaIndex integration](/docs/framework-integration/llamaindex): query-engine postprocessor, tool wrapper, and Memory API block.
- [LiteLLM integration](/docs/framework-integration/litellm): drop the `compresr` guardrail into the proxy and compress tool messages across every provider transparently.

{/* www mirror round-trip test 2026-09-12 */}


---

# https://compresr.ai/docs/api-reference/models

> Human-readable page: https://compresr.ai/docs/api-reference/models

Compresr exposes three query-specific compression models on the public API:

- **`latte_v1`** — battle-tested, stable.
- **`latte_v2`** — the newer, faster model: same compression quality, 2× the base rate limits, plus a `dynamic` mode that picks the compression ratio per input automatically.
- **`latte_v2.1`** — the newest checkpoint on the `latte_v2` serving stack. Same parameters, speed and price as `latte_v2`; everything below that says `latte_v2` applies to `latte_v2.1` too.

Both consume a `context` plus a `query` and return only the spans of `context` that carry signal for the query. **`latte_v2` accepts every parameter `latte_v1` accepts**, with the same defaults and semantics — swapping models is a single string change to `compression_model_name`. When in doubt, start with **`latte_v2` + `target_compression_ratio=0.5`**.

Both models are served by the same endpoints: [`POST /compress/question-specific/`](/docs/api-reference/compress-qs) (single), [`…/stream`](/docs/api-reference/compress-qs-stream) (SSE), and [`…/batch`](/docs/api-reference/compress-qs-batch) (up to 100 rows).

## Parameters

| Parameter | `latte_v1` | `latte_v2` | Default | Purpose |
|---|---|---|---|---|
| `context` | ✓ required | ✓ required | — | Source text to compress. Empty string returns an empty result, no billing. |
| `query` | ✓ required | ✓ required | — | Question/intent that grounds relevance. Cannot be empty. |
| `compression_model_name` | ✓ required | ✓ required | — | `"latte_v1"`, `"latte_v2"` or `"latte_v2.1"`; anything else is a `422`. |
| `target_compression_ratio` | ✓ | ✓ (ignored if `dynamic=true`) | model default | Fixed compression strength — see [below](#target_compression_ratio). |
| `coarse` | ✓ | ✓ | `true` | Paragraph-level scoring; set `false` for finer precision at higher latency. |
| `heuristic_chunking` | ✓ | ✓ | `false` | Structure-aware chunking (paragraphs, code blocks, sections) before scoring. |
| `disable_placeholders` | ✓ | ✓ | `false` | Return kept spans without `[...]` markers between dropped regions. |
| `dynamic` | — | ✓ | `false` | Pick the ratio per input; overrides `target_compression_ratio`. |
| `dynamic_min_ratio` | — | ✓ | model default | Floor on the chosen Nx ratio. Must be `≥ 1.0`. |
| `dynamic_max_ratio` | — | ✓ | model default | Ceiling on the chosen Nx ratio. Must be `≥ dynamic_min_ratio`. |

A `—` means the parameter is **rejected with `422 Unprocessable Entity`** on that model. The wire format is always snake_case; the TypeScript SDK accepts the camelCase forms (`targetCompressionRatio`, `dynamicMinRatio`, …).

## Dynamic mode (`latte_v2` only)

Set `dynamic=true` and the model chooses a compression ratio per input, always inside `[dynamic_min_ratio, dynamic_max_ratio]`. Use it when input difficulty varies and one fixed ratio would be a guess; use a fixed `target_compression_ratio` when you need a predictable token budget per call.

> **Fixed vs dynamic — pick one per call**
> Sending both is not an error, but `dynamic=true` always wins and `target_compression_ratio` is silently ignored for that request. The `dynamic*` parameters are rejected on `latte_v1` with `422`.

## target_compression_ratio

Controls how aggressive the compression is, interpreted **two different ways** depending on the value. Both models share these semantics; every page that mentions a ratio refers back to this table.

| Value | Meaning | Example |
|---|---|---|
| `0 < r ≤ 1` | Removal strength | `0.5` removes ~50% of tokens |
| `r > 1` | Nx target (max `200`) | `4` → ~¼ original |
| omit | Model default: `2` (2× compression) for both models | — |

> **Bounds**
> Values above `200` (and below `0`) are rejected with `422 Unprocessable Entity` — the API does not silently clamp. Exactly `0` is accepted but treated as unset: the model default applies.

> **Not a keep-fraction**
> `0.3` does **not** mean "keep 30%"; it means "remove ~30%".

## Examples

```python
import os
from compresr import CompressionClient

client = CompressionClient(api_key=os.environ["COMPRESR_API_KEY"])

# Fixed ratio — start here.
result = client.compress(
    context=long_document,
    query="What were the key findings of the study?",
    compression_model_name="{{DEFAULT_MODEL}}",
    target_compression_ratio=0.5,
)

# Dynamic ratio — for mixed-difficulty inputs.
result_dyn = client.compress(
    context=long_document,
    query="What were the key findings of the study?",
    compression_model_name="{{DEFAULT_MODEL}}",
    dynamic=True,
)

print(result.data.compressed_context)
```

**TypeScript**

```typescript
import { CompressionClient } from '@compresr/sdk';

const client = new CompressionClient({ apiKey: process.env.COMPRESR_API_KEY! });

// Fixed ratio — start here.
const result = await client.compress({
  context: longDocument,
  query: 'What were the key findings of the study?',
  compressionModelName: '{{DEFAULT_MODEL}}',
  targetCompressionRatio: 0.5,
});

// Dynamic ratio — for mixed-difficulty inputs.
const resultDyn = await client.compress({
  context: longDocument,
  query: 'What were the key findings of the study?',
  compressionModelName: '{{DEFAULT_MODEL}}',
  dynamic: true,
});

console.log(result.data.compressed_context);
```

**cURL**

```bash
# Fixed ratio — start here.
curl -X POST https://api.compresr.ai/api/compress/question-specific/ \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "...",
    "query": "What were the key findings of the study?",
    "compression_model_name": "{{DEFAULT_MODEL}}",
    "target_compression_ratio": 0.5
  }'

# Dynamic ratio — for mixed-difficulty inputs.
curl -X POST https://api.compresr.ai/api/compress/question-specific/ \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "...",
    "query": "What were the key findings of the study?",
    "compression_model_name": "{{DEFAULT_MODEL}}",
    "dynamic": true
  }'
```

For workload patterns (RAG, agent retrieval, search-result trimming) and end-to-end examples, see the [query-specific compression guide](/docs/guides/query-specific).


---

# https://compresr.ai/docs/api-reference/billing-tiers

> Human-readable page: https://compresr.ai/docs/api-reference/billing-tiers

`GET /api/billing/tiers`

Public, unauthenticated endpoint. Returns the same tier schedule that drives the pricing page and the live [`TierTable`](/docs/api-reference/rate-limits) component. Most readers want [Rate limits & headers](/docs/api-reference/rate-limits); this page documents the raw shape.

## Response

| Field | Type | Description |
| --- | --- | --- |
| `success` | boolean | Always true on 200. |
| `message` | string \| null | Always null here. |
| `tiers` | array | Ordered list of tiers, lowest to highest. Note: at the top level, not under data. |
| `tiers.name` | string | Stable tier identifier: tier1, tier2, tier3, tier4, tier5. |
| `tiers.display_name` | string | Human-readable name (e.g. "Tier 1"). |
| `tiers.min_monthly_tokens` | integer | Minimum monthly compressed tokens to reach this tier. |
| `tiers.max_api_keys` | integer | Maximum number of API keys the tier permits. |
| `tiers.rate_limits` | object |  |
| `tiers.rate_limits.rpm` | integer | Requests per minute. -1 means unlimited. |
| `tiers.rate_limits.tpm` | integer | Tokens per minute. -1 means unlimited. |
| `tiers.rate_limits.rpd` | integer | Requests per day. -1 means unlimited. |
| `scale_limits` | object | Per-model-scale limit matrices, keyed by scale ("base", "x2", "x5"). Models with a higher scale run at multiplied limits — see Rate limits. |
| `scale_limits.` | array | One entry per tier: { tier, rate_limits: { rpm, tpm, rpd } }. |

## Status codes

| Status | Description |
| --- | --- |
| 200 | Tiers returned. |
| 500 | Internal error fetching the tier schedule. |

```python
import httpx

response = httpx.get("https://api.compresr.ai/api/billing/tiers", timeout=10)
response.raise_for_status()

for tier in response.json()["tiers"]:
    limits = tier["rate_limits"]
    print(
        f"{tier['display_name']:<12} "
        f"rpm={limits['rpm']:>6} tpm={limits['tpm']:>10} rpd={limits['rpd']:>8}"
    )
```

**TypeScript**

```typescript
const response = await fetch('https://api.compresr.ai/api/billing/tiers');
if (!response.ok) {
  throw new Error(`Failed to load tiers: ${response.status}`);
}

const { tiers } = await response.json();
for (const tier of tiers) {
  const { rpm, tpm, rpd } = tier.rate_limits;
  console.log(`${tier.display_name}  rpm=${rpm} tpm=${tpm} rpd=${rpd}`);
}
```

**cURL**

```bash
curl https://api.compresr.ai/api/billing/tiers
```

  response={

```json
{
  "success": true,
  "message": null,
  "tiers": [
    {
      "name": "tier1",
      "display_name": "Tier 1",
      "min_monthly_tokens": 0,
      "max_api_keys": 2,
      "rate_limits": { "rpm": 60, "tpm": 250000, "rpd": 5000 }
    },
    {
      "name": "tier2",
      "display_name": "Tier 2",
      "min_monthly_tokens": 1000000,
      "max_api_keys": 4,
      "rate_limits": { "rpm": 120, "tpm": 500000, "rpd": 10000 }
    }
  ],
  "scale_limits": {
    "base": [
      { "tier": "tier1", "rate_limits": { "rpm": 60, "tpm": 250000, "rpd": 5000 } }
    ],
    "x2": [
      { "tier": "tier1", "rate_limits": { "rpm": 120, "tpm": 500000, "rpd": 10000 } }
    ]
  }
}
```


---

# https://compresr.ai/docs/api-reference/compress-qs

> Human-readable page: https://compresr.ai/docs/api-reference/compress-qs

`POST /api/compress/question-specific/`

Supply a long `context` and a `query`; the model keeps the parts of the context that matter for that query and drops the rest.

## Request body

`latte_v2` accepts every parameter `latte_v1` accepts, plus three `latte_v2`-only knobs for dynamic compression-ratio selection (rejected with `422` on `latte_v1`). [Models](/docs/api-reference/models) is the canonical parameter reference and decision guide.

### Shared parameters (both models)

| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `context` | string | yes | The text to compress. Pass null or an empty string to get an empty result back with no billing. |
| `query` | string | yes | The question or topic to preserve relevance for. Cannot be empty. |
| `compression_model_name` | "latte_v1" \| "latte_v2" | yes | Routes the call. `{{DEFAULT_MODEL}}` is the recommended default; pick `latte_v1` when you need the original model explicitly. See [Models](/docs/api-reference/models). |
| `target_compression_ratio` | number | no | Compression strength. Removal fraction when `0 < r ≤ 1`, Nx target when `r > 1`. See [Models › target_compression_ratio](/docs/api-reference/models#target_compression_ratio). Ignored on `latte_v2` when `dynamic=true`. |
| `coarse` | boolean | no | Paragraph-level compression. Faster and cheaper, less precise than token-level. |
| `heuristic_chunking` | boolean | no | Pre-chunk the context with structure-aware heuristics (paragraphs, code blocks, markdown sections) before scoring. |
| `disable_placeholders` | boolean | no | Drop the `[...]` markers the model normally inserts between kept spans. |

### `latte_v2`-only parameters

| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `dynamic` | boolean | no | Pick the compression ratio per-input automatically inside `[dynamic_min_ratio, dynamic_max_ratio]`. Overrides `target_compression_ratio` when `true`. Rejected on `latte_v1` with `422`. |
| `dynamic_min_ratio` | number | no | Floor on the chosen Nx ratio when `dynamic=true`. Must be `≥ 1.0`. Only consulted when `dynamic=true`. |
| `dynamic_max_ratio` | number | no | Ceiling on the chosen Nx ratio when `dynamic=true`. Must be `≥ 1.0` and `≥ dynamic_min_ratio`. Only consulted when `dynamic=true`. |

## Response

| Field | Type | Description |
| --- | --- | --- |
| `success` | boolean | true on success. |
| `data` | object |  |
| `data.compressed_context` | string | The compressed output you forward to your LLM. |
| `data.original_tokens` | integer | Token count of the input context. |
| `data.compressed_tokens` | integer | Token count after compression. |
| `data.tokens_saved` | integer | original_tokens − compressed_tokens. |
| `data.target_compression_ratio` | number \| null | The ratio you requested, if any. |
| `data.actual_compression_ratio` | number | The ratio actually achieved (0 to 1). |
| `data.duration_ms` | integer | Server-side processing time in milliseconds. |

## Status codes

| Status | Description |
| --- | --- |
| 200 | Compression succeeded. |
| 400 | Malformed JSON body. |
| 401 | Missing or invalid `X-API-Key`. |
| 422 | A field failed validation (e.g. empty `query`, `target_compression_ratio > 200`). |
| 429 | Rate limit hit for your tier. |
| 500 | Upstream compression service error. |
| 503 | Upstream compression service error. |

```python
import os
from compresr import CompressionClient

client = CompressionClient(api_key=os.environ["COMPRESR_API_KEY"])

result = client.compress(
    context=(
        "The James Webb Space Telescope (JWST) is a space telescope designed "
        "primarily to conduct infrared astronomy. As the most powerful telescope "
        "ever launched, its 6.5-metre primary mirror is composed of 18 gold-coated "
        "hexagonal beryllium segments. It orbits the Sun near the Sun-Earth L2 "
        "Lagrange point, about 1.5 million kilometres from Earth."
    ),
    query="What is the diameter of JWST's primary mirror?",
    compression_model_name="{{DEFAULT_MODEL}}",
    target_compression_ratio=0.5,
)

print(result.data.compressed_context)
```

**TypeScript**

```typescript
import { CompressionClient } from '@compresr/sdk';

const client = new CompressionClient({
  apiKey: process.env.COMPRESR_API_KEY!,
});

const result = await client.compress({
  context:
    'The James Webb Space Telescope (JWST) is a space telescope ' +
    'designed primarily to conduct infrared astronomy. Its 6.5-metre ' +
    'primary mirror is composed of 18 gold-coated hexagonal beryllium ' +
    'segments. It orbits near the Sun-Earth L2 Lagrange point.',
  query: "What is the diameter of JWST's primary mirror?",
  compressionModelName: '{{DEFAULT_MODEL}}',
  targetCompressionRatio: 0.5,
});

console.log(result.data.compressed_context);
```

**cURL**

```bash
curl -X POST https://api.compresr.ai/api/compress/question-specific/ \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "The James Webb Space Telescope (JWST) ...",
    "query": "What is the diameter of JWST'"'"'s primary mirror?",
    "compression_model_name": "{{DEFAULT_MODEL}}",
    "target_compression_ratio": 0.5
  }'
```

  response={

```json
{
  "success": true,
  "data": {
    "compressed_context": "JWST's 6.5-metre primary mirror is composed of 18 gold-coated hexagonal beryllium segments.",
    "original_tokens": 78,
    "compressed_tokens": 21,
    "tokens_saved": 57,
    "target_compression_ratio": 0.5,
    "actual_compression_ratio": 0.27,
    "duration_ms": 412
  }
}
```


---

# https://compresr.ai/docs/api-reference/compress-qs-batch

> Human-readable page: https://compresr.ai/docs/api-reference/compress-qs-batch

`POST /api/compress/question-specific/batch`

Each input has its own `context` and `query`, so you can run a batch where every row is a different document scored against a different question. Useful for RAG pipelines where you compress one chunk per retrieved document.

## Request body

All compression knobs are request-level — they apply to every row in `inputs`. `latte_v2` accepts every parameter `latte_v1` accepts, plus the three `latte_v2`-only `dynamic*` knobs (rejected on `latte_v1` with `422`). [Models](/docs/api-reference/models) is the canonical parameter reference.

### Shared parameters (both models)

| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `inputs` | Array<{context, query}> | yes | Between 1 and 100 items. Each item has a context (string, nullable) and a non-empty query. |
| `compression_model_name` | "latte_v1" \| "latte_v2" | yes | Routes the call. Applies to every row. See [Models](/docs/api-reference/models). |
| `target_compression_ratio` | number | no | Compression strength. See [Models › target_compression_ratio](/docs/api-reference/models#target_compression_ratio). Ignored on `latte_v2` when `dynamic=true`. |
| `coarse` | boolean | no | Paragraph-level compression. Applies to every row. |

### `latte_v2`-only parameters

| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `dynamic` | boolean | no | Picks the compression ratio per-input; overrides `target_compression_ratio` when `true`. Applies to every row. Rejected on `latte_v1` with `422`. |
| `dynamic_min_ratio` | number | no | Floor on the chosen Nx ratio when `dynamic=true`. Must be `≥ 1.0`. |
| `dynamic_max_ratio` | number | no | Ceiling on the chosen Nx ratio when `dynamic=true`. Must be `≥ 1.0`. |

Items with `context: null` or `context: ""` return an empty result and are not billed.

> The Python and TypeScript SDKs flatten this into parallel `contexts` and `queries` arrays. See the [batch guide](/docs/guides/batch). The REST shape shown above is what goes on the wire when you hit the endpoint directly with cURL or fetch.

## Response

| Field | Type | Description |
| --- | --- | --- |
| `success` | boolean | true when the batch was accepted and processed. |
| `data` | object |  |
| `data.results` | array | Per-row results, in input order. Each entry has the same shape as the single-compression response data. |
| `data.results.compressed_context` | string |  |
| `data.results.original_tokens` | integer |  |
| `data.results.compressed_tokens` | integer |  |
| `data.results.tokens_saved` | integer |  |
| `data.results.target_compression_ratio` | number \| null |  |
| `data.results.actual_compression_ratio` | number |  |
| `data.results.duration_ms` | integer |  |
| `data.total_original_tokens` | integer | Sum across all rows. |
| `data.total_compressed_tokens` | integer | Sum across all rows. |
| `data.total_tokens_saved` | integer | Sum across all rows. |
| `data.average_compression_ratio` | number | Mean actual_compression_ratio across all rows. |
| `data.count` | integer | Number of rows processed. |

## Status codes

| Status | Description |
| --- | --- |
| 200 | Batch processed. Inspect each row in `data.results`. |
| 401 | Missing or invalid `X-API-Key`. |
| 422 | Validation failed (empty `inputs`, more than 100 rows, empty `query` on a row, ratio out of range). |
| 429 | Rate limit hit. Batch usage counts as one request but full token-volume. |
| 500 | Upstream error. |
| 503 | Upstream error. |

```python
import os
from compresr import CompressionClient

client = CompressionClient(api_key=os.environ["COMPRESR_API_KEY"])

# The SDK takes parallel `contexts` and `queries` arrays, not
# REST-style `inputs`; it pairs them row-by-row on the wire.
batch = client.compress_batch(
    contexts=[
        "Returns are accepted within 30 days of delivery...",
        "Standard EU shipping takes 3-5 business days...",
    ],
    queries=["What is the refund window?", "How long is EU shipping?"],
    compression_model_name="{{DEFAULT_MODEL}}",
    target_compression_ratio=0.5,
)

for i, row in enumerate(batch.data.results):
    print(f"row {i}: {row.original_tokens} -> {row.compressed_tokens}")
```

**TypeScript**

```typescript
import { CompressionClient } from '@compresr/sdk';

const client = new CompressionClient({
  apiKey: process.env.COMPRESR_API_KEY!,
});

// The SDK takes parallel `contexts` and `queries` arrays, not
// REST-style `inputs`; it pairs them row-by-row on the wire.
const batch = await client.compressBatch({
  contexts: [
    'Returns are accepted within 30 days of delivery...',
    'Standard EU shipping takes 3-5 business days...',
  ],
  queries: ['What is the refund window?', 'How long is EU shipping?'],
  compressionModelName: '{{DEFAULT_MODEL}}',
  targetCompressionRatio: 0.5,
});

batch.data.results.forEach((row, i) => {
  console.log(`row ${i}: ${row.original_tokens} -> ${row.compressed_tokens}`);
});
```

**cURL**

```bash
curl -X POST https://api.compresr.ai/api/compress/question-specific/batch \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": [
      {"context": "Returns are accepted within 30 days of delivery...", "query": "What is the refund window?"},
      {"context": "Standard EU shipping takes 3-5 business days...", "query": "How long is EU shipping?"}
    ],
    "compression_model_name": "{{DEFAULT_MODEL}}",
    "target_compression_ratio": 0.5
  }'
```

  response={

```json
{
  "success": true,
  "data": {
    "results": [
      {
        "compressed_context": "Returns are accepted within 30 days",
        "original_tokens": 9,
        "compressed_tokens": 7,
        "tokens_saved": 2,
        "target_compression_ratio": 0.5,
        "actual_compression_ratio": 0.22,
        "duration_ms": 188
      }
    ],
    "total_original_tokens": 18,
    "total_compressed_tokens": 11,
    "total_tokens_saved": 7,
    "average_compression_ratio": 0.39,
    "count": 2
  }
}
```


---

# https://compresr.ai/docs/api-reference/compress-qs-stream

> Human-readable page: https://compresr.ai/docs/api-reference/compress-qs-stream

`POST /api/compress/question-specific/stream`

Same input as [`POST /compress/question-specific/`](/docs/api-reference/compress-qs), but the response is a Server-Sent Events stream instead of a JSON envelope. When to use it (and why plain compress usually wins today): [streaming guide](/docs/guides/streaming).

## Request body

Identical to the [non-streaming endpoint](/docs/api-reference/compress-qs). `latte_v2` accepts every parameter `latte_v1` accepts, plus the three `latte_v2`-only `dynamic*` knobs (rejected on `latte_v1` with `422`).

### Shared parameters (both models)

| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `context` | string | yes | The text to compress. |
| `query` | string | yes | The question or topic to preserve relevance for. |
| `compression_model_name` | "latte_v1" \| "latte_v2" | yes | Routes the call. See [Models](/docs/api-reference/models). |
| `target_compression_ratio` | number | no | Compression strength. See [Models › target_compression_ratio](/docs/api-reference/models#target_compression_ratio). Ignored on `latte_v2` when `dynamic=true`. |
| `coarse` | boolean | no | Paragraph-level compression (faster, less precise). |
| `heuristic_chunking` | boolean | no | Pre-chunk with structure-aware heuristics. |
| `disable_placeholders` | boolean | no | Drop the `[...]` markers between kept spans. |

### `latte_v2`-only parameters

| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `dynamic` | boolean | no | Picks the compression ratio per-input; overrides `target_compression_ratio` when `true`. Rejected on `latte_v1` with `422`. |
| `dynamic_min_ratio` | number | no | Floor on the chosen Nx ratio when `dynamic=true`. Must be `≥ 1.0`. |
| `dynamic_max_ratio` | number | no | Ceiling on the chosen Nx ratio when `dynamic=true`. Must be `≥ 1.0`. |

## Response

The response uses `Content-Type: text/event-stream`. Each event is a `data: ` frame. On success the server emits **one** content event carrying the full compressed output, then closes the stream:

```text
data: {"content": "<full compressed output>", "done": true}
```

If compression fails or times out (120s server-side budget) after the stream has opened, the server emits an error event instead, then closes:

```text
data: {"error": "Stream timeout exceeded"}
```

| Field | Type | Description |
| --- | --- | --- |
| `content` | string | Compressed text. Currently the full output arrives in a single event; concatenate content across events to stay forward-compatible with incremental chunks. |
| `done` | boolean | true on the final content event. The stream closes after it. |
| `error` | string | Present instead of content when the stream aborts mid-flight. The stream closes after it. |

## Status codes

| Status | Description |
| --- | --- |
| 200 | Stream opened. Body is `text/event-stream`. |
| 401 | Missing or invalid `X-API-Key`. |
| 422 | Field validation failure. |
| 429 | Rate limit hit. |
| 500 | Upstream error. Stream may include an `error` chunk before closing. |
| 503 | Upstream error. Stream may include an `error` chunk before closing. |

> The streaming endpoint returns events, not the standard response envelope, and it never returns the token-accounting metadata (`original_tokens`, `compressed_tokens`, `tokens_saved`, `actual_compression_ratio`, `duration_ms`). If you need those, use the [non-streaming endpoint](/docs/api-reference/compress-qs). Once the stream is open, failures arrive as an `error` event rather than an HTTP status. The SDK iterators skip that event, so an empty result is the only signal; the examples below treat it as a failure. Full handling, including retries, is in the [streaming guide](/docs/guides/streaming).

```python
import os
from compresr import CompressionClient

client = CompressionClient(api_key=os.environ["COMPRESR_API_KEY"])

stream = client.compress_stream(
    context="...long incident report...",
    query="What was the root cause of the outage?",
    compression_model_name="{{DEFAULT_MODEL}}",
)

buffer = []
for chunk in stream:
    buffer.append(chunk.content)
    if chunk.done:
        break

compressed = "".join(buffer)
if not compressed:
    # A server-side error mid-stream ends the iteration without raising.
    raise RuntimeError("Compression stream returned no output")

print(compressed)
```

**TypeScript**

```typescript
import { CompressionClient } from '@compresr/sdk';

const client = new CompressionClient({
  apiKey: process.env.COMPRESR_API_KEY!,
});

const stream = await client.compressStream({
  context: '...long incident report...',
  query: 'What was the root cause of the outage?',
  compressionModelName: '{{DEFAULT_MODEL}}',
});

const buffer: string[] = [];
for await (const chunk of stream) {
  buffer.push(chunk.content);
  if (chunk.done) break;
}

const compressed = buffer.join('');
if (!compressed) {
  // A server-side error mid-stream ends the iteration without throwing.
  throw new Error('Compression stream returned no output');
}

console.log(compressed);
```

**cURL**

```bash
curl -N -X POST https://api.compresr.ai/api/compress/question-specific/stream \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -H "Accept: text/event-stream" \
  -d '{
    "context": "...long incident report...",
    "query": "What was the root cause of the outage?",
    "compression_model_name": "{{DEFAULT_MODEL}}"
  }'
```

  response={

```text
data: {"content": "Root cause: an expired TLS certificate ... <full compressed output>", "done": true}
```


---

# https://compresr.ai/docs/api-reference/conventions

> Human-readable page: https://compresr.ai/docs/api-reference/conventions

Shared contract for every Compresr REST endpoint: base URL, auth header, request format, response envelope, status codes, rate limits, and versioning. Per-endpoint pages only document what's specific to that route.

## Base URL

All Compresr endpoints are served from a single host over HTTPS. Plain HTTP is not supported; requests to `http://api.compresr.ai` are rejected at the edge.

```text
https://api.compresr.ai
```

Endpoints are mounted under `/api`. For example, the question-specific compression endpoint is `https://api.compresr.ai/api/compress/question-specific/`.

## Authentication

Authenticated endpoints expect an API key in the `X-API-Key` header (the SDKs attach it for you):

```bash
-H "X-API-Key: $COMPRESR_API_KEY"
```

How to obtain, supply, and manage keys: [Authentication](/docs/authentication).

## Content type

Requests with a body use `Content-Type: application/json`. Responses are always JSON (except streaming endpoints, which use `text/event-stream`, see below). The SDKs set the request header for you; with cURL you must set it explicitly.

```python
# Content-Type: application/json is set by the SDK.
result = client.compress(
    context="...",
    query="...",
    compression_model_name="{{DEFAULT_MODEL}}",
)
```

**TypeScript**

```typescript
// Content-Type: application/json is set by the SDK.
const result = await client.compress({
  context: '...',
  query: '...',
  compressionModelName: '{{DEFAULT_MODEL}}',
});
```

**cURL**

```bash
curl -X POST https://api.compresr.ai/api/compress/question-specific/ \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "context": "...", "query": "...", "compression_model_name": "{{DEFAULT_MODEL}}" }'
```

Field names use `snake_case` (`compression_model_name`, `target_compression_ratio`). The TypeScript SDK exposes the same fields as `camelCase` and translates them on the wire.

## Response envelope

`success` is the boolean discriminant on every response. Success responses carry the endpoint payload under `data`; error responses are flat, with the message in `error` and a stable `code` to switch on.

On success:

| Field | Type | Description |
| --- | --- | --- |
| `success` | boolean | true. |
| `message` | string \| null | Optional human-readable note. Usually null. |
| `data` | object | Endpoint-specific payload. |

On failure:

| Field | Type | Description |
| --- | --- | --- |
| `success` | boolean | false. |
| `error` | string | Human-readable message safe to log. Wording may change. |
| `code` | string | Stable machine-readable error code (e.g. authentication_error). Switch on this in client code. |
| `detail` | string \| null | Extra context when available. |
| `retry_after` | integer \| null | Seconds to wait before retrying, on rate-limit errors. Mirrored in the Retry-After header. |
| `field` | string \| null | The request field that failed validation, when attributable to one. |

A concrete error response looks like:

```json
{
  "success": false,
  "error": "Invalid or expired API key",
  "code": "authentication_error",
  "detail": "Authentication failed. Please verify your API key at https://compresr.ai/dashboard/keys",
  "retry_after": null,
  "field": null
}
```

Two framework-level exceptions bypass the envelope: a missing required header or unparseable body returns `422` with FastAPI's `{"detail": [...]}` shape, and an unknown route returns `404` with `{"detail": "Not Found"}`. See [Error codes](/docs/api-reference/errors) for both.

Streaming endpoints (paths ending in `/stream`) do not use this envelope; they return Server-Sent Events. See [`POST /compress/question-specific/stream`](/docs/api-reference/compress-qs-stream) for the event shape.

## Status codes


---

# https://compresr.ai/docs/api-reference/errors

> Human-readable page: https://compresr.ai/docs/api-reference/errors

Every error the API generates uses the same flat envelope. The HTTP status code is the category; the `code` string is the specific failure mode, safe to switch on in client code.

## Error envelope

| Field | Type | Description |
| --- | --- | --- |
| `success` | boolean | Always false on error responses. |
| `error` | string | Human-readable message safe to log or surface to operators. Wording may change. |
| `code` | string | Stable machine-readable error code (e.g. authentication_error). Switch on this in client code. |
| `detail` | string \| null | Extra context when available, e.g. a pointer to the dashboard. |
| `retry_after` | integer \| null | Seconds to wait before retrying. Set on rate-limit errors; mirrored in the Retry-After header. |
| `field` | string \| null | The request field that failed validation, when the error is attributable to one. |

A concrete error response:

```json
{
  "success": false,
  "error": "Invalid or expired API key",
  "code": "authentication_error",
  "detail": "Authentication failed. Please verify your API key at https://compresr.ai/dashboard/keys",
  "retry_after": null,
  "field": null
}
```

> **Two framework-level exceptions**
> Errors raised before a request reaches the API's handlers use FastAPI's default shape instead of the envelope: a **missing required header or unparseable body** returns `422` with `{"detail": [{"type": "missing", "loc": ["header", "X-API-Key"], ...}]}`, and an **unknown route or method** returns `404` with `{"detail": "Not Found"}`. Everything else uses the envelope above.

## 401 Unauthorized

The `X-API-Key` header is present but the key is malformed, unknown, revoked, or expired.

- `code: authentication_error`: covers all key failures — the `error` and `detail` strings say which.
- A request with **no** `X-API-Key` header at all returns `422` in the FastAPI shape (see callout above), not `401`.
- **Recover by**: issuing a fresh key in the [dashboard](/dashboard/api-keys) and updating your secret store. See [Authentication](/docs/authentication).

## 402 Payment Required

A billing operation failed at the payment layer.

- `code: insufficient_credits`, `budget_limit_reached`, `payment_failed`.
- Note: running out of usage balance on **compression** requests surfaces as `429` (see below), not `402`.
- **Recover by**: topping up or resolving the payment method in the dashboard.

## 403 Forbidden

The key authenticates but its scope doesn't permit this endpoint (for example, a demo-scoped key calling a compression endpoint).

- `code: scope_error`.
- **Recover by**: using a key with `user` or `enterprise` scope. Retrying with the same key returns the same error.

## 404 Not Found

- `code: not_found`: the referenced resource (e.g. an API-key ID or model name in a path) doesn't exist or isn't visible to you.
- An unknown URL path returns the FastAPI shape `{"detail": "Not Found"}` instead.
- **Recover by**: checking the path and identifiers; do not retry unchanged.

## 422 Unprocessable Entity

The request was well-formed but one or more fields failed validation.

- `code: validation_error`: typical causes are empty `query`, `target_compression_ratio` outside `0` to `200`, more than 100 `inputs` items, or an unknown `compression_model_name`. When the failure is attributable to a single field, `field` names it.
- **Recover by**: fixing the field indicated by `field` / `error` and retrying. Identical retries fail identically.

## 429 Too Many Requests

Your tier's per-minute or per-day quota is exhausted — or your usage balance / budget can't cover the request (the API pre-authorizes at worst-case cost).

- `code: rate_limit_exceeded`: request, token, or spend budget exceeded. Rate-limit hits include `retry_after` and a `Retry-After` header; balance exhaustion does not clear on its own — check the `error` message to tell them apart.
- **Recover by**: backing off with exponential delay that respects `Retry-After` (see [Rate limits](/docs/api-reference/rate-limits#handling-429)); if the message says usage limit, top up or raise your budget instead of retrying.

## 500 Internal Server Error

Unexpected server-side failure. Not your fault.

- `code: server_error`.
- **Recover by**: retrying once with a short delay. If it persists, check the [status page](https://status.compresr.ai) and contact support.

## 503 Service Unavailable

An upstream dependency is unhealthy or the API is shedding load deliberately.

- `code: service_unavailable`, `connection_error`, or `circuit_open`.
- May include a `Retry-After` header.
- **Recover by**: honouring `Retry-After`. Do not hammer the endpoint; that's what the circuit breaker is preventing.

## 504 Gateway Timeout

The request exceeded the server-side time budget.

- `code: timeout`.
- **Recover by**: retrying once; for very large contexts, consider splitting the input or using [batch](/docs/api-reference/compress-qs-batch).

> **Best practices**
> - Validate inputs client-side before making the request. Most `422` errors are catchable in your own code.
> - Retry only on `429`, `500`, `503`, and `504`. Never retry other `4xx` errors; the response will be identical.
> - Use exponential backoff that respects `Retry-After` as a floor, not a ceiling.
> - Log `code` (stable) rather than `error` (human-readable, may change).


---

# https://compresr.ai/docs/api-reference/pricing-estimate

> Human-readable page: https://compresr.ai/docs/api-reference/pricing-estimate

`POST /api/pricing/estimate-cost`

Public, unauthenticated endpoint used by the pricing calculator. Given a compression model, a number of input tokens, and a target ratio, it returns the estimated processing cost and how many tokens would survive compression. For your account's actual quotas and per-tier limits, see [Rate limits & tiers](/docs/api-reference/rate-limits) and [`GET /billing/tiers`](/docs/api-reference/billing-tiers).

## Request body

| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `compression_model` | string | no | Compression model to estimate against (e.g. "{{DEFAULT_MODEL}}"). |
| `input_tokens` | integer | yes | Number of input tokens to compress. Must be greater than 0. |
| `compression_ratio` | number | no | Fraction of tokens removed, between 0 and 1 (exclusive). 0.3 means remove ~30% of tokens (keep ~70%). Note this differs from the compression endpoints, where values above 1 are also accepted as an Nx factor. |

## Response

| Field | Type | Description |
| --- | --- | --- |
| `success` | boolean |  |
| `data` | object |  |
| `data.input_tokens` | integer | Echoed input volume. |
| `data.compressed_tokens` | integer | Estimated token count after compression. |
| `data.tokens_saved` | integer | input_tokens − compressed_tokens. |
| `data.compression_ratio` | number | Echoed removal fraction (0 to 1). |
| `data.processing_cost_usd` | number | Estimated cost in USD for the compression step alone. |
| `data.model` | string | Model used for the estimate. |
| `data.input_price_per_1m` | number | Input price per 1M tokens at the time of the estimate. |

## Status codes

| Status | Description |
| --- | --- |
| 200 | Estimate returned. |
| 422 | Field validation failure: `input_tokens <= 0`, ratio outside 0 to 1, or unknown `compression_model` (the `field` property names the offender). |
| 429 | Per-IP rate limit on this public endpoint. |
| 500 | Internal pricing-service failure. |

```python
import httpx

response = httpx.post(
    "https://api.compresr.ai/api/pricing/estimate-cost",
    json={
        "compression_model": "{{DEFAULT_MODEL}}",
        "input_tokens": 100_000,
        "compression_ratio": 0.3,
    },
    timeout=10,
)
response.raise_for_status()

data = response.json()["data"]
print(
    f"{data['input_tokens']} -> {data['compressed_tokens']} tokens, "
    f"${data['processing_cost_usd']:.4f} processing cost"
)
```

**TypeScript**

```typescript
const response = await fetch(
  'https://api.compresr.ai/api/pricing/estimate-cost',
  {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      compression_model: '{{DEFAULT_MODEL}}',
      input_tokens: 100_000,
      compression_ratio: 0.3,
    }),
  },
);

if (!response.ok) throw new Error(`Estimate failed: ${response.status}`);

const { data } = await response.json();
console.log(
  `${data.input_tokens} -> ${data.compressed_tokens} tokens, ` +
    `$${data.processing_cost_usd.toFixed(4)} processing cost`,
);
```

**cURL**

```bash
curl -X POST https://api.compresr.ai/api/pricing/estimate-cost \
  -H "Content-Type: application/json" \
  -d '{
    "compression_model": "{{DEFAULT_MODEL}}",
    "input_tokens": 100000,
    "compression_ratio": 0.3
  }'
```

  response={

```json
{
  "success": true,
  "message": null,
  "data": {
    "input_tokens": 100000,
    "compressed_tokens": 70000,
    "tokens_saved": 30000,
    "compression_ratio": 0.3,
    "processing_cost_usd": 0.01,
    "model": "{{DEFAULT_MODEL}}",
    "input_price_per_1m": 0.1
  }
}
```


---

# https://compresr.ai/docs/api-reference/rate-limits

> Human-readable page: https://compresr.ai/docs/api-reference/rate-limits

Compresr rate-limits per user across three separate quotas: requests per minute (`rpm`), tokens per minute (`tpm`), and requests per day (`rpd`). Your tier is determined by monthly compressed-token volume; hitting any quota returns `429` with a `Retry-After` header.

## Tiers

Here are the current limits per tier. To read them programmatically, use [`GET /billing/tiers`](/docs/api-reference/billing-tiers).

Limits also scale with the model you call: `latte_v2`-family models run at roughly twice the base `rpm`/`tpm`. The per-model matrices are in the [`GET /billing/tiers`](/docs/api-reference/billing-tiers) response under `scale_limits`. Individual keys can also carry an optional daily request cap, set at key creation.

## Handling 429

The `Retry-After` header (seconds to wait, mirrored as `retry_after` in the [error envelope](/docs/api-reference/errors)) is the only rate-limit signal the API sends — there are no `X-RateLimit-*` headers. Retry with exponential backoff that treats `Retry-After` as a floor.

`429` also covers **usage-balance exhaustion** (the API pre-authorizes each request at worst-case cost). A rate-limit 429 clears when the window rolls over; a balance 429 does not — check the `error` message to tell them apart.

> **Need higher limits?**
> If you hit 429 routinely under normal load, retries won't fix it — contact us at [support@compresr.ai](mailto:support@compresr.ai) for a tier upgrade.

```python
import os
import time
import httpx

def compress_with_retry(payload, *, max_attempts=4):
    url = "https://api.compresr.ai/api/compress/question-specific/"
    headers = {
        "X-API-Key": os.environ["COMPRESR_API_KEY"],
        "Content-Type": "application/json",
    }

    for attempt in range(max_attempts):
        response = httpx.post(url, json=payload, headers=headers, timeout=60)

        if response.status_code != 429:
            response.raise_for_status()
            return response.json()

        retry_after = int(response.headers.get("Retry-After", "1"))
        backoff = max(retry_after, 2 ** attempt)
        time.sleep(backoff)

    raise RuntimeError("Exceeded retry budget")
```

**TypeScript**

```typescript
const sleep = (ms: number) =>
  new Promise<void>((resolve) => setTimeout(resolve, ms));

interface CompressPayload {
  context: string;
  query: string;
  compression_model_name: '{{DEFAULT_MODEL}}';
}

async function compressWithRetry(
  payload: CompressPayload,
  { maxAttempts = 4 }: { maxAttempts?: number } = {},
) {
  const apiKey = process.env.COMPRESR_API_KEY!;

  for (let attempt = 0; attempt < maxAttempts; attempt += 1) {
    const response = await fetch(
      'https://api.compresr.ai/api/compress/question-specific/',
      {
        method: 'POST',
        headers: { 'X-API-Key': apiKey, 'Content-Type': 'application/json' },
        body: JSON.stringify(payload),
      },
    );

    if (response.status !== 429) {
      if (!response.ok) throw new Error(`Failed: ${response.status}`);
      return response.json();
    }

    const retryAfter = Number(response.headers.get('Retry-After') ?? '1');
    const backoff = Math.max(retryAfter, 2 ** attempt);
    await sleep(backoff * 1000);
  }

  throw new Error('Exceeded retry budget');
}
```

**cURL**

```bash
# cURL's built-in retry honours Retry-After on 429/5xx.
curl --retry 3 --retry-delay 2 --retry-all-errors \
  -X POST https://api.compresr.ai/api/compress/question-specific/ \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "...",
    "query": "...",
    "compression_model_name": "{{DEFAULT_MODEL}}"
  }'
```

  response={

```text
HTTP/1.1 429 Too Many Requests
Retry-After: 12

{
  "success": false,
  "error": "Rate limit exceeded. Please retry in 12 seconds.",
  "code": "rate_limit_exceeded",
  "detail": null,
  "retry_after": 12,
  "field": null
}
```


---

# https://compresr.ai/docs/authentication

> Human-readable page: https://compresr.ai/docs/authentication

Every request needs an API key in the `X-API-Key` header. Create one in the [dashboard](/dashboard/api-keys) — keys start with `cmp_` and the full value is shown only once, so copy it immediately.

There are three ways to authenticate yourself:

## 1. Pass the key explicitly

Read it from the environment (or your secret manager) and hand it to the client. Never paste a key literal into source.

```python
import os
from compresr import CompressionClient

client = CompressionClient(api_key=os.environ["COMPRESR_API_KEY"])
```

**TypeScript**

```typescript
import { CompressionClient } from '@compresr/sdk';

const client = new CompressionClient({ apiKey: process.env.COMPRESR_API_KEY! });
```

**cURL**

```bash
curl -X POST https://api.compresr.ai/api/compress/question-specific/ \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "...",
    "query": "...",
    "compression_model_name": "{{DEFAULT_MODEL}}"
  }'
```

## 2. Set `COMPRESR_API_KEY`

Both SDKs read the `COMPRESR_API_KEY` environment variable when no key is passed explicitly:

```python
# export COMPRESR_API_KEY="cmp_your_api_key"
from compresr import CompressionClient

client = CompressionClient()  # key picked up from the environment
```

**TypeScript**

```typescript
// export COMPRESR_API_KEY="cmp_your_api_key"
import { CompressionClient } from '@compresr/sdk';

// Note: the options object is required, even when empty.
const client = new CompressionClient({}); // key picked up from the environment
```

**cURL**

```bash
export COMPRESR_API_KEY="cmp_your_api_key"

# cURL has no implicit pickup; reference the variable in the header.
curl -X POST https://api.compresr.ai/api/compress/question-specific/ \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "...",
    "query": "...",
    "compression_model_name": "{{DEFAULT_MODEL}}"
  }'
```

## 3. Log in once with the CLI

Both SDKs ship a `compresr-sdk` binary that stores a credential on your machine:

```bash
compresr-sdk login    # opens a browser flow, saves the key locally
compresr-sdk logout   # clears the local credential AND revokes the key server-side
```

`login` writes `~/.compresr/credentials` (INI, read by Python automatically) or `~/.compresr/credentials.json` (JSON, read by TypeScript via `createClient` from `@compresr/sdk/auth`). Deleting the file by hand only removes the local copy — use `logout` to actually revoke the key.

## When authentication fails

| Situation | Status | What you get |
|---|---|---|
| `X-API-Key` header missing | `422` | FastAPI's `{"detail": [...]}` shape, not the standard envelope |
| Key invalid, revoked, or expired | `401` | `code: authentication_error` |
| Wrong scope (e.g. a demo key on a compression endpoint) | `403` | `code: scope_error` |
| Balance or budget exhausted | `429` | `code: rate_limit_exceeded`, "usage limit exceeded" message |

Full envelope shapes and recovery guidance: [Error codes](/docs/api-reference/errors).

## Managing keys

Treat keys like passwords — anyone who has one can incur charges on your account until you revoke it.

**Storage.** Read keys from environment variables (`COMPRESR_API_KEY`); never hardcode them in source. Add `.env` to `.gitignore` before writing a key into it. In production, use a real secret manager (AWS/Google Secret Manager, Vault, Doppler, …) and inject at deploy time. Never put a key in client-side code, chat, tickets, or screenshots — if you do, treat it as leaked.

**Rotation.** Rotate on a schedule (90 days is a sane production default) and immediately on suspected compromise. Mint the new key, deploy it alongside the old one, confirm zero traffic on the old key in the dashboard usage log, then revoke it — revocation is immediate and returns `401` from that point on.

**Budgets and expiry.** A key can carry a monthly USD budget set at creation: once exhausted, requests return `429 rate_limit_exceeded` until you raise it or rotate. Small budgets cap the blast radius of keys handed to CI, contractors, or demos. A key can also carry an `expires_at` timestamp — past it, the key returns `401` with no further action needed. Use expiry for time-boxed access, budget-plus-rotation for permanent services.

**If a key leaks.** Revoke it in the [dashboard](/dashboard/api-keys) first — don't wait to confirm anyone saw it. Then rotate to a new key everywhere, audit the usage log for unfamiliar traffic, and if unexpected charges occurred, email founders@compresr.ai with the masked key preview from the dashboard (never the full key). Finish with a repo-wide scan (`git log -S "cmp_"`) — where one key leaked, others may have.


---

# https://compresr.ai/docs/deployment/on-prem

> Human-readable page: https://compresr.ai/docs/deployment/on-prem

Compresr on-prem runs the same compression API you use at `api.compresr.ai` inside your own network. Your documents are processed on your hardware and never leave it; only anonymous usage counts are reported back for billing. The deployment is a two-container Docker stack delivered via [Distr](https://distr.sh), with model weights baked into the image.

## When to choose on-prem

- **Data can't leave your network.** Compliance, client contracts, or internal policy rule out sending context to a hosted API.
- **Air-gapped or egress-restricted environments.** The engine container needs no internet access at any point — weights ship inside the image.
- **Latency control.** Compression runs next to your retrieval stack, with no WAN round-trip.

If none of these apply, the hosted API is simpler: no GPU to provision, no upgrades to schedule.

## Architecture

| Container | Role | Network exposure |
|---|---|---|
| `compresr-proxy` | Customer-facing API: auth, validation, admission control, usage metering | Publishes port `8000` |
| `compresr-engine` | GPU compression engine | **No host port.** Joined only to an internal Docker network with no outbound internet |

The proxy is the only thing your clients (and your network) ever talk to. The engine is reachable solely from the proxy over the internal network, and every proxy→engine call carries a short-lived signed token — hitting the engine directly is rejected even from inside the Docker host.

Which compression model the engine serves is selected at install time through Distr; the API surface is the same regardless.

## Requirements

- **GPU:** an NVIDIA GPU with **at least 24 GB VRAM**, plus the NVIDIA Container Toolkit on the host. There is no CPU-only mode.
- **Docker** with Compose (the stack is deployed and upgraded through Distr).
- **A Compresr license key** (`cmp_...`) — see [How to get access](#how-to-get-access).

There is no configuration file to fill in: the stack boots with zero customer-supplied settings, and your license key is presented per request, exactly like the hosted API.

## Using it

The proxy exposes the same question-specific compression endpoints as the hosted API, on port `8000`:

- `POST /api/compress/question-specific/` — single request
- `POST /api/compress/question-specific/batch` — up to 100 items
- `POST /api/compress/question-specific/stream` — Server-Sent Events
- `GET /api/compress/question-specific/models`, `GET /api/compress/question-specific/health`

Point the SDK at your deployment and everything else is unchanged — parameters, response shapes, and [error envelope](/docs/api-reference/errors) match the hosted API:

```python
import os
from compresr import CompressionClient

client = CompressionClient(
    api_key=os.environ["COMPRESR_API_KEY"],   # your Compresr license key
    base_url="https://compresr.internal",     # TLS in front of the proxy's port 8000
)

result = client.compress(
    context=long_document,
    query="What is our policy on parental leave?",
    compression_model_name="{{SDK_DEFAULT_MODEL}}",
)
```

**TypeScript**

```typescript
import { CompressionClient } from '@compresr/sdk';

const client = new CompressionClient({
  apiKey: process.env.COMPRESR_API_KEY!,   // your Compresr license key
  baseUrl: 'https://compresr.internal',    // TLS in front of the proxy's port 8000
});

const result = await client.compress({
  context: longDocument,
  query: 'What is our policy on parental leave?',
  compressionModelName: '{{SDK_DEFAULT_MODEL}}',
});
```

**cURL**

```bash
curl -X POST https://compresr.internal/api/compress/question-specific/ \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "<your long document>",
    "query": "What is our policy on parental leave?",
    "compression_model_name": "{{SDK_DEFAULT_MODEL}}"
  }'
```

> **Plain HTTP and the SDKs**
> The proxy itself speaks plain HTTP on port `8000`. The Python SDK refuses a non-loopback `http://` `base_url` with `CompresrError("insecure_base_url")` unless `COMPRESR_ALLOW_INSECURE=1` is set; the TypeScript SDK logs a warning and proceeds. Terminate TLS in front of the proxy (your ingress or reverse proxy) and point the SDKs at that `https://` URL, as above. cURL has no such check.

On first authenticated request, the instance **binds itself to your license key**: the proxy writes a signed identity manifest, and requests with a different `cmp_` key are rejected with `401` from then on. One instance serves one customer.

For operations, the proxy also exposes unauthenticated `GET /health`, `GET /ready`, and a Prometheus-style `GET /metrics` (request counts and latencies by route and status, plus backpressure rejections).

## Security model

1. **The engine has no internet access.** It runs on an internal-only Docker network with no host port and no outbound route; documents processed there cannot leave.
2. **The engine holds no customer secrets.** Your license key exists only in requests to the proxy; the proxy authenticates to the engine with its own short-lived signed tokens, provisioned automatically on first boot.
3. **Instance-to-customer binding.** The first authenticated key claims the instance (signed identity manifest); other keys get `401`.
4. **Weights ship in the image.** Nothing is downloaded at runtime, so the stack works fully air-gapped after installation.
5. **Request content is never persisted.** `context`, `query`, and compressed output are not logged, not queued, and not stored beyond the lifetime of the request.

## What crosses the network

The proxy reports usage events to the Compresr platform every few seconds for billing: the model name, `original_tokens`, `compressed_tokens`, `duration_ms`, and your license key. **Never the content** — no context, no queries, no compressed output.

If the platform is unreachable, a local queue buffers usage for a grace window of roughly 10 minutes; after that, compression endpoints return `503` until connectivity is restored. Health and metrics endpoints keep working throughout.

## Limits

| Limit | Value |
|---|---|
| Concurrent in-flight compressions | 8 (override with `COMPRESR_MAX_INFLIGHT_REQUESTS`); excess requests get `503` + `Retry-After` |
| Per-request time budget | 600 s (then `504`) |
| Batch size | 1–100 items |
| Context size | ~10 MB per context |
| Query length | ≤ 10,000 characters |

As with the hosted API, very short contexts are returned unchanged rather than compressed.

## How to get access

On-prem is available on enterprise plans. Contact [founders@compresr.ai](mailto:founders@compresr.ai) and we'll provision your Distr access and license key, and walk your team through the install.


---

# https://compresr.ai/docs/framework-integration/hermes

> Human-readable page: https://compresr.ai/docs/framework-integration/hermes

[Hermes Agent](https://github.com/NousResearch/hermes-agent) is Nous Research's open-source coding and CLI agent. The Compresr Python SDK ships a first-party Hermes plugin (under `compresr.integrations.hermes`) that compresses the two biggest token sinks in a long agent session — mid-conversation context compaction and large tool outputs — query-aware, using Compresr's compression models. Both features are opt-in and fail open: any API error falls back to Hermes's built-in behavior and never drops context.

> **Opt-in and fail-open**
> The plugin stays inert until you enable a feature and set an API key, and every compression call falls back to Hermes's native behavior on error — it never breaks a tool call or loses a transcript.

## 1. Install

Guided install (prompts for your API key and enables the plugin):

```bash
hermes plugins install Compresr-ai/Compresr-SDK/hermes-plugin
pip install compresr            # in the same environment that runs Hermes
hermes plugins enable compresr
```

The installer asks for `COMPRESR_API_KEY` (get one at [compresr.ai](https://compresr.ai) → Dashboard → API Keys) and writes it to `~/.hermes/.env`. `compresr-sdk login` works too.

Already have the SDK? `pip install compresr` alone ships the plugin via the `hermes_agent.plugins` entry point — just `hermes plugins enable compresr` and set `COMPRESR_API_KEY` yourself.

## 2. Enable in `config.yaml`

The plugin exposes two independent features. Turn on whichever you want in `~/.hermes/config.yaml`:

```yaml
context:
  engine: compresr             # (1) context-engine compression

compresr:
  tool_output_enabled: true    # (2) tool-output compression
  # base_url: https://api.compresr.ai   # for on-prem; omit for cloud default
  # model: latte_v2                      # context-engine model (latte_v1 | latte_v2)
  # target_ratio: 5                      # context-engine override: 0-1 fraction or Nx factor
  # timeout: 60                          # context-engine API timeout (s)
  # tool_output_model: toc_latte_v2
  # tool_output_min_tokens: 1500
  # tool_output_target_ratio: 2.0
  # tool_output_timeout: 30
  # tool_output_max_cache_mb: 256
```

Environment variables take precedence over the `compresr:` block. With an API key set and one of the switches above flipped, everything else is automatic.

## 3. Config reference

| Env var | Config key (`compresr:`) | Default | What it does |
|---|---|---|---|
| `COMPRESR_API_KEY` | — (stored in `~/.hermes/.env`) | — | Compresr API key (`cmp_...`). Without it the plugin stays inert. |
| `COMPRESR_BASE_URL` | `base_url` | `https://api.compresr.ai` | Override for self-hosted / on-prem. |
| `COMPRESR_MODEL` | `model` | `latte_v2` | Context-engine model (`latte_v1` or `latte_v2`). |
| `COMPRESR_TARGET_RATIO` | `target_ratio` | derived from Hermes's `compression.target_ratio` | Context-engine ratio: a `0-1` fraction or an `Nx` factor. |
| `COMPRESR_TIMEOUT` | `timeout` | `60` | Context-engine API timeout (seconds). |
| `COMPRESR_TOOL_OUTPUT_ENABLED` | `tool_output_enabled` | `false` | Master switch for tool-output compression. |
| `COMPRESR_TOOL_OUTPUT_MODEL` | `tool_output_model` | `toc_latte_v2` | Tool-output compression model. |
| `COMPRESR_TOOL_OUTPUT_MIN_TOKENS` | `tool_output_min_tokens` | `1500` | Skip tool outputs smaller than this (avoids latency on trivial results). |
| `COMPRESR_TOOL_OUTPUT_TARGET_RATIO` | `tool_output_target_ratio` | `2.0` | Tool-output `Nx` compression factor. |
| `COMPRESR_TOOL_OUTPUT_TIMEOUT` | `tool_output_timeout` | `30` | Tool-output API timeout (seconds). |
| `COMPRESR_TOOL_OUTPUT_MAX_CACHE_MB` | `tool_output_max_cache_mb` | `256` | Cap on the on-disk recovery cache (LRU-pruned). |

## 4. What gets compressed

The plugin compresses two things, each independently:

**(1) Conversation context.** When Hermes hits its compaction threshold, instead of asking an auxiliary LLM to summarize the mid-conversation window, the Compresr context engine sends it to `latte_v2` query-aware and folds the prior summary back in for continuity. Head and tail turns are protected by Hermes's own pruning; only the middle window is compressed. If the call fails, Hermes aborts compaction and preserves the transcript verbatim (30-second cooldown) — it never inserts a lossy placeholder.

**(2) Tool outputs.** As each tool result arrives, if it is at least `tool_output_min_tokens` (1500 by default), it is compressed with `toc_latte_v2` before it reaches the model. The compression query is derived from the tool's own arguments — a `query`, `pattern`, `command`, `url`, or `file_path` — so the kept content is the part relevant to why the tool was called, not a generic summary. The verbatim original stays recoverable (see below).

## 5. Nothing is lost — tool-output recovery

Every compressed tool output ends with a recovery footer, for example:

```
[compresr:recover] Tool output compressed 4210→1090 tokens (~74% saved). A copy of
the original (with secrets/PII masked) is cached at <path> — recover it with
read_file("<path>") or search_files.
```

The original output — with secrets and PII masked (see [Security](#6-security)) — is written to a content-addressed file under `~/.hermes/cache/compresr/tool-output/`, so the agent can recover the full detail that was summarized away using an ordinary `read_file` or `search_files`; no special API is needed. The cache is LRU-pruned to `tool_output_max_cache_mb`.

## 6. Security

Before anything leaves the process — and before it is written to the recovery cache — the plugin redacts secrets and PII from tool outputs and compression queries (PEM keys, `sk_`/`pk_`/`cmp_` tokens, GitHub/Slack/AWS/GCP tokens, JWTs, URI credentials, and emails). The Compresr `base_url` is SSRF-validated: non-HTTPS endpoints (except localhost), cloud-metadata hosts, and private IP ranges are rejected.

## 7. Observability

- `/compresr` in any session prints live stats: calls, tokens saved, errors, and recoveries.
- Every compression call is tagged `source: integration:hermes`, so usage and spend are attributable in your Compresr dashboard.
- On a permanent configuration error (an invalid model or key), the affected feature disables itself loudly with an actionable message instead of retrying silently.

## When this helps

- **Long agentic sessions:** context compaction and repeated large tool outputs are the two biggest token sinks in a multi-hour Hermes run — this targets both.
- **Big tool outputs:** file reads, shell output, search dumps, and code-execution results are shrunk to what the current step needs, with full recovery on demand.
- **Zero workflow change:** flip a switch and keep working — the agent behaves the same, just with a shorter, cheaper context.

## Related

- [LiteLLM](/docs/framework-integration/litellm): the same query-aware compression as a LiteLLM proxy guardrail.
- [Models](/docs/api-reference/models): `latte_v2` and `toc_latte_v2` parameter semantics.


---

# https://compresr.ai/docs/framework-integration/langchain

> Human-readable page: https://compresr.ai/docs/framework-integration/langchain

`compresr.integrations.langchain` (Python) and `@compresr/sdk/integrations/langchain` (TypeScript) cover the four places LangChain users typically burn tokens: tool outputs in an agent loop, long chat history, the final outbound prompt, and retrieved documents in a `ContextualCompressionRetriever`. Every entry point makes the same Compresr call as the SDK — same auth, same models, same [parameter semantics](/docs/api-reference/models).

> **Python is class-based, TypeScript is factory-based**
> In Python the three middlewares are **classes** (`CompresrToolMiddleware(...)`); in TypeScript they are **factory functions** (`compresrToolMiddleware({...})`). The document compressor `CompresrExtractor` is a class in both. Snake_case kwargs in Python; camelCase options in TypeScript.

## 1. Install

```bash
pip install compresr "langchain>=1.0"
export COMPRESR_API_KEY="cmp_..."
```

**TypeScript**

```bash
npm install @compresr/sdk langchain @langchain/core
export COMPRESR_API_KEY="cmp_..."
```

LangChain 1.0+ is required for the `create_agent` / `createAgent` middleware mechanism; the document compressor (`CompresrExtractor`) works on any version that ships `BaseDocumentCompressor`. LangChain deps ship in the base install — `pip install compresr[langchain]` still resolves for back-compat but adds nothing (extras are no-ops since 2.6.0).

## 2. Compress tool outputs in an agent

`CompresrToolMiddleware` plugs into `create_agent`. Each time a tool returns, the middleware compresses the result against the user's question before it lands in agent state, so the LLM's next reasoning step sees a shorter `ToolMessage`. `api_key=` can be omitted when `COMPRESR_API_KEY` is set or you've run `compresr-sdk login` — see [Authentication](/docs/authentication).

```python
import os
from langchain.agents import create_agent
from langchain_openai import ChatOpenAI
from compresr.integrations.langchain import CompresrToolMiddleware

agent = create_agent(
    model=ChatOpenAI(model="gpt-4o-mini"),
    tools=[search_tool, fetch_url],
    middleware=[
        CompresrToolMiddleware(
            api_key=os.environ["COMPRESR_API_KEY"],
            target_compression_ratio=0.5,    # remove ~50% of tokens
            min_tokens=200,                   # skip tiny outputs
            allow_tools={"search_tool", "fetch_url"},
            query_arg="query",                # pull query out of tool args
        ),
    ],
)

result = agent.invoke({"messages": [{"role": "user", "content": "Why did the sale fall through?"}]})
```

**TypeScript**

```typescript
import { createAgent } from 'langchain';
import { ChatOpenAI } from '@langchain/openai';
import { compresrToolMiddleware } from '@compresr/sdk/integrations/langchain';

const agent = createAgent({
  model: new ChatOpenAI({ model: 'gpt-4o-mini' }),
  tools: [searchTool, fetchUrl],
  middleware: [
    compresrToolMiddleware({
      apiKey: process.env.COMPRESR_API_KEY!,
      targetCompressionRatio: 0.5,
      minTokens: 200,
      allowTools: ['search_tool', 'fetch_url'],
      queryArg: 'query',
    }),
  ],
});

const result = await agent.invoke({
  messages: [{ role: 'user', content: 'Why did the sale fall through?' }],
});
```

## 3. Replace old history with a compressed summary

`CompresrSummarizationMiddleware` is a KV-cache-friendly alternative to LangChain's `SummarizationMiddleware`. When the conversation crosses a token threshold, it compresses everything older than the last N messages into a single `[Earlier conversation summary] ...` `HumanMessage` and leaves recent messages untouched. **No LLM round-trip**: token-level compression. It detects its own prior summary via the `[Earlier conversation summary]` prefix and refuses to re-summarize an already-summarized prefix, so it's safe to run on every turn.

```python
from compresr.integrations.langchain import CompresrSummarizationMiddleware

CompresrSummarizationMiddleware(
    api_key=os.environ["COMPRESR_API_KEY"],
    max_tokens_before_summary=4_000,   # trigger
    messages_to_keep=20,                # recent messages preserved verbatim
    target_compression_ratio=4,         # 4× compression of the older block
)
```

**TypeScript**

```typescript
import { compresrSummarizationMiddleware } from '@compresr/sdk/integrations/langchain';

compresrSummarizationMiddleware({
  apiKey: process.env.COMPRESR_API_KEY!,
  maxTokensBeforeSummary: 4_000,
  messagesToKeep: 20,
  targetCompressionRatio: 4,
});
```

## 4. Cap the outbound prompt with `CompresrPromptMiddleware`

The last-mile budget cap. `CompresrPromptMiddleware` runs in `wrap_model_call` and walks the messages largest-first, compressing just enough to fit under `max_tokens`. It mutates the model request, **not** agent state; the next turn still sees the original messages.

```python
from compresr.integrations.langchain import CompresrPromptMiddleware

CompresrPromptMiddleware(
    api_key=os.environ["COMPRESR_API_KEY"],
    max_tokens=8_000,        # hard ceiling, REQUIRED
    min_tokens=500,           # override default 200 to skip more short messages
    target_compression_ratio=0.5,
)
```

**TypeScript**

```typescript
import { compresrPromptMiddleware } from '@compresr/sdk/integrations/langchain';

compresrPromptMiddleware({
  apiKey: process.env.COMPRESR_API_KEY!,
  maxTokens: 8_000,    // hard ceiling, REQUIRED
  minTokens: 500,      // override default 200 to skip more short messages
  targetCompressionRatio: 0.5,
});
```

The three middlewares cover orthogonal token sources and compose in the same `create_agent` / `createAgent` call:

```python
middleware=[
    CompresrToolMiddleware(api_key=KEY, query_arg="query"),            # per-tool
    CompresrSummarizationMiddleware(api_key=KEY, max_tokens_before_summary=4_000),
    CompresrPromptMiddleware(api_key=KEY, max_tokens=8_000),           # last-mile cap
]
```

## 5. Middleware options

All keyword-only (Python) / options-object fields (TypeScript). The compression knobs share their semantics with the agent client's [compression policy](/docs/sdks/overview#compression-knobs). Shared by all three middlewares:

| Python | TypeScript | Default | Notes |
|---|---|---|---|
| `api_key` | `apiKey` | n/a | Required unless `client` is passed. |
| `client` | `client` | n/a | Pre-built `CompressionClient`: bypasses `api_key`/`base_url`. |
| `base_url` | `baseUrl` | `https://api.compresr.ai` | Override for self-hosted. |
| `compression_model` | `compressionModel` | `"{{SDK_DEFAULT_MODEL}}"` | Query-aware, requires a query. Pass `"{{DEFAULT_MODEL}}"` to opt into the newer model. |
| `target_compression_ratio` | `targetCompressionRatio` | `0.5` | `0 < r ≤ 1` removes that fraction; `r > 1` is Nx target. |
| `coarse` | `coarse` | server default | Paragraph-level vs token-level. |
| `query` | `query` | n/a | Static query overriding everything else. |
| `query_extractor` | `queryExtractor` | n/a | Custom resolution: `(tool_call, messages) -> str` for the tool middleware, `(messages) -> str` for the other two. |
| `on_error` | `onError` | `"passthrough"` | `"raise"` to fail fast in tests. The type is exported as `ErrorPolicy` for typed configs. |

`CompresrToolMiddleware` only:

| Python | TypeScript | Default | Notes |
|---|---|---|---|
| `min_tokens` | `minTokens` | `200` | Skip tool outputs shorter than this. |
| `allow_tools` | `allowTools` | `None` | Whitelist of tool names. Pass `allow_tools` OR `ignore_tools`, not both. |
| `ignore_tools` | `ignoreTools` | `None` | Blacklist. |
| `query_arg` | `queryArg` | n/a | Pull the query directly from a named tool arg. |

`CompresrSummarizationMiddleware` only:

| Python | TypeScript | Default | Notes |
|---|---|---|---|
| `max_tokens_before_summary` (alias `trigger`) | `maxTokensBeforeSummary` (alias `trigger`) | `4000` | Token threshold that triggers a summary. Aliases match LangChain's own `SummarizationMiddleware` naming. |
| `messages_to_keep` (alias `keep`) | `messagesToKeep` (alias `keep`) | `20` | Recent messages preserved verbatim. |
| `token_counter` | `tokenCounter` | SDK `estimate_tokens` | char/4 heuristic, upgrades to tiktoken `cl100k_base` when available. |

`CompresrPromptMiddleware` only:

| Python | TypeScript | Default | Notes |
|---|---|---|---|
| `max_tokens` | `maxTokens` | n/a | **Required.** Hard ceiling for the outbound prompt. |
| `min_tokens` | `minTokens` | `200` | Don't touch messages smaller than this. |
| `token_counter` | `tokenCounter` | SDK `estimate_tokens` | Same char/4 heuristic with tiktoken upgrade. |

## 6. Compress retrieved documents

`CompresrExtractor` is a `BaseDocumentCompressor`: a drop-in replacement for `LLMChainExtractor` inside a `ContextualCompressionRetriever`. Batches all eligible documents into a single Compresr call (up to 100 per batch) — pairs well with high-recall retrieval (`k=20+`). See the [RAG guide](/docs/guides/rag) for the surrounding pipeline.

```python
from langchain.retrievers import ContextualCompressionRetriever
from compresr.integrations.langchain import CompresrExtractor

extractor = CompresrExtractor(
    api_key=os.environ["COMPRESR_API_KEY"],
    target_compression_ratio=0.6,
    min_tokens=120,
    drop_below_min=False,    # set True to filter out docs that come back empty
)

compressing_retriever = ContextualCompressionRetriever(
    base_compressor=extractor,
    base_retriever=vectorstore.as_retriever(search_kwargs={"k": 20}),
)

docs = compressing_retriever.invoke("What changed in the Q3 forecast?")
```

**TypeScript**

```typescript
import { ContextualCompressionRetriever } from 'langchain/retrievers/contextual_compression';
import { CompresrExtractor } from '@compresr/sdk/integrations/langchain';

const extractor = new CompresrExtractor({
  apiKey: process.env.COMPRESR_API_KEY!,
  targetCompressionRatio: 0.6,
  minTokens: 120,
  dropBelowMin: false,
});

const compressingRetriever = new ContextualCompressionRetriever({
  baseCompressor: extractor,
  baseRetriever: vectorstore.asRetriever({ k: 20 }),
});

const docs = await compressingRetriever.invoke("What changed in the Q3 forecast?");
```

The extractor sets `metadata["compresr"] = True` on every document it touched and leaves documents below `min_tokens` unchanged (or filters them out if `drop_below_min=True`).

## 7. Wrap a single tool

If you only need compression on one tool, without an agent middleware, wrap it directly. `wrap_tool_with_compression` / `wrapToolWithCompression` returns a new `StructuredTool` preserving `name`, `description`, `args_schema`, `return_direct`, and error handlers.

```python
from langchain_core.tools import tool
from compresr.integrations.langchain import wrap_tool_with_compression

@tool
def fetch_article(url: str) -> str:
    """Fetch the article body at the given URL."""
    ...

compressed_fetch = wrap_tool_with_compression(
    fetch_article,
    api_key=os.environ["COMPRESR_API_KEY"],
    target_compression_ratio=4,
    query_arg="url",    # or query_extractor=lambda args: args["topic"]
    min_tokens=200,
)
```

**TypeScript**

```typescript
import { tool } from '@langchain/core/tools';
import { z } from 'zod';
import { wrapToolWithCompression } from '@compresr/sdk/integrations/langchain';

const fetchArticle = tool(
  async ({ url }) => fetchArticleBody(url),
  {
    name: 'fetch_article',
    description: 'Fetch the article body at the given URL.',
    schema: z.object({ url: z.string() }),
  },
);

const compressedFetch = wrapToolWithCompression(fetchArticle, {
  apiKey: process.env.COMPRESR_API_KEY!,
  targetCompressionRatio: 4,
  queryArg: 'url',
  minTokens: 200,
});
```

Python raises `TypeError` if the input isn't a `StructuredTool`; wrap a raw function with `@tool` first. TypeScript is more permissive: `wrapToolWithCompression` accepts anything exposing `name` plus `func`, `_call`, or `invoke`.

For the case where you own the tool's source, there is also a decorator form with the same options, `compress_tool_output` / `compressToolOutput`, applied on top of `@tool`:

```python
@compress_tool_output(api_key=os.environ["COMPRESR_API_KEY"], query_arg="url", target_compression_ratio=4)
@tool
def fetch_article(url: str) -> str: ...
```

## Related

- [LangGraph](/docs/framework-integration/langgraph): same middlewares applied inside `StateGraph`, plus node-level helpers, lossy checkpoint serializer, lossy store wrapper, and multi-agent handoff.
- [Models](/docs/api-reference/models): `latte_v2` parameter semantics (`target_compression_ratio`, `coarse`, and friends).
- [RAG guide](/docs/guides/rag): the underlying retrieve → compress → answer pipeline.


---

# https://compresr.ai/docs/framework-integration/langgraph

> Human-readable page: https://compresr.ai/docs/framework-integration/langgraph

LangGraph 1.0+ shares the `create_agent` middleware mechanism with LangChain, so the three agent-level Compresr middlewares are re-exported from `compresr.integrations.langgraph` (Python) and `@compresr/sdk/integrations/langgraph` (TypeScript). On top, the package adds four LangGraph-native components: a state-graph node (`make_compresr_node`), a checkpoint serializer (`CompresrCheckpointSerializer`), a store wrapper (`CompresrStore`), and a multi-agent handoff tool (`compresr_handoff_tool`). Both SDKs ship the full surface with the same defaults — snake_case classes in Python, camelCase factory functions for middleware plus `new`-able classes in TypeScript.

## 1. Install

```bash
pip install "compresr[langgraph]"
export COMPRESR_API_KEY="cmp_..."
```

**TypeScript**

```bash
npm install @compresr/sdk @langchain/langgraph @langchain/core
export COMPRESR_API_KEY="cmp_..."
```

## 2. Agent-level middleware (re-exported)

`CompresrToolMiddleware`, `CompresrSummarizationMiddleware`, and `CompresrPromptMiddleware` work unchanged inside LangGraph's `create_agent`; every keyword argument is identical to the [LangChain reference](/docs/framework-integration/langchain). Only the import path differs:

```python
from compresr.integrations.langgraph import (
    CompresrToolMiddleware,
    CompresrSummarizationMiddleware,
    CompresrPromptMiddleware,
)
```

**TypeScript**

```typescript
import {
  compresrToolMiddleware,
  compresrSummarizationMiddleware,
  compresrPromptMiddleware,
} from '@compresr/sdk/integrations/langgraph';
```

## 3. Compress a state field inside a custom `StateGraph`

For graphs you build with `StateGraph` directly (not the pre-built ReAct shape), use `make_compresr_node`. It returns a callable node that reads `state[context_key]`, compresses it, and writes the shortened text back into the same field.

```python
import os
from typing import TypedDict
from langgraph.graph import StateGraph, START, END
from compresr.integrations.langgraph import make_compresr_node

class ResearchState(TypedDict):
    question: str
    raw_notes: str          # the field we want to compress
    answer: str

compress_notes = make_compresr_node(
    api_key=os.environ["COMPRESR_API_KEY"],
    context_key="raw_notes",          # field to compress in state
    query_key="question",             # field to use as the query
    target_compression_ratio=0.5,
    min_tokens=200,
)

builder = StateGraph(ResearchState)
builder.add_node("gather", gather)
builder.add_node("compress_notes", compress_notes)
builder.add_node("answer", answer)
builder.add_edge(START, "gather")
builder.add_edge("gather", "compress_notes")
builder.add_edge("compress_notes", "answer")
builder.add_edge("answer", END)

graph = builder.compile()
result = graph.invoke({"question": "What changed in the Q3 forecast?", "raw_notes": "", "answer": ""})
```

**TypeScript**

```typescript
import { Annotation } from '@langchain/langgraph';
import { makeCompresrNode } from '@compresr/sdk/integrations/langgraph';

const GraphState = Annotation.Root({
  question: Annotation<string>(),
  raw_notes: Annotation<string>(),
  answer: Annotation<string>(),
});
type State = typeof GraphState.State;

const compressNotes = makeCompresrNode<State>({
  apiKey: process.env.COMPRESR_API_KEY!,
  contextKey: 'raw_notes',
  queryKey: 'question',
  targetCompressionRatio: 0.5,
  minTokens: 200,
});

// Wire compressNotes into your StateGraph like any other node:
// new StateGraph(GraphState).addNode('compress_notes', compressNotes)...
```

The node returns `{context_key: new_text}` when it actually changed the field and `{}` otherwise, so it composes cleanly with LangGraph's diff-merging state updates. Python also exports `compresr_node` as an alias; TypeScript exports `compresrNode`.

### Options

| Python | TypeScript | Required? | Notes |
|---|---|---|---|
| `context_key` | `contextKey` | **Yes** | Name of the state field to compress. |
| `query_key` | `queryKey` | Recommended | Field whose value is used as the query; the compression model is query-aware. |
| `query` | `query` | n/a | Static query overriding extractor and `query_key`. |
| `query_extractor` | `queryExtractor` | n/a | `(state) -> Optional[str]`, custom extraction. |
| `target_compression_ratio` | `targetCompressionRatio` | n/a | Default `0.5`. |
| `min_tokens` | `minTokens` | n/a | Default `200`. Skip if field shorter than this. |
| `coarse` | `coarse` | n/a | Paragraph vs token level; server default. |
| `compression_model` | `compressionModel` | n/a | Default `"latte_v1"`. Pass `"{{DEFAULT_MODEL}}"` for the faster model. |
| `on_error` | `onError` | n/a | `"passthrough"` (default) or `"raise"`. |
| `api_key` / `base_url` / `client` | `apiKey` / `baseUrl` / `client` | n/a | Standard auth knobs. |

## 4. Compress checkpoints: `CompresrCheckpointSerializer`

Long agent state can balloon at-rest storage: Postgres rewrites the whole TOAST row on every checkpoint, Redis pays for every byte. `CompresrCheckpointSerializer` is a `JsonPlusSerializer` subclass that walks the state on `dumps_typed`, finds long strings, and rewrites them as `{"__compresr__": true, "v": ""}` sentinels before encoding. Default `min_tokens` is `500` (vs `200` for middleware); at-rest cost only matters above a certain payload size.

> **Lossy by design**
> This serializer is **one-way**: there is no decompression on `loads_typed`. Anything you compress at write time is what subsequent reads see. Use the `fields` allowlist in production to compress only the keys that carry bulk data (retrieved notes, scratchpads), not control state.

```python
from compresr.integrations.langgraph import CompresrCheckpointSerializer

serializer = CompresrCheckpointSerializer(
    api_key=os.environ["COMPRESR_API_KEY"],
    fields={"raw_notes", "scratchpad"},   # allowlist — recommended for prod
    min_tokens=500,                        # storage tier defaults higher than middleware
    target_compression_ratio=0.5,
)

# Wire into any checkpointer that accepts a custom serializer:
from langgraph.checkpoint.postgres import PostgresSaver
checkpointer = PostgresSaver(conn, serde=serializer)
```

**TypeScript**

```typescript
import { CompresrCheckpointSerializer } from '@compresr/sdk/integrations/langgraph';

const serializer = new CompresrCheckpointSerializer({
  apiKey: process.env.COMPRESR_API_KEY!,
  fields: new Set(['raw_notes', 'scratchpad']),
  minTokens: 500,
  targetCompressionRatio: 0.5,
});

const [tag, encoded] = await serializer.dumpsTyped(state);
const decoded = serializer.loadsTyped([tag, encoded]) as Record<string, unknown>;
// Compressed values come back wrapped: decoded.raw_notes === { __compresr__: true, v: '<compressed>' }
```

## 5. Compress at-rest store entries: `CompresrStore`

`CompresrStore` wraps any `BaseStore` (in-memory, Postgres, Redis, ...) and compresses long string fields on the write side — same lossy contract and same `min_tokens=500` default as the checkpoint serializer. It exposes the full `BaseStore` surface (`put`, `get`, `delete`, `search`, `list_namespaces`, `batch`, plus async siblings); only writes are mutated, everything else delegates straight to the inner store.

```python
from langgraph.store.memory import InMemoryStore
from compresr.integrations.langgraph import CompresrStore

inner = InMemoryStore()
store = CompresrStore(
    inner,
    api_key=os.environ["COMPRESR_API_KEY"],
    fields={"retrieved_text"},
    min_tokens=500,
)

store.put(("users", "alice"), "notes", {"retrieved_text": article, "tag": "physics"})
```

**TypeScript**

```typescript
import { InMemoryStore } from '@langchain/langgraph';
import { CompresrStore } from '@compresr/sdk/integrations/langgraph';

const inner = new InMemoryStore();
const store = new CompresrStore(inner, {
  apiKey: process.env.COMPRESR_API_KEY!,
  fields: new Set(['retrieved_text']),
  minTokens: 500,
});

await store.put(['users', 'alice'], 'notes', { retrieved_text: article, tag: 'physics' });
```

## 6. Compress payloads passed between agents: `compresr_handoff_tool`

In a supervisor / sub-agent architecture, the supervisor often passes a large `context` blob (retrieved docs, prior conversation) to a sub-agent via a handoff tool. `compresr_handoff_tool` returns a LangChain `BaseTool` that compresses `task_description` and `context` before emitting `Command(goto=agent_name, graph=Command.PARENT)`.

```python
from compresr.integrations.langgraph import compresr_handoff_tool

researcher_handoff = compresr_handoff_tool(
    "researcher",                          # positional: target agent name
    api_key=os.environ["COMPRESR_API_KEY"],
    min_tokens=100,
)

# Inside an agent loop:
tool_call = {
    "name": "transfer_to_researcher",
    "args": {
        "task_description": "Summarise the Transformer architecture.",
        "context": full_article,
    },
    "type": "tool_call",
    "id": "tc1",
}
cmd = researcher_handoff.invoke(tool_call)
# cmd.goto == "researcher", cmd.graph == Command.PARENT
# cmd.update["task_description"] and cmd.update["context"] are both compressed.
```

**TypeScript**

```typescript
import { compresrHandoffTool } from '@compresr/sdk/integrations/langgraph';

const researcherHandoff = compresrHandoffTool('researcher', {
  apiKey: process.env.COMPRESR_API_KEY!,
  minTokens: 100,
});

// Invoke with a tool_call object, as in the Python tab:
const cmd = await researcherHandoff.invoke(toolCall);
// cmd.goto === 'researcher'; cmd.update.task_description and cmd.update.context are both compressed.
```

The tool name is auto-generated as `f"transfer_to_{agent_name}"`. Both `task_description` and `context` are compressed with synthesised queries (`"task for {agent_name}"` and the task description itself), so the compression model always has a meaningful query even though it's query-specific.

## Related

- [LangChain](/docs/framework-integration/langchain): full reference for the three middlewares + `CompresrExtractor`.
- [Models](/docs/api-reference/models): `latte_v2` parameter semantics.


---

# https://compresr.ai/docs/framework-integration/litellm

> Human-readable page: https://compresr.ai/docs/framework-integration/litellm

[LiteLLM](https://docs.litellm.ai/) abstracts 100+ providers behind one `completion()` interface. The Compresr Python SDK ships a first-party guardrail under `compresr.integrations.litellm` that plugs into the **LiteLLM proxy server** as a `pre_call` hook, compressing bulky tool/function messages query-aware before the request is forwarded upstream. No application changes are required: any client that calls the proxy gets the savings transparently, and the same config works against OpenAI, Anthropic, Bedrock, Vertex, Ollama, or anything else LiteLLM supports.

## 1. Install and register

In the environment where the LiteLLM proxy runs:

```bash
pip install "compresr[litellm]"
export COMPRESR_API_KEY="cmp_..."
```

LiteLLM's proxy walks `litellm/proxy/guardrails/guardrail_hooks//` to discover guardrails, so until the upstream PR registering `compresr` directly merges, it needs to be registered explicitly. The package ships two console scripts that handle this — pick one:

```bash
# Option A: shim-free wrapper — registers the guardrail in-process, then
# delegates straight to the stock litellm proxy CLI. No files are written.
compresr-litellm --config config.yaml --port 4000

# Option B: install a small discovery shim into the litellm package once,
# then run the proxy however you normally would (litellm --config ...).
install-compresr-shim
install-compresr-shim --uninstall   # to remove it later
```

`COMPRESR_AUTO_INSTALL_SHIM=1` also installs the shim automatically on first `import compresr.integrations.litellm`, if you'd rather not run a separate step. Once the LiteLLM PR merges upstream, both paths become no-ops.

## 2. Enable the guardrail in `config.yaml`

Add a `guardrails:` entry to your LiteLLM proxy config and start the proxy as usual (`litellm --config config.yaml`). Calls that hit a route with `default_on: true` (or that explicitly list `compresr` in their `guardrails:`) run through the guardrail before going upstream.

```yaml
guardrails:
  - guardrail_name: "compresr"
    litellm_params:
      guardrail: compresr
      mode: pre_call
      api_key: os.environ/COMPRESR_API_KEY
      compression_model_name: "{{DEFAULT_MODEL}}"   # guardrail default is {{SDK_DEFAULT_MODEL}}
      # api_base: os.environ/COMPRESR_BASE_URL   # for on-prem; omit for cloud default
      default_on: true
      # ...plus any of the optional knobs below
```

All optional keys live under `litellm_params` (defaults shown):

| Key | Default | What it does |
|---|---|---|
| `api_key` | env `COMPRESR_API_KEY` | Compresr API key (`cmp_...`). Missing both → guardrail fails to load. |
| `api_base` | env `COMPRESR_BASE_URL` or `https://api.compresr.ai` | Override for self-hosted/on-prem. |
| `timeout` | env `COMPRESR_TIMEOUT` or `10.0` (seconds) | HTTP timeout for the compress call. Invalid env values warn-log and fall back to default. |
| `compression_model_name` | `"{{SDK_DEFAULT_MODEL}}"` | Query-specific compression. The guardrail's own default is `{{SDK_DEFAULT_MODEL}}`; set `{{DEFAULT_MODEL}}` explicitly (as the config above does) to get the recommended model. |
| `target_compression_ratio` | `0.5` | `0 < r ≤ 1` removal strength; `r > 1` Nx factor. See [Models](/docs/api-reference/models). |
| `target_ratio_by_role` | unset | Per-role ratio overrides, e.g. `{"system": 0.3, "tool": 0.6}`. Roles not listed fall back to `target_compression_ratio`. |
| `coarse` | `true` | Paragraph-level (faster); set `false` for token-level. |
| `min_chars_to_compress` | `500` | Skip messages shorter than this (avoids latency on trivial messages). |
| `compress_tool_outputs` | `true` | Compress `tool` / `function` result messages. |
| `compress_system` | `false` | Opt-in: compress the system prompt. |
| `compress_history` | `false` | Opt-in: compress prior (non-last) user messages. |
| `compress_last_user` | `false` | Opt-in: also rewrite the last user message (the query sent to Compresr is always the verbatim original). |
| `fail_closed` | `false` | On a Compresr availability error (timeout/connection/5xx), forward the original uncompressed request. Set `true` to raise instead. Validation and auth errors are always raised. |
| `cache_ttl` | `300` (seconds) | How long to cache a compression result in LiteLLM's `DualCache`, keyed by `(content, query, model, ratio, coarse)`. Saves a round-trip when the same tool output repeats in an agent loop. Instance-level only — not overridable per request. |

## 3. What gets compressed by default

The defaults are deliberately conservative — only messages that are usually pure data:

- **Compressed:** `tool` / `function` result messages (search hits, RAG dumps, API responses).
- **Skipped:** `system`, `assistant` (model reasoning, never compressed regardless of flags), prior `user` turns, the last `user` message. Flip the matching `compress_*` flag to opt in.
- **Multimodal messages:** for list-of-parts `content`, the text parts are extracted, compressed, and written back into the first text part; non-text parts (images, audio, files) pass through untouched.

### Per-target intent query

For `tool` / `function` result messages, the query sent to Compresr is **not** the last user message; it's the originating tool call's `name + arguments`, rendered as a single string:

```
web_search: {"query":"2026 electric vehicles over 400 miles EPA range"}
```

The guardrail walks back from the target message through assistant turns, matches by `tool_call_id` (or legacy `function_call.name`), and renders the matched call, falling back to the most recent user message if no assistant turn matches. This produces a much more specific compression query than "what the user asked overall." For `system` / prior-user / last-user compression, the query is the verbatim last user message.

## 4. Per-request overrides

Clients can override any optional config field for a single request by setting `metadata.guardrail_config`:

```python
response = litellm.completion(
    model="gpt-4o-mini",
    messages=messages,
    metadata={
        "guardrail_config": {
            "target_compression_ratio": 4,        # this call only
            "compress_history": True,
        }
    },
)
```

Override-able keys: `compression_model_name`, `target_compression_ratio`, `target_ratio_by_role`, `coarse`, `min_chars_to_compress`, `compress_tool_outputs`, `compress_system`, `compress_history`, `compress_last_user`, and `fail_closed`. `api_key`, `api_base`, `timeout`, and `cache_ttl` are instance-only.

## 5. Observability

After a successful compression pass, the guardrail stashes aggregate stats under `data["metadata"]["compresr_stats"]` — use these for savings telemetry:

```python
{
    "messages_compressed": 1,
    "tokens_saved": 368,
    "compression_ratio": 1.88,
    "duration_ms": 142
}
```

It also adds itself to LiteLLM's standard applied-guardrails header — use it as a quick smoke test that the guardrail fired:

```
x-litellm-applied-guardrails: compresr
```

If Compresr is unavailable and `fail_closed: false` (the default), the guardrail forwards the original uncompressed request and marks the header `compresr:fail_open`, so you can tell "didn't fire" apart from "fired but failed open".

> **Releasing resources**
> If you construct `CompresrGuardrail` directly (outside the proxy's own lifecycle), call `await guardrail.aclose()` to release its underlying HTTP client. The proxy itself doesn't currently call this on shutdown.

## 6. Manual client-side pattern

If you aren't running the proxy and just want to compress a payload before calling `litellm.completion()` directly, use the SDK like you would in any other framework:

```python
import os
import litellm
from compresr import CompressionClient

compresr = CompressionClient(api_key=os.environ["COMPRESR_API_KEY"])

result = compresr.compress(
    context=retrieved_docs,
    query=user_question,
    compression_model_name="{{DEFAULT_MODEL}}",
    target_compression_ratio=0.5,
)

response = litellm.completion(
    model="openai/gpt-4o-mini",  # or anthropic/..., bedrock/..., etc.
    messages=[
        {"role": "system", "content": result.data.compressed_context},
        {"role": "user", "content": user_question},
    ],
)
```

Switching providers is the usual one-line LiteLLM change to `model=`; the Compresr step is unchanged.

## Related

- [Hermes](/docs/framework-integration/hermes): the same query-aware compression built into the Hermes Agent.
- [LLM provider recipes](/docs/framework-integration/llm-providers): the manual pattern called directly against OpenAI, Anthropic, Gemini, or a local Ollama.
- [Models](/docs/api-reference/models): `latte_v2` parameter semantics.


---

# https://compresr.ai/docs/framework-integration/llamaindex

> Human-readable page: https://compresr.ai/docs/framework-integration/llamaindex

`compresr.integrations.llamaindex` (Python) and `@compresr/sdk/integrations/llamaindex` (TypeScript) cover the three places LlamaIndex users typically burn tokens: retrieved nodes feeding a query engine, verbose `FunctionTool` outputs in an agent, and long-running chat history under the Memory API. Every class is a drop-in subclass of the canonical LlamaIndex base, so they slot into existing pipelines without touching the rest of your graph.

## 1. Install

```bash
pip install "compresr[llamaindex]"
# CompresrMemoryBlock needs llama-index-core>=0.12; the extras only pin >=0.11,
# so a fresh install may land on 0.11.x and the memory import will fail —
# bump llama-index-core>=0.12 yourself if so.
export COMPRESR_API_KEY="cmp_..."
```

**TypeScript**

```bash
npm install @compresr/sdk llamaindex
export COMPRESR_API_KEY="cmp_..."
```

## 2. Compress retrieved nodes (query engines)

`CompresrNodePostprocessor` is a `BaseNodePostprocessor`; pass it to `as_query_engine` or any `RetrieverQueryEngine` and the retrieved nodes are compressed query-aware before synthesis ever sees them — retrieve aggressively (`similarity_top_k=20+`) and let compression keep the synthesis prompt tight. See the [RAG guide](/docs/guides/rag) for the surrounding pipeline.

```python
import os
from llama_index.core import VectorStoreIndex
from compresr.integrations.llamaindex import CompresrNodePostprocessor

index = VectorStoreIndex.from_documents(docs)

postprocessor = CompresrNodePostprocessor(
    api_key=os.environ["COMPRESR_API_KEY"],
    target_compression_ratio=4,      # 4× compression, keep ~25%
    min_tokens=120,                   # skip nodes shorter than this
)

query_engine = index.as_query_engine(
    similarity_top_k=20,              # retrieve aggressively
    node_postprocessors=[postprocessor],
)

response = query_engine.query("What changed in the Q3 forecast?")
```

**TypeScript**

```typescript
import { CompresrNodePostprocessor } from '@compresr/sdk/integrations/llamaindex';

const postprocessor = new CompresrNodePostprocessor({
  apiKey: process.env.COMPRESR_API_KEY!,
  targetCompressionRatio: 4,
  minTokens: 120,
});

// Apply to retrieved nodes — either inside a custom query engine or directly:
const out = await postprocessor.postprocessNodes(nodes, { queryStr: 'What changed in the Q3 forecast?' });
// queryStr (camelCase) or query_str (snake_case) both accepted on the bundle.
```

The postprocessor copies each `NodeWithScore` (originals are untouched in the index), partitions by `min_tokens`, batches the eligible nodes into slices of 100 and issues one Compresr call per slice, and writes new text on the copies via `node.set_content()` (falling back to `node.text =` and finally to `metadata["compresr_compressed"]` with a warning). If the query can't be resolved, it logs a warning and passes nodes through unchanged: `query` is required by both models, so no call is made without one.

### Options

| Python | TypeScript | Default | Notes |
|---|---|---|---|
| `compression_model` | `compressionModel` | `"{{SDK_DEFAULT_MODEL}}"` | See [Models](/docs/api-reference/models). |
| `target_compression_ratio` | `targetCompressionRatio` | `0.5` | `0 < r ≤ 1` removal fraction; `r > 1` Nx target. |
| `target_token` | `targetToken` | n/a | Alternative: absolute output budget per node. Overrides ratio (`ratio = max(avg_chunk_tokens / target_token, 1.0)`). When `target_token` > current tokens the ratio pins to `1.0` (no compression is applied). |
| `min_tokens` | `minTokens` | `200` | Skip nodes shorter than this. Estimator uses `tiktoken` when available; otherwise falls back to `chars/4`. |
| `coarse` | `coarse` | `None` | When `None`, defers to backend default (paragraph-level). |
| `query` | `query` | n/a | Override the query string (otherwise pulled from `QueryBundle.query_str`). |
| `on_error` | `onError` | `"passthrough"` | Fail-open by default. |
| `api_key` / `base_url` / `client` | `apiKey` / `baseUrl` / `client` | n/a | Standard auth knobs. |

## 3. Compress an agent tool's output

`wrap_tool_with_compresr` / `wrapToolWithCompresr` takes a `FunctionTool` (or tool-like duck type in TypeScript) and returns a new one whose return value is compressed transparently. The Python version preserves the original tool's `name`, `description`, `fn_schema`, and `return_direct` via `FunctionTool.from_defaults(...)`.

```python
import os
from llama_index.core.tools import FunctionTool
from compresr.integrations.llamaindex import wrap_tool_with_compresr

def fetch_article(url: str) -> str:
    """Fetch the article body at the given URL."""
    ...

tool = FunctionTool.from_defaults(fn=fetch_article)

compressed_tool = wrap_tool_with_compresr(
    tool,
    api_key=os.environ["COMPRESR_API_KEY"],
    target_compression_ratio=4,
    query_arg="url",   # or query_extractor=lambda kwargs: kwargs["topic"]
    min_tokens=200,
)

# Plug compressed_tool into any LlamaIndex agent that takes a list of tools.
```

**TypeScript**

```typescript
import { wrapToolWithCompresr } from '@compresr/sdk/integrations/llamaindex';

const wikiTool = {
  metadata: { name: 'wiki_lookup', description: 'Wikipedia lookup' },
  async call({ query }: { query: string }) {
    return fetchWikipedia(query);
  },
};

const wrapped = wrapToolWithCompresr(wikiTool, {
  apiKey: process.env.COMPRESR_API_KEY!,
  targetCompressionRatio: 4,
  queryArg: 'query',
  minTokens: 200,
});
```

If the wrapped function returns anything other than a string, it's passed through unchanged. Python's wrapper also automatically wires an async branch when the source tool exposes `async_fn`. The TypeScript wrapper currently overrides only `.call` — tools that expose `.acall` bypass compression on the async path.

Query resolution: static `query` wins if set; otherwise `query_extractor(args)` is called if provided; otherwise, and only if you set neither, the wrapper consults `args[query_arg]` when `query_arg` is set (strict: if the named key is missing, the resolver returns `None` and does NOT fall back), or smart-picks from common keys (`query`, `question`, `search_query`, `q`, `prompt`, `input`, `text`) when `query_arg` is unset.

## 4. Compress chat history (Memory API)

`CompresrMemoryBlock` is a `BaseMemoryBlock[str]`; register it on `Memory.from_defaults` and the long-running buffer is compressed via Compresr when the memory layer needs to free tokens — no extra LLM-summary call.

> **Python and TypeScript compress at different moments**
> Python compresses inside `atruncate()`, the Memory layer's truncation hook. TypeScript compresses inside `get()`, since LlamaIndex.TS's `BaseMemoryBlock` has no `atruncate`. The observable output (a single compressed `system` message containing the history) is identical.

```python
import os
from llama_index.core.memory import Memory
from compresr.integrations.llamaindex import CompresrMemoryBlock

memory = Memory.from_defaults(
    token_limit=8_000,
    memory_blocks=[
        CompresrMemoryBlock(
            api_key=os.environ["COMPRESR_API_KEY"],
            target_token=2_000,         # target output-token budget
            min_tokens=200,
        ),
    ],
)

chat_engine = index.as_chat_engine(memory=memory)
response = chat_engine.chat("And what about the regional split?")
```

**TypeScript**

```typescript
import { CompresrMemoryBlock } from '@compresr/sdk/integrations/llamaindex';

const block = new CompresrMemoryBlock({
  apiKey: process.env.COMPRESR_API_KEY!,
  targetToken: 2_000,
  minTokens: 200,
});

// Buffer additions are uncompressed; get() returns a single compressed system message
// once the buffer crosses minTokens.
await block.put([
  { role: 'user', content: 'Tell me about Transformers.' },
  { role: 'assistant', content: longExplanation },
]);
const memoryMessages = await block.get();
```

When the buffer overflows, the block calls Compresr with `{{SDK_DEFAULT_MODEL}}`, using the last `user:` line of the buffer as the query (or `"conversation history"` if no user line exists). Override with `query=` for a fixed query string, or set `target_token` for a target output-token budget (best-effort — translated to a compression ratio, not a hard cap). Other fields: `name` (default `"compresr_compressed_history"`), `priority` (default `2`), `target_compression_ratio` (default `0.5`), `coarse` (default `None`), `on_error` (default `"passthrough"`).

## Related

- [LangChain](/docs/framework-integration/langchain): equivalent middlewares for LangChain 1.0+ agents and retrievers.
- [Models](/docs/api-reference/models): `latte_v2` parameter semantics.
- [RAG guide](/docs/guides/rag): the underlying retrieve → compress → answer pipeline.


---

# https://compresr.ai/docs/framework-integration/llm-providers

> Human-readable page: https://compresr.ai/docs/framework-integration/llm-providers

Compresr doesn't call the provider for you. It returns a shorter string that you drop into the call you already make, in the slot the provider treats as reference material: `system` for OpenAI, Anthropic, and Ollama; `system_instruction` for Gemini. The user's actual question stays in the user-role slot.

## Step 1: compress

Every recipe shares the same first step — build the client once ([SDK overview](/docs/sdks/overview#initialize)) and compress the context against the user's question. The provider snippets below pick up from the variables defined here.

```python
import os
from compresr import CompressionClient

client = CompressionClient(api_key=os.environ["COMPRESR_API_KEY"])

compressed = client.compress(
    context=long_document,
    query=user_question,
    compression_model_name="{{DEFAULT_MODEL}}",
).data.compressed_context
```

**TypeScript**

```typescript
import { CompressionClient } from '@compresr/sdk';

const client = new CompressionClient({
  apiKey: process.env.COMPRESR_API_KEY!,
});

const { data } = await client.compress({
  context: longDocument,
  query: userQuestion,
  compressionModelName: '{{DEFAULT_MODEL}}',
});
```

**cURL**

```bash
export COMPRESR_API_KEY="cmp_your_api_key"

COMPRESSED=$(curl -s -X POST https://api.compresr.ai/api/compress/question-specific/ \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "<long document>",
    "query": "<user question>",
    "compression_model_name": "{{DEFAULT_MODEL}}"
  }' | jq -r '.data.compressed_context')
```

> `CompressResponse.data` is `Optional[CompressResult]` in Python and nullable in TS; on error it is `None`/`null`. The recipes below read `.data.compressed_context` directly and assume the call succeeded. In production, wrap the compress call in `try/except CompresrError` (Py) or check `if (!result.data) throw new Error(result.error ?? 'compress failed')` (TS) before piping the string to the provider. See [errors](/docs/api-reference/errors).

## Step 2: call your provider

Compresr's token counts use tiktoken; every provider tokenizes differently, so the savings *ratio* carries over but the exact billed token numbers diverge. Pick your provider:

The OpenAI chat completions API treats the first `{"role": "system", ...}` message as instructions. Drop the compressed text there; keep the user's actual question in a `user` message.

```python
from openai import OpenAI

openai_client = OpenAI()

response = openai_client.chat.completions.create(
    model="gpt-5",
    messages=[
        {"role": "system", "content": compressed},
        {"role": "user", "content": user_question},
    ],
)
print(response.choices[0].message.content)
```

**TypeScript**

```typescript
import OpenAI from 'openai';

const openai = new OpenAI();

const response = await openai.chat.completions.create({
  model: 'gpt-5',
  messages: [
    { role: 'system', content: data.compressed_context },
    { role: 'user', content: userQuestion },
  ],
});
console.log(response.choices[0].message.content);
```

**cURL**

```bash
curl -X POST https://api.openai.com/v1/chat/completions \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"gpt-5\",
    \"messages\": [
      {\"role\": \"system\", \"content\": $(jq -Rs . <<< \"$COMPRESSED\")},
      {\"role\": \"user\", \"content\": \"<user question>\"}
    ]
  }"
```

OpenAI is the one provider where the counts nearly match: Compresr counts with tiktoken (`cl100k_base` for the GPT-4 family, `o200k_base` for GPT-5), so deltas from OpenAI's billing are usually within 1%.

Anthropic's Messages API takes the system prompt as a top-level `system` field, not as a message in the array. Drop the compressed string into that field; the user's question goes into a single `user` message. Anthropic caches only blocks tagged with `cache_control: {"type": "ephemeral"}` — the snippets set that marker on the compressed `system` block, so reusing the same string across turns lands cache hits on subsequent calls.

```python
from anthropic import Anthropic

anthropic = Anthropic()

response = anthropic.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=1024,
    system=[
        {
            "type": "text",
            "text": compressed,
            "cache_control": {"type": "ephemeral"},
        }
    ],
    messages=[{"role": "user", "content": user_question}],
)
print(response.content[0].text)
```

**TypeScript**

```typescript
import Anthropic from '@anthropic-ai/sdk';

const anthropic = new Anthropic();

const response = await anthropic.messages.create({
  model: 'claude-sonnet-4-5',
  max_tokens: 1024,
  system: [
    {
      type: 'text',
      text: data.compressed_context,
      cache_control: { type: 'ephemeral' },
    },
  ],
  messages: [{ role: 'user', content: userQuestion }],
});
console.log((response.content[0] as { text: string }).text);
```

**cURL**

```bash
curl -X POST https://api.anthropic.com/v1/messages \
  -H "x-api-key: $ANTHROPIC_API_KEY" \
  -H "anthropic-version: 2023-06-01" \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"claude-sonnet-4-5\",
    \"max_tokens\": 1024,
    \"system\": [
      {
        \"type\": \"text\",
        \"text\": $(jq -Rs . <<< \"$COMPRESSED\"),
        \"cache_control\": {\"type\": \"ephemeral\"}
      }
    ],
    \"messages\": [{\"role\": \"user\", \"content\": \"<user question>\"}]
  }"
```

Gemini's `generate_content` API takes reference material in `config.system_instruction`. Put the compressed text there; the user's actual question stays in `contents`.

```python
from google import genai

genai_client = genai.Client()  # reads GEMINI_API_KEY

response = genai_client.models.generate_content(
    model="gemini-2.5-flash",
    contents=user_question,  # string shortcut requires google-genai>=0.3.0
    config={"system_instruction": compressed},
)
print(response.text)
```

**TypeScript**

```typescript
import { GoogleGenAI } from '@google/genai';

const genai = new GoogleGenAI({}); // reads GEMINI_API_KEY

const response = await genai.models.generateContent({
  model: 'gemini-2.5-flash',
  contents: userQuestion,
  config: { systemInstruction: data.compressed_context },
});
console.log(response.text);
```

**cURL**

```bash
curl -X POST "https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash:generateContent?key=$GEMINI_API_KEY" \
  -H "Content-Type: application/json" \
  -d "{
    \"system_instruction\": {\"parts\": [{\"text\": $(jq -Rs . <<< \"$COMPRESSED\")}]},
    \"contents\": [{\"role\": \"user\", \"parts\": [{\"text\": \"<user question>\"}]}]
  }"
```

For exact Gemini counts, call `genai_client.models.count_tokens(model="gemini-2.5-flash", contents=compressed)` on the compressed string.

Ollama runs open-weight models locally over a small HTTP API. The shape is the same as OpenAI's chat completions, `system` + `user` messages, just pointed at `localhost`.

```python
import ollama

response = ollama.chat(
    model="llama3.2",
    messages=[
        {"role": "system", "content": compressed},
        {"role": "user", "content": user_question},
    ],
)
print(response["message"]["content"])
```

**TypeScript**

```typescript
import { Ollama } from 'ollama';

const ollama = new Ollama();

const response = await ollama.chat({
  model: 'llama3.2',
  messages: [
    { role: 'system', content: data.compressed_context },
    { role: 'user', content: userQuestion },
  ],
});
console.log(response.message.content);
```

**cURL**

```bash
curl -X POST http://localhost:11434/api/chat \
  -H "Content-Type: application/json" \
  -d "{
    \"model\": \"llama3.2\",
    \"stream\": false,
    \"messages\": [
      {\"role\": \"system\", \"content\": $(jq -Rs . <<< \"$COMPRESSED\")},
      {\"role\": \"user\", \"content\": \"<user question>\"}
    ]
  }"
```

On local hardware the win is latency as much as cost: shorter prefill means a faster first token on CPU/single-GPU, and less KV-cache pressure leaves room for longer generations on the same machine.

> **Compress retrieved context, not instructions**
> Don't compress short, hand-written system prompts (instructions, output schemas, tool definitions) — Compresr is built for long, semi-redundant retrieved or accumulated context. Parameter semantics: [Models](/docs/api-reference/models). Failure handling for this two-call chain: [Errors](/docs/api-reference/errors).


---

# https://compresr.ai/docs/guides/batch

> Human-readable page: https://compresr.ai/docs/guides/batch

Use `compress_batch` to compress a list of `contexts` in one HTTP call. It fits any batch that shares one compression model - typically RAG re-ranking against a single user question, or independent documents each with their own query. Batches are capped at **100 inputs per request**. Wire-level request and response schemas: [batch endpoint reference](/docs/api-reference/compress-qs-batch).

Python also has `compress_batch_async` with the same signature. TypeScript's `compressBatch` is already `async` and returns a `Promise`.

## Why batch beats N parallel calls

- **One HTTP round trip** instead of N. TLS handshake, request parsing, and response framing happen once.
- **One billing transaction** in usage logs instead of N.
- **Friendlier rate limits.** A batch of 10 counts as one request against your per-second limit, not 10.
- **Simpler retries.** One retry decision for the whole batch instead of partial state across N in-flight requests.

The tradeoff: **a batch is atomic**. If any single context fails validation, the whole request fails with `422 Unprocessable`. For mixed or untrusted inputs, sanitize first, or use parallel single calls so one bad item doesn't sink the rest.

## Shape 1: `contexts` + `queries`

The most common batch shape is RAG re-ranking: several retrieved chunks, one user question, each chunk filtered against that question. Pass `queries` as a single string to apply it to every context. Pass a list the same length as `contexts` to give each context its own question.

```python
# Five chunks retrieved from a vector DB, one user question.
result = client.compress_batch(
    contexts=[chunk_1, chunk_2, chunk_3, chunk_4, chunk_5],
    queries="What did the customer cite as the reason for churn?",
    compression_model_name="{{DEFAULT_MODEL}}",
)

for item in result.data.results:
    print(item.compressed_context)
```

**TypeScript**

```typescript
// Five chunks retrieved from a vector DB, one user question.
const result = await client.compressBatch({
  contexts: [chunk1, chunk2, chunk3, chunk4, chunk5],
  queries: 'What did the customer cite as the reason for churn?',
  compressionModelName: '{{DEFAULT_MODEL}}',
});

for (const item of result.data.results) {
  console.log(item.compressed_context);
}
```

**cURL**

```bash
# contexts/queries is SDK-only sugar. On the wire there is one shape:
# inputs=[{context, query}, ...] — repeat the query per item.
curl -X POST https://api.compresr.ai/api/compress/question-specific/batch \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": [
      {"context": "<chunk 1>", "query": "What did the customer cite as the reason for churn?"},
      {"context": "<chunk 2>", "query": "What did the customer cite as the reason for churn?"},
      {"context": "<chunk 3>", "query": "What did the customer cite as the reason for churn?"},
      {"context": "<chunk 4>", "query": "What did the customer cite as the reason for churn?"},
      {"context": "<chunk 5>", "query": "What did the customer cite as the reason for churn?"}
    ],
    "compression_model_name": "{{DEFAULT_MODEL}}"
  }'
```

## Shape 2: `inputs=[{context, query}, ...]`

The SDK also accepts the wire-level form: an `inputs` list of `{context, query}` pairs. Use `contexts` + `queries` for re-ranking shapes (many contexts, one query, or aligned lists). Use `inputs=` when each item has its own query, or your data already comes as pairs. The two forms are mutually exclusive - pass one or the other.

```python
result = client.compress_batch(
    inputs=[
        {"context": contract_a, "query": "Who signed the contract?"},
        {"context": contract_b, "query": "When was the renewal date?"},
        {"context": contract_c, "query": "What was the agreed unit price?"},
    ],
    compression_model_name="{{DEFAULT_MODEL}}",
)
```

**TypeScript**

```typescript
const result = await client.compressBatch({
  inputs: [
    { context: contractA, query: 'Who signed the contract?' },
    { context: contractB, query: 'When was the renewal date?' },
    { context: contractC, query: 'What was the agreed unit price?' },
  ],
  compressionModelName: '{{DEFAULT_MODEL}}',
});
```

**cURL**

```bash
curl -X POST https://api.compresr.ai/api/compress/question-specific/batch \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": [
      {"context": "<contract A>", "query": "Who signed the contract?"},
      {"context": "<contract B>", "query": "When was the renewal date?"},
      {"context": "<contract C>", "query": "What was the agreed unit price?"}
    ],
    "compression_model_name": "{{DEFAULT_MODEL}}"
  }'
```

> **Client-side validation before any HTTP call**
> The SDK raises `ValidationError` client-side, before the request goes out, in two cases: (1) you pass both `contexts` and `inputs`, or neither; (2) `queries` is a list whose length doesn't match `contexts`. There is no `422` round trip. Fix the request; don't retry.

## Response shape

The response is one envelope with a `results` array and aggregate totals: `count`, plus `total_original_tokens`, `total_compressed_tokens`, and `total_tokens_saved` summed across the batch. `average_compression_ratio` is the mean of per-item ratios, not a sum. Each `results` entry has the same fields as a single `compress` call, except `target_compression_ratio`. That one is request-level and applies to every item.

Full response schema: [batch endpoint reference](/docs/api-reference/compress-qs-batch); batch size and cost limits: [pricing estimate endpoint](/docs/api-reference/pricing-estimate).

## Batch-level knobs

Most knobs on single `compress` also work on `compress_batch` and apply to every item: `target_compression_ratio`, `coarse`, plus the `latte_v2` adaptive trio `dynamic`, `dynamic_min_ratio`, `dynamic_max_ratio`. `heuristic_chunking` and `disable_placeholders` are **not** accepted on the batch endpoint. Semantics match the single-call form; see [`POST /compress/question-specific/`](/docs/api-reference/compress-qs). To vary a knob per item, or to use the single-call-only knobs, use parallel single calls.

## Scaling beyond a single batch

If you have more contexts than fit in one batch, send several batch requests in parallel instead of one giant one. A few dozen contexts per batch, with 4-8 batches in flight, usually works well. Each request stays cheap to retry while you still amortize HTTP overhead.

## When to use batch vs parallel single calls

| Use batch | Use parallel `compress()` calls |
|---|---|
| Same model + ratio across all contexts | Per-context model or ratio |
| Want one rate-limit-friendly request | Want independent retry per context |
| OK with all-or-nothing failure | Want partial results if some calls fail |
| Optimizing for cost and latency at scale | Long contexts where total payload would exceed batch size limits |


---

# https://compresr.ai/docs/guides/coarse-mode

> Human-readable page: https://compresr.ai/docs/guides/coarse-mode

You can trade precision for speed with the `coarse` flag. Both `latte_v1` and `latte_v2` score at paragraph granularity by default: that's coarse mode. Set `coarse=false` for the slower, more precise token-level pass.

## How coarse vs fine compares

In coarse mode the model keeps or drops whole paragraphs based on how relevant each one is to your `query`. Setting `coarse=false` switches to token-level scoring. A fine pass can keep one relevant sentence from an otherwise irrelevant paragraph, at the cost of much more work per request. Coarse mode can't slice into a paragraph; it keeps it or drops it.

`query` and `target_compression_ratio` work the same in both modes. Only the unit of decision changes.

## When the default works

- **Very long contexts.** Well past the typical RAG chunk, token-level scoring's wall-clock cost grows quickly. That's a rough field observation, not a hard threshold.
- **Structured documents where block boundaries already carry meaning.** Markdown, knowledge-base articles, transcripts, code files. Keeping a whole markdown section is usually what you wanted anyway.
- **Pre-filtering before a fine pass.** Coarse-cut a 100K-token corpus down to 20K, then run a fine pass over the cut for precision.
- **Real-time pipelines.** Agent loops and chat backends where compression sits on the latency critical path and a few percent precision is an acceptable trade.

## When to set `coarse=false`

- **Short or medium inputs.** The coarse savings don't matter at this scale, so take the precision.
- **You need specific sentences from a mostly irrelevant paragraph.** That's what token-level scoring is for.
- **High-stakes RAG.** A single relevant span shouldn't be dropped along with its paragraph.
- **Output feeds a noise-sensitive downstream model.** Tighter, sentence-level output leaves less for the next model to filter.

## Example

You don't need to mention `coarse` at all; paragraph-level scoring is the default. The SDK client field defaults to `None`/`undefined` and is omitted from the wire. The backend then applies its own default of `True`, paragraph-level. Passing `coarse=True` explicitly is the same as omitting it.

```python
result = client.compress(
    context=very_long_document,
    query="What is our policy on parental leave?",
    compression_model_name="latte_v1",
    target_compression_ratio=0.7,
)

print(result.data.compressed_context)
print(f"Took {result.data.duration_ms} ms")
```

**TypeScript**

```typescript
const result = await client.compress({
  context: veryLongDocument,
  query: 'What is our policy on parental leave?',
  compressionModelName: 'latte_v1',
  targetCompressionRatio: 0.7,
});

console.log(result.data.compressed_context);
console.log(`Took ${result.data.duration_ms} ms`);
```

**cURL**

```bash
curl -X POST https://api.compresr.ai/api/compress/question-specific/ \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "<very long document>",
    "query": "What is our policy on parental leave?",
    "compression_model_name": "latte_v1",
    "target_compression_ratio": 0.7
  }'
```

To switch to token-level precision, set `coarse=False` (Python), `coarse: false` (TypeScript), or `"coarse": false` in the JSON body:

```python
result = client.compress(
    context=mid_length_document,
    query="What is our policy on parental leave?",
    compression_model_name="latte_v1",
    coarse=False,
    target_compression_ratio=0.7,
)

print(result.data.compressed_context)
```

**TypeScript**

```typescript
const result = await client.compress({
  context: midLengthDocument,
  query: 'What is our policy on parental leave?',
  compressionModelName: 'latte_v1',
  coarse: false,
  targetCompressionRatio: 0.7,
});

console.log(result.data.compressed_context);
```

**cURL**

```bash
curl -X POST https://api.compresr.ai/api/compress/question-specific/ \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "<mid-length document>",
    "query": "What is our policy on parental leave?",
    "compression_model_name": "latte_v1",
    "coarse": false,
    "target_compression_ratio": 0.7
  }'
```

## The tradeoff at a glance

| Dimension | Coarse mode (default) | Fine mode (`coarse=false`) |
|---|---|---|
| Unit of decision | Paragraph | Token / sentence |
| Latency on long inputs | Baseline | Higher |
| Cost | Lower | Higher |
| Precision | Paragraph-level | Sentence-level |
| Steering signal (`query`) | Honored | Honored |
| Response shape | Identical schema in both modes | Identical schema in both modes |

## Relationship to `heuristic_chunking`

`heuristic_chunking` is a separate boolean on the same request. It changes the chunker (paragraphs / code blocks / fixed-size), not the scoring granularity. The two knobs are independent: `coarse` picks the unit of decision, `heuristic_chunking` picks how the input is split before scoring.

## Where `coarse` is accepted

You can pass `coarse` on `compress`, `compress_stream`, `compress_batch`, `compress_async`, and the middleware compression policy (`compression={"coarse": ...}`). It works on both `latte_v1` and `latte_v2`; swap `compression_model_name` and the flag behaves the same:

```python
result = client.compress(
    context=mid_length_document,
    query="What is our policy on parental leave?",
    compression_model_name="latte_v2",
    coarse=False,
    target_compression_ratio=0.7,
)
```

**TypeScript**

```typescript
const result = await client.compress({
  context: midLengthDocument,
  query: 'What is our policy on parental leave?',
  compressionModelName: 'latte_v2',
  coarse: false,
  targetCompressionRatio: 0.7,
});
```

**cURL**

```bash
curl -X POST https://api.compresr.ai/api/compress/question-specific/ \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "<mid-length document>",
    "query": "What is our policy on parental leave?",
    "compression_model_name": "latte_v2",
    "coarse": false,
    "target_compression_ratio": 0.7
  }'
```

> `coarse` is a plain optional bool, spelled the same in Python, TypeScript, and JSON; multi-word fields follow the standard camelCase→snake_case transform (e.g. `heuristicChunking` → `heuristic_chunking`).


---

# https://compresr.ai/docs/guides/query-specific

> Human-readable page: https://compresr.ai/docs/guides/query-specific

Query-specific compression shortens your `context` while keeping the parts that answer your `query`. You send both; the API drops everything irrelevant. Both `latte_v1` and `latte_v2` support this. Both require a non-empty `query`; the API rejects an empty or missing one with `422`.

> **SDK default is {{SDK_DEFAULT_MODEL}}**
> The SDK uses `{{SDK_DEFAULT_MODEL}}` unless you set `compression_model_name`. Samples on this page pass `compression_model_name="{{DEFAULT_MODEL}}"` explicitly. Drop that argument and you're back on `{{SDK_DEFAULT_MODEL}}`.

## What query-specific means

With a `query`, you tell the compressor what to keep instead of letting it guess. A query-agnostic compressor keeps whatever looks generally interesting. The latte models instead score spans of the context against your `query` and keep the ones with the highest signal. You usually already know what the next step does with the context: answer a question, extract a field, summarize against a goal. That knowledge is what the compressor needs.

The `query` is not a prompt the model answers. It only steers what gets kept. The output is still your original context, just shorter.

## What makes a good query

Write the `query` like a one-line task description for the next step. Concrete beats vague. It should tell the model *what to preserve*.

- **Good**: `"What was the project's Q3 churn rate?"`
- **Good**: `"Find the renewal date in the customer contract."`
- **Good**: `"Summarize the customer's stated reasons for canceling."`
- **Weak**: `"churn"` - single keyword, no intent
- **Weak**: `"important info"` - no signal at all
- **Rejected**: `""` - both SDK schemas enforce `min_length=1` and raise `ValidationError` before the request goes out. To skip query-aware behavior on `latte_v2`, omit the field entirely. In Python, don't pass `query=`; in TS, don't set `query:`. On `latte_v1`, omitting `query` fails server-side with a 422 that the SDK maps to `ValidationError`.

The query does not need a question mark. The last user message in a chat, the current sub-goal in an agent loop, or the task summary in a job spec all work.

## A working example

The input below is an earnings summary covering several figures. The query asks for one: the full-year guidance. The compressor keeps the guidance sentence and drops the rest.

> **Response nullability**
> On error, `result.data` is `None` in Python / `null` in TypeScript. Check it before reading `compressed_context`. The samples below skip that check for readability.

```python
context = """
Acme reported third-quarter revenue of $312 million, up 18% year over year,
driven primarily by growth in the subscription segment. Gross margin expanded
to 71% from 68% a year ago. Operating expenses rose 9%, reflecting continued
investment in R&D. The company ended the quarter with $1.1 billion in cash
and equivalents. Management raised full-year revenue guidance to $1.25 billion
and announced a $200 million share buyback program.
""".strip()

result = client.compress(
    context=context,
    query="What is the full-year revenue guidance?",
    compression_model_name="{{DEFAULT_MODEL}}",
)

print(result.data.compressed_context)
# -> "...Management raised full-year revenue guidance to $1.25 billion..."
print(f"{result.data.original_tokens} -> {result.data.compressed_tokens} tokens")
```

**TypeScript**

```typescript
const context =
  'Acme reported third-quarter revenue of $312 million, up 18% year over year, ' +
  'driven primarily by growth in the subscription segment. Gross margin expanded ' +
  'to 71% from 68% a year ago. Operating expenses rose 9%, reflecting continued ' +
  'investment in R&D. The company ended the quarter with $1.1 billion in cash ' +
  'and equivalents. Management raised full-year revenue guidance to $1.25 billion ' +
  'and announced a $200 million share buyback program.';

const result = await client.compress({
  context,
  query: 'What is the full-year revenue guidance?',
  compressionModelName: '{{DEFAULT_MODEL}}',
});

console.log(result.data.compressed_context);
console.log(`${result.data.original_tokens} -> ${result.data.compressed_tokens} tokens`);
```

**cURL**

```bash
curl -X POST https://api.compresr.ai/api/compress/question-specific/ \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "Acme reported third-quarter revenue of $312 million, up 18% year over year, driven primarily by growth in the subscription segment. Gross margin expanded to 71% from 68% a year ago. Operating expenses rose 9%, reflecting continued investment in R&D. The company ended the quarter with $1.1 billion in cash and equivalents. Management raised full-year revenue guidance to $1.25 billion and announced a $200 million share buyback program.",
    "query": "What is the full-year revenue guidance?",
    "compression_model_name": "{{DEFAULT_MODEL}}"
  }'
```

Exact numbers vary by run.

## Tuning compression strength

Set `target_compression_ratio` to control how much the model cuts. Values in `0 < r <= 1` are removal-strength; values `> 1` are Nx-factor mode. See the [Models reference](/docs/api-reference/models#target_compression_ratio) for full semantics, defaults, and bounds.

```python
result = client.compress(
    context=context,
    query="What is the full-year revenue guidance?",
    compression_model_name="{{DEFAULT_MODEL}}",
    target_compression_ratio=0.5,   # gentle: remove about half
)
```

**TypeScript**

```typescript
const result = await client.compress({
  context,
  query: 'What is the full-year revenue guidance?',
  compressionModelName: '{{DEFAULT_MODEL}}',
  targetCompressionRatio: 0.5,   // gentle: remove about half
});
```

**cURL**

```bash
curl -X POST https://api.compresr.ai/api/compress/question-specific/ \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "...",
    "query": "What is the full-year revenue guidance?",
    "compression_model_name": "{{DEFAULT_MODEL}}",
    "target_compression_ratio": 0.5
  }'
```

On the earnings summary, `0.5` keeps supporting figures alongside the guidance sentence — the revenue growth, the buyback. That suits answers that need some grounding. A setting like `4` targets roughly a quarter of the original and collapses the output to the guidance sentence itself. That suits extractive Q&A with tight token budgets.

Truncation slices off whatever happens to be at the end. Query-specific compression instead keeps the *coherent spans* that answer the query, wherever they sit in the document.

> **Building a RAG pipeline?**
> The [RAG guide](/docs/guides/rag) shows the full retrieve, compress, and LLM flow with LangChain, LlamaIndex, and direct vector-DB examples.


---

# https://compresr.ai/docs/guides/rag

> Human-readable page: https://compresr.ai/docs/guides/rag

Add Compresr between retrieval and your LLM call to shrink retrieved chunks against the user's question. A typical RAG pipeline retrieves top-k chunks, joins them, and puts them in the prompt. Compresr adds one step in the middle: it scores the chunks against the question and passes only the parts that answer it to the LLM. Your vector DB, embedding model, and LLM stay the same.

## Pipeline shape

```
User query
  -> Embed
  -> Vector DB retrieval (top-k chunks)
  -> Compresr (query = user query, context = joined chunks)
  -> LLM call with the compressed context
  -> Answer
```

The `query` you pass to Compresr is the string the user typed. The `context` is the retrieved chunks, joined. With batch, pass them as a list instead.

## With LangChain: `CompresrExtractor`

Use `CompresrExtractor`, the SDK's first-party `BaseDocumentCompressor`. Drop it into `ContextualCompressionRetriever` and retrieved documents are compressed against the retrieval query in one batched Compresr call.

```python
import os
from langchain.retrievers import ContextualCompressionRetriever
from compresr.integrations.langchain import CompresrExtractor

extractor = CompresrExtractor(
    api_key=os.environ["COMPRESR_API_KEY"],
    target_compression_ratio=0.5,
    min_tokens=120,            # skip tiny chunks; default 200
)

compressing_retriever = ContextualCompressionRetriever(
    base_compressor=extractor,
    base_retriever=vectorstore.as_retriever(search_kwargs={"k": 20}),
)

docs = compressing_retriever.invoke("What was Q3 churn?")
```

**TypeScript**

```typescript
import { ContextualCompressionRetriever } from 'langchain/retrievers/contextual_compression';
import { CompresrExtractor } from '@compresr/sdk/integrations/langchain';

const extractor = new CompresrExtractor({
  apiKey: process.env.COMPRESR_API_KEY!,
  targetCompressionRatio: 0.5,
  minTokens: 120,
});

const compressingRetriever = new ContextualCompressionRetriever({
  baseCompressor: extractor,
  baseRetriever: vectorstore.asRetriever({ k: 20 }),
});

const docs = await compressingRetriever.invoke('What was Q3 churn?');
```

**cURL**

```bash
# LangChain is a framework; cURL is not. Retrieve, join, then call the batch
# endpoint directly:
curl -X POST https://api.compresr.ai/api/compress/question-specific/batch \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": [
      {"context": "<chunk 1>", "query": "What was Q3 churn?"},
      {"context": "<chunk 2>", "query": "What was Q3 churn?"},
      {"context": "<chunk 3>", "query": "What was Q3 churn?"}
    ],
    "compression_model_name": "{{DEFAULT_MODEL}}",
    "target_compression_ratio": 0.5
  }'
```

For LangChain LCEL / `RunnableSequence` setups, `CompresrExtractor` also implements `acompress_documents`. The [LangChain integration page](/docs/framework-integration/langchain) has the full option table (`drop_below_min`, batching behavior, metadata tagging) and the agent middlewares for tool-output, history, and outbound-prompt compression.

## With LlamaIndex: `CompresrNodePostprocessor`

Use `CompresrNodePostprocessor`, a `BaseNodePostprocessor`. Pass it to `as_query_engine` or any `RetrieverQueryEngine`. Retrieved nodes are compressed against the query before synthesis sees them. Originals in the index stay untouched.

```python
import os
from llama_index.core import VectorStoreIndex
from compresr.integrations.llamaindex import CompresrNodePostprocessor

postprocessor = CompresrNodePostprocessor(
    api_key=os.environ["COMPRESR_API_KEY"],
    target_compression_ratio=0.5,
    min_tokens=120,            # skip tiny chunks; default 200
)

query_engine = index.as_query_engine(
    similarity_top_k=20,                    # retrieve aggressively
    node_postprocessors=[postprocessor],
)

response = query_engine.query("What was Q3 churn?")
```

**TypeScript**

```typescript
import { CompresrNodePostprocessor } from '@compresr/sdk/integrations/llamaindex';

const postprocessor = new CompresrNodePostprocessor({
  apiKey: process.env.COMPRESR_API_KEY!,
  targetCompressionRatio: 0.5,
  minTokens: 120,
});

const query = 'What was Q3 churn?';
const retriever = index.asRetriever({ similarityTopK: 20 });
const nodes = await retriever.retrieve(query);
const compressed = await postprocessor.postprocessNodes(nodes, { queryStr: query });

// Pass compressed nodes to your synthesizer / query engine.
```

**cURL**

```bash
# LlamaIndex is a framework. With cURL, retrieve outside, join the chunks, and
# call the batch endpoint:
curl -X POST https://api.compresr.ai/api/compress/question-specific/batch \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": [
      {"context": "<node 1>", "query": "What was Q3 churn?"},
      {"context": "<node 2>", "query": "What was Q3 churn?"}
    ],
    "compression_model_name": "{{DEFAULT_MODEL}}",
    "target_compression_ratio": 0.5
  }'
```

The [LlamaIndex integration page](/docs/framework-integration/llamaindex) has the full option table (including `target_token` for absolute per-node token budgets), plus tool wrapping and memory-block compression.

## Direct vector DB (no framework)

You can call a vector DB directly (pgvector, Pinecone, Qdrant, Chroma, Weaviate) and keep the same shape: retrieve, compress, pass to LLM. Use `compress_batch` / `compressBatch` to filter each chunk against the same question in one HTTP round trip. That's cheaper than `N` parallel `compress()` calls.

```python
# pgvector example.
question = "What was Q3 churn?"
embedding = embed(question)

rows = db.execute(
    "SELECT content FROM docs ORDER BY embedding <-> %s LIMIT 5",
    (embedding,),
).fetchall()

result = compresr.compress_batch(
    contexts=[row["content"] for row in rows],
    queries=question,                        # broadcast to all
    compression_model_name="{{DEFAULT_MODEL}}",
    target_compression_ratio=0.5,
)

joined = "\n\n".join(item.compressed_context for item in result.data.results)

answer = openai.chat.completions.create(
    model="gpt-5",
    messages=[
        {"role": "system", "content": joined},
        {"role": "user", "content": question},
    ],
)
```

**TypeScript**

```typescript
// Pinecone example.
const question = 'What was Q3 churn?';
const embedding = await embed(question);

const { matches } = await pinecone
  .index('docs')
  .query({ vector: embedding, topK: 5, includeMetadata: true });

const result = await compresr.compressBatch({
  contexts: matches.map((m) => m.metadata!.content as string),
  queries: question,
  compressionModelName: '{{DEFAULT_MODEL}}',
  targetCompressionRatio: 0.5,
});

if (!result.data) throw new Error('compression failed');
const joined = result.data.results
  .map((item) => item.compressed_context)
  .join('\n\n');

const answer = await openai.chat.completions.create({
  model: 'gpt-5',
  messages: [
    { role: 'system', content: joined },
    { role: 'user', content: question },
  ],
});
```

**cURL**

```bash
# After you have retrieved your chunks (in another script or tool):
curl -X POST https://api.compresr.ai/api/compress/question-specific/batch \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "inputs": [
      {"context": "<chunk 1>", "query": "What was Q3 churn?"},
      {"context": "<chunk 2>", "query": "What was Q3 churn?"},
      {"context": "<chunk 3>", "query": "What was Q3 churn?"}
    ],
    "compression_model_name": "{{DEFAULT_MODEL}}",
    "target_compression_ratio": 0.5
  }'
```

For hybrid search or multi-hop RAG where each chunk has its own question, pass `queries` as a list the same length as `contexts`, or use the `inputs=[{"context": ..., "query": ...}, ...]` pair form. `contexts` / `queries` is SDK sugar; the wire format is always `inputs`, which is what the cURL tabs send. Input shapes, atomicity, and the async twin are covered in the [batch guide](/docs/guides/batch).

## Tips

- **Use the framework-native integration when you can.** `CompresrExtractor` (LangChain) and `CompresrNodePostprocessor` (LlamaIndex) handle batching, node cloning, `min_tokens` partitioning, and error policy for you. Manual calls work, but you rebuild that wiring.
- **Pick a ratio that matches your token budget.** Lighter ratios (around `0.3-0.5`) keep enough surrounding context for citation-style or extractive Q&A. Heavier settings (`0.7+`, or Nx like `4`) work for summarization-style answers. Start at `0.5`, measure on a held-out set, then tune. Ratio semantics and bounds: [Models reference](/docs/api-reference/models#target_compression_ratio).
- **Put the compressed text in the system message.** It's reference material, not a turn in the conversation. Keep the user's question in the user message. See the [LLM provider recipes](/docs/framework-integration/llm-providers) for the exact slot per provider.

## When NOT to compress

- **Tiny contexts.** Under ~500 tokens, the API call overhead isn't worth the savings. Raise `min_tokens` to skip them automatically.
- **Tightly structured retrieval results** like JSON, tool outputs, and schemas. Compresr is tuned for prose. For structured payloads, turn on `heuristic_chunking` and `disable_placeholders` to reduce field-loss risk, or route tool outputs through `CompresrToolMiddleware` instead.

## Variable chunk sizes: `latte_v2` + dynamic ratios

If your chunk sizes vary a lot, say short FAQ snippets next to long PDF pages, one `target_compression_ratio` over- or under-compresses at the extremes. Set `compression_model_name="latte_v2"` and pass `dynamic=True` with `dynamic_min_ratio` / `dynamic_max_ratio`. The model then picks a per-chunk ratio inside that window based on input size. `latte_v2`-only; see [Models reference](/docs/api-reference/models).

## Related

- [LangChain integration](/docs/framework-integration/langchain): full middleware + extractor reference.
- [LlamaIndex integration](/docs/framework-integration/llamaindex): postprocessor, tool wrapper, memory block.
- [Batch compression](/docs/guides/batch): full reference for `compress_batch`.


---

# https://compresr.ai/docs/guides/streaming

> Human-readable page: https://compresr.ai/docs/guides/streaming

Use `compress_stream` / `compressStream` to consume `/api/compress/question-specific/stream` as an iterator. The endpoint returns Server-Sent Events: `data:` frames only, no `event:`, `id:`, or `retry:` lines.

**What streaming does today:** the server compresses the full input, then sends the complete output as one final event. There is no token-by-token delivery yet, so streaming has **no time-to-first-token advantage** over the plain `compress` call. The iterator interface exists for forward-compatibility: if incremental delivery ships, code written against it won't need to change.

## The wire protocol

On success the server sends one content event, then closes the connection:

```text
data: {"content": "<full compressed output>", "done": true}
```

If compression fails or exceeds the 120-second server-side budget *after* the stream opens, the server sends an error event, then closes:

```text
data: {"error": "Stream timeout exceeded"}
```

Failures *before* the stream opens (bad key, validation, rate limit) arrive as normal HTTP error responses with the standard [error envelope](/docs/api-reference/errors).

## Consuming the stream

Iterate, concatenate `content` across chunks, and stop on `done`. Both SDKs yield each content event as `{content, done: false}`, then a final synthetic `{content: "", done: true}` when the stream closes. That pattern keeps working whether the output arrives in one event (today) or many (future).

```python
buffer = []
for chunk in client.compress_stream(
    context=long_document,
    query="What was the project's Q3 churn rate?",
    compression_model_name="{{DEFAULT_MODEL}}",
):
    buffer.append(chunk.content)
    if chunk.done:
        break

compressed = "".join(buffer)
```

**TypeScript**

```typescript
const buffer: string[] = [];
for await (const chunk of client.compressStream({
  context: longDocument,
  query: "What was the project's Q3 churn rate?",
  compressionModelName: '{{DEFAULT_MODEL}}',
})) {
  buffer.push(chunk.content);
  if (chunk.done) break;
}

const compressed = buffer.join('');
```

**cURL**

```bash
curl -N -X POST https://api.compresr.ai/api/compress/question-specific/stream \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "<your long document>",
    "query": "What was the project'"'"'s Q3 churn rate?",
    "compression_model_name": "{{DEFAULT_MODEL}}"
  }' \
| while IFS= read -r line; do
    # Frames arrive as: data: {"content": "...", "done": true}
    # or, on failure:   data: {"error": "..."}
    # The server closes the connection after the final frame.
    case "$line" in
      data:*) echo "${line#data: }" ;;
    esac
  done
```

The stream carries text only. For the token-accounting metadata (`original_tokens`, `compressed_tokens`, `tokens_saved`, `actual_compression_ratio`, `duration_ms`), use the non-streaming [`compress`](/docs/api-reference/compress-qs) endpoint.

## Handling errors

Handle errors by when they happen:

- **Before the stream opens.** The SDK's initial status check maps `4xx` / `5xx` responses to typed errors (`RateLimitError`, `AuthenticationError`, `CompresrError`, ...). Catch these to back off or re-auth.
- **After the stream opens.** A server-side failure arrives as an `error` event on the wire. The current SDK iterators skip events without a `content` field. So a mid-stream server error ends the iteration with whatever content arrived, possibly none, instead of raising. If an empty result is unacceptable, treat an empty buffer as a failure and retry. Transport-level drops do raise: `httpx.HTTPError` in Python, `ConnectionError` in TypeScript.
- **Retries.** `RetryConfig` applies only to the initial connect, never to a mid-stream failure. `retry_after` / `retryAfter` is nullable; guard the sleep.

```python
import httpx
from compresr.exceptions import RateLimitError, CompresrError

buffer = []
try:
    for chunk in client.compress_stream(
        context=long_document,
        query="What was the project's Q3 churn rate?",
        compression_model_name="{{DEFAULT_MODEL}}",
    ):
        buffer.append(chunk.content)
        if chunk.done:
            break
except RateLimitError as e:
    # 429 mapped before the stream opened. retry_after may be None.
    time.sleep(e.retry_after or 1)
except CompresrError as e:
    # Other typed errors raised before the stream opened.
    logger.exception("Request rejected: %s", e)
except httpx.HTTPError as e:
    # Transport drop mid-stream; buffer holds partial output.
    logger.exception("Stream interrupted: %s", e)

compressed = "".join(buffer)
if not compressed:
    # Covers server-side mid-stream errors, which end the
    # iteration silently instead of raising.
    raise RuntimeError("Compression stream returned no output")
```

**TypeScript**

```typescript
import { RateLimitError, CompresrError, ConnectionError } from '@compresr/sdk';

const buffer: string[] = [];
try {
  for await (const chunk of client.compressStream({
    context: longDocument,
    query: "What was the project's Q3 churn rate?",
    compressionModelName: '{{DEFAULT_MODEL}}',
  })) {
    buffer.push(chunk.content);
    if (chunk.done) break;
  }
} catch (error: unknown) {
  if (error instanceof RateLimitError) {
    // 429 mapped before the stream opened. retryAfter may be undefined.
    await new Promise((r) => setTimeout(r, (error.retryAfter ?? 1) * 1000));
  } else if (error instanceof ConnectionError) {
    // Transport drop mid-stream; buffer holds partial output.
    console.error('Stream interrupted', error);
  } else if (error instanceof CompresrError) {
    console.error('Request rejected', error);
  } else {
    throw error;
  }
}

const compressed = buffer.join('');
if (!compressed) {
  // Covers server-side mid-stream errors, which end the
  // iteration silently instead of throwing.
  throw new Error('Compression stream returned no output');
}
```

**cURL**

```bash
# 4xx / 5xx failures arrive as a normal HTTP response BEFORE any data frames,
# so -i prints them as headers + JSON body. After the stream opens, a failure
# arrives as a data: {"error": "..."} frame and the connection closes.
curl -N -i -X POST https://api.compresr.ai/api/compress/question-specific/stream \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "<your long document>",
    "query": "What was the project'"'"'s Q3 churn rate?",
    "compression_model_name": "{{DEFAULT_MODEL}}"
  }'
```

## SDK caveats

- **Python is sync-only.** `compress_stream` returns a `Generator`, not an `AsyncGenerator`. There is no `compress_stream_async`. On FastAPI or asyncio, offload the loop with `asyncio.to_thread`, or use the TypeScript SDK, which is async-native.
- **The TypeScript timeout covers the whole stream.** The client arms one `AbortController` for `timeout` ms from connect. It is not idle-based. Long compressions may need a larger `timeout` at client construction.
- **Agent-layer streaming is not wired up.** `client.messages.stream(...)` and `client.chat.completions.stream(...)` on the TS agent interfaces throw `CompresrError('streaming not yet implemented', code: 'not_implemented')`. The Python interfaces don't expose `.stream(...)` at all. `compress_stream` / `compressStream` is the working streaming path.

## When to use it

Use the plain [`compress`](/docs/api-reference/compress-qs) call for almost everything today. It returns the same output **plus** the token-accounting metadata, and it works fully with typed errors and `RetryConfig`. Use the streaming endpoint only if you're deliberately writing against the iterator interface for forward-compatibility. Use [`compress_batch`](/docs/api-reference/compress-qs-batch) when you have many contexts to send at once.


---

# https://compresr.ai/docs/guides/web-search

> Human-readable page: https://compresr.ai/docs/guides/web-search

Use `WebSearchTool` to add web search to your agent loop with automatic compression. It returns a real LangChain `BaseTool`. Pass it to any of the [agent client](/docs/sdks/overview#agent-client) interfaces and `CompresrToolMiddleware` compresses search results before they re-enter the model context. No extra wiring.

Three backends ship out of the box:

| Provider | Strengths | Bring your own key |
|---|---|---|
| **Tavily** (default) | LLM-tuned results, strong on recency and citations, `allowed_domains` / `blocked_domains` filtering, `max_results` forwarded verbatim | `TAVILY_API_KEY` |
| **Brave** | Generalist web index, independent crawler, native domain filtering uses Goggles (out of scope for v1) | `BRAVE_SEARCH_API_KEY` (fallback: `BRAVE_API_KEY`) |
| **AgentCore** | Amazon Bedrock web search over MCP + Cognito OAuth, `max_results` clamped 1..25 | Cognito client credentials (see [AgentCore](#agentcore) below) |

## Why not Anthropic / OpenAI / Gemini server search?

Provider-native server search tools (Anthropic's `web_search_20250305`, OpenAI's `web_search_preview`, Gemini's `google_search`) **run server-side and return opaque or encrypted content** that Compresr cannot read or compress. The model sees the search content but the SDK doesn't, so there is nothing to intercept.

`WebSearchTool` calls the search API client-side instead. Plaintext results reach the SDK first, pass through `CompresrToolMiddleware`, and arrive at the model compressed. This is the only path where Compresr's compression actually fires for web search results.

## Quick start: Tavily

```python
import os
from compresr import CompressionClient, WebSearchTool

client = CompressionClient(
    api_key=os.environ["COMPRESR_API_KEY"],
    llm="anthropic",
    llm_api_key=os.environ["ANTHROPIC_API_KEY"],
    compression={"target_compression_ratio": 0.5, "min_tokens": 300},
)

tavily = WebSearchTool.tavily(
    api_key=os.environ["TAVILY_API_KEY"],
    max_results=5,
)

msg = client.messages.create(
    model="claude-haiku-4-5",
    max_tokens=512,
    messages=[
        {"role": "user", "content": "What are this week's top stories on AI agents?"},
    ],
    tools=[tavily],
)

print(msg.content[0].text)
```

**TypeScript**

```typescript
import { CompressionClient, WebSearchTool } from '@compresr/sdk';

const client = new CompressionClient({
  apiKey: process.env.COMPRESR_API_KEY!,
  llm: 'anthropic',
  llmApiKey: process.env.ANTHROPIC_API_KEY!,
  compression: { targetCompressionRatio: 0.5, minTokens: 300 },
});

const tavily = await WebSearchTool.tavily({
  apiKey: process.env.TAVILY_API_KEY!,
  maxResults: 5,
});

const msg = await client.messages.create({
  model: 'claude-haiku-4-5',
  maxTokens: 512,
  messages: [
    { role: 'user', content: "What are this week's top stories on AI agents?" },
  ],
  tools: [tavily],
});

console.log(msg.content[0]);
```

The model decides when to call `tavily_search` (or `brave_search` / `agentcore_web_search`). The SDK calls the underlying API, and the middleware compresses the result before it re-enters the model context. Mechanics: [How compression interacts](#how-compression-interacts-with-web-search) below.

## Construction options

All three backends take a small, deliberately narrow option set. Anything provider-specific (e.g. Tavily `search_depth`) goes through the `extra` / `**extra` escape hatch.

### Tavily

```python
WebSearchTool.tavily(
    api_key=os.environ["TAVILY_API_KEY"],   # optional; falls back to TAVILY_API_KEY env
    max_results=5,                           # forwarded verbatim to Tavily
    allowed_domains=["nytimes.com", "reuters.com"],
    blocked_domains=["example.com"],
    # Anything else (e.g. search_depth="advanced") goes via **extra.
)
```

**TypeScript**

```typescript
await WebSearchTool.tavily({
  apiKey: process.env.TAVILY_API_KEY!,     // required; the TS factory does not read TAVILY_API_KEY itself
  maxResults: 5,                           // forwarded verbatim to Tavily
  allowedDomains: ['nytimes.com', 'reuters.com'],
  blockedDomains: ['example.com'],
  // Anything else (e.g. searchDepth: 'advanced') goes via extra: { ... }.
});
```

### Brave

```python
WebSearchTool.brave(
    api_key=os.environ["BRAVE_SEARCH_API_KEY"],   # optional; env fallback tries BRAVE_SEARCH_API_KEY first, then BRAVE_API_KEY
    max_results=5,
)
```

**TypeScript**

```typescript
await WebSearchTool.brave({
  apiKey: process.env.BRAVE_SEARCH_API_KEY!, // env fallback (Python only): BRAVE_SEARCH_API_KEY, then BRAVE_API_KEY
  maxResults: 5,
});
```

> **No native domain filtering on Brave**
> Brave doesn't expose native `allowed_domains` / `blocked_domains`. Its filtering flows through Goggles, which is out of scope for v1. Python emits a `UserWarning` and proceeds; TypeScript silently ignores the kwargs. Use Tavily if you need real domain filtering.

> **TypeScript Brave + Claude tool_use**
> `WebSearchTool.brave` in TypeScript returns the upstream `BraveSearch` directly, which ships without an `args_schema`. Anthropic (Claude) tool_use calls can then fail with `missing 1 required positional argument: 'query'`. Workaround: wrap it in your own `tool()` factory that declares a `query: z.string()` schema, or use Tavily or AgentCore. Python wraps Brave with an explicit schema, so this is a TypeScript-only pitfall.

### AgentCore

AgentCore is Amazon Bedrock web search over an MCP streamable-HTTP session, authenticated with a Cognito OAuth client-credentials handshake. Runtime deps are optional. Install the extra: `pip install compresr[agentcore]` (Python) or `npm install @modelcontextprotocol/sdk` (TypeScript).

```python
WebSearchTool.agentcore(
    gateway_url=os.environ["AGENTCORE_GATEWAY_MCP_URL"],
    cognito_token_url=os.environ["AGENTCORE_COGNITO_TOKEN_URL"],
    client_id=os.environ["AGENTCORE_COGNITO_CLIENT_ID"],
    client_secret=os.environ["AGENTCORE_COGNITO_CLIENT_SECRET"],
    scope=os.environ["AGENTCORE_COGNITO_SCOPE"],
    max_results=5,                     # clamped 1..25
    # allowed_domains / blocked_domains accepted for signature parity —
    # AgentCore has no native domain filter, emits a UserWarning.
)
```

**TypeScript**

```typescript
await WebSearchTool.agentcore({
  gatewayUrl: process.env.AGENTCORE_GATEWAY_MCP_URL!,
  cognitoTokenUrl: process.env.AGENTCORE_COGNITO_TOKEN_URL!,
  clientId: process.env.AGENTCORE_COGNITO_CLIENT_ID!,
  clientSecret: process.env.AGENTCORE_COGNITO_CLIENT_SECRET!,
  scope: process.env.AGENTCORE_COGNITO_SCOPE!,
  maxResults: 5,                       // clamped 1..25
  // allowedDomains / blockedDomains: accepted for parity, silently ignored.
});
```

Each config field resolves from the explicit argument first, then two env vars in order:

| Field | Primary env var | Fallback env var |
|---|---|---|
| `gateway_url` / `gatewayUrl` | `AGENTCORE_GATEWAY_MCP_URL` | `GATEWAY_MCP_URL` |
| `cognito_token_url` / `cognitoTokenUrl` | `AGENTCORE_COGNITO_TOKEN_URL` | `COGNITO_TOKEN_URL` |
| `client_id` / `clientId` | `AGENTCORE_COGNITO_CLIENT_ID` | `COGNITO_CLIENT_ID` |
| `client_secret` / `clientSecret` | `AGENTCORE_COGNITO_CLIENT_SECRET` | `COGNITO_CLIENT_SECRET` |
| `scope` | `AGENTCORE_COGNITO_SCOPE` | `COGNITO_SCOPE` |

Runtime behaviour:

- **HTTPS-only URLs (TypeScript).** `gatewayUrl` and `cognitoTokenUrl` must start with `https://`. A plaintext URL throws `CompresrError('invalid_config')` before any credential leaves the process.
- **`max_results` clamp.** TypeScript enforces `1..25` at build time. Python passes the value through to the underlying client, which applies the same bound.
- **Bearer-token cache.** The Cognito token is minted once and cached on the shared client. A `401` from the gateway triggers exactly one automatic re-mint before the call is retried.
- **Timeouts and response cap.** Cognito requests time out at 30s, tool calls at 30s, and responses are hard-capped at 1 MB.

### Constructor form

Use the classmethod factories above when you can. The constructor form is equivalent and useful when the provider is a runtime variable:

```python
# Equivalent to WebSearchTool.tavily(...)
WebSearchTool(provider="tavily", api_key=..., max_results=5)

# provider="brave" and provider="agentcore" work the same way, taking
# the same options as their factory counterparts.
```

**TypeScript**

```typescript
import { createWebSearchTool } from '@compresr/sdk';

// Equivalent to WebSearchTool.tavily({ ... })
await createWebSearchTool('tavily', { apiKey: ..., maxResults: 5 });

// 'brave' and 'agentcore' work the same way, taking the same options
// as their factory counterparts.
```

## How compression interacts with web search

`CompresrToolMiddleware` runs on every tool return inside the agent loop:

1. The model emits a tool call (e.g. `tavily_search({ query: "..." })`).
2. The SDK invokes the provider and normalises the response via `_flatten_search_results` / `flattenSearchResults`. The raw JSON becomes blank-line-separated plain-text blocks of `title\nurl\ncontent`. This is the shape `latte_v1` can actually compress; JSON input is a no-op for compression.
3. The middleware checks the serialized string length. At or below `compression.min_tokens` (default `200`), it's forwarded untouched. Above that, the middleware calls `client.compress(...)` with the result body as `context` and the user's last message as `query`.
4. The compressed body replaces the original in the agent state. The model never sees the uncompressed search results.

Two consequences:

- **Tune `compression.min_tokens` to your search backend.** Tavily returns long content per hit and easily exceeds `200` tokens. Brave returns shorter snippets, so you may want to lower `min_tokens` to e.g. `100` to catch them.
- **Compression uses the user's intent as the `query`**, not the model's tool-call query string. So compression stays aimed at what the user actually asked, even when the model rewords the search.

## Errors & failure modes

| Scenario | Behaviour | Mitigation |
|---|---|---|
| Missing peer dep (`langchain-tavily` / `langchain-community` / `@modelcontextprotocol/sdk`) | Python raises `ImportError` naming the extra. TypeScript raises `CompresrError` code `missing_peer_dependency`. | Install the extra: `pip install compresr[agents-tavily\|agents-brave\|agentcore]` or `npm install @langchain/tavily @langchain/community @modelcontextprotocol/sdk`. |
| Missing API key or config | Python raises `ValueError` naming the arg and env-var fallback. TypeScript raises `CompresrError` with code `missing_api_key` (Brave), `missing_config` (AgentCore), or `invalid_config` (AgentCore non-HTTPS URL), plus `invalid_provider` for the constructor form. | Pass the arg or set the env var. |
| Search-provider 401 / 403 / rate limit at call time | Python: the provider's exception propagates inside the agent loop. TypeScript: raised as `CompresrError` — for AgentCore specifically, codes `agentcore_auth_error`, `agentcore_no_tool`, `agentcore_bad_response`, `agentcore_tool_error`. | Rotate the key or back off / lower `max_results`. |
| Compresr backend down while the middleware fires | Returns the original (uncompressed) search result by default (`on_error="passthrough"` on the policy). | Set `compression={"on_error": "raise"}` if you want to fail loudly instead. |
| No tools fired (model answered without searching) | No compression call. The middleware only runs on tool returns. | Encourage the model to search via the user prompt or a `system` instruction. |

## Research facade: `client.research`

Use `client.research` when you want web search, snippet compression, and citation extraction in one call, instead of wiring `WebSearchTool` into an agent loop by hand. It requires a client constructed with `llm=`; accessing it otherwise raises `CompresrError`. `client.research.run(question)` runs the full search-and-synthesize loop, up to `max_steps`. `client.research.search(question)` is the same loop capped at 2 steps for quick lookups.

```python
from compresr import CompressionClient

client = CompressionClient(
    api_key=os.environ["COMPRESR_API_KEY"],
    llm="anthropic:claude-haiku-4-5",
    llm_api_key=os.environ["ANTHROPIC_API_KEY"],
)

result = client.research.run(
    "What did the JWST discover about early galaxies in 2025?",
    search="tavily",           # or "brave", or a preconstructed WebSearchTool
)

print(result.answer)
for c in result.citations:
    print(f"- {c.title or c.url}\n  {c.url}")
```

**TypeScript**

```typescript
import { CompressionClient } from '@compresr/sdk';

const client = new CompressionClient({
  apiKey: process.env.COMPRESR_API_KEY!,
  llm: 'anthropic:claude-haiku-4-5',
  llmApiKey: process.env.ANTHROPIC_API_KEY!,
});

const result = await client.research.run(
  'What did the JWST discover about early galaxies in 2025?',
  { search: 'tavily' },        // or 'brave', or a preconstructed WebSearchTool
);

console.log(result.answer);
for (const c of result.citations) {
  console.log(`- ${c.title ?? c.url}\n  ${c.url}`);
}
```

Options (Python names; TypeScript uses camelCase):

| Option | Default | Effect |
|---|---|---|
| `search` | `"tavily"` | `"tavily"` or `"brave"` (env-var key fallbacks apply), or a preconstructed `WebSearchTool`, which is the only way to use AgentCore here. |
| `max_steps` | `10` | Upper bound on search / synthesize iterations. `.search()` overrides this to 2. |
| `model` | client `llm` | Override the client-level model for this call. |
| `compress_snippets` | `true` | Route each search snippet through the compression API before it enters the LLM context. |
| `compression_model` | `"{{SDK_DEFAULT_MODEL}}"` | Which model compresses the snippets. |
| `min_compress_tokens` | `100` | Skip compression for snippets shorter than this. |
| `max_context_tokens` | `120,000` | Hard ceiling on total tokens across all compressed snippets before synthesis. |
| `system_prompt` | built-in | Override the research system prompt. |

`ResearchResult` fields: `answer`, `explanation`, `confidence`, `text`, `citations` (each with `url`, `title`, `snippet`), `trajectory`, `usage` (`input_tokens`, `output_tokens`, `cache_read_tokens`, `cache_creation_tokens`, `calls`, `search_calls`), `raw`.

## Next steps

- [Agent client](/docs/sdks/overview#agent-client): the full client surface (Anthropic shape, OpenAI shape, native).
- [LangChain integration](/docs/framework-integration/langchain): lower-level `CompresrToolMiddleware`, `wrap_tool_with_compression`, and `CompresrExtractor` for custom retrieval pipelines.
- [LangGraph integration](/docs/framework-integration/langgraph): drop-in compression node for `StateGraph`-style agents.


---

# https://compresr.ai/docs/resources/changelog

> Human-readable page: https://compresr.ai/docs/resources/changelog

This changelog will list user-visible Compresr SDK and API changes. Each entry will be dated and tied to a specific release. For the broader product changelog, see [/changelog](/changelog).

> **Coming soon**
> The technical changelog will be populated once the release pipeline emits versioned tags. Until then, see the package pages for tag-level release notes: [pypi.org/project/compresr](https://pypi.org/project/compresr) for the Python SDK and [npmjs.com/package/@compresr/sdk](https://www.npmjs.com/package/@compresr/sdk) for the TypeScript SDK.


---

# https://compresr.ai/docs/sdks/overview

> Human-readable page: https://compresr.ai/docs/sdks/overview

`compresr` (Python 3.9+) and `@compresr/sdk` (Node 20+) wrap the REST API with typed methods, auth, retries, and async support. Every snippet is shown in all three flavors — pick a tab once and the choice sticks across the docs.

This page covers what only the SDKs have: installation, the client and its options, the core `compress` call, the agent client, and the exception types.

## Install

```bash
pip install compresr

# or with your package manager of choice:
uv add compresr
poetry add compresr
```

**TypeScript**

```bash
npm install @compresr/sdk

# or with your package manager of choice:
yarn add @compresr/sdk
pnpm add @compresr/sdk
bun add @compresr/sdk
```

**cURL**

```bash
# Nothing to install — set the key once:
export COMPRESR_API_KEY="cmp_your_api_key"
```

The [agent client](#agent-client) ships in the base install (Python `compresr ≥ 2.8.2`, TypeScript `@compresr/sdk ≥ 1.5`); the old `compresr[agents]` extras still work as no-op aliases.

## Initialize

Construct the client once and reuse it — it keeps an internal connection pool. See [Authentication](/docs/authentication) for the three ways to supply the key.

```python
import os
from compresr import CompressionClient

client = CompressionClient(api_key=os.environ["COMPRESR_API_KEY"])
```

**TypeScript**

```typescript
import { CompressionClient } from '@compresr/sdk';

const client = new CompressionClient({ apiKey: process.env.COMPRESR_API_KEY! });
```

**cURL**

```bash
# No client object — every call is a POST with the X-API-Key header.
```

### Constructor options

The table follows the language toggle — flip any snippet above to TypeScript and it re-renders with the camelCase spellings, types, and defaults.

| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `api_key` | str \| None | no | Python also falls back to `~/.compresr/credentials` — see [Authentication](/docs/authentication) for all three ways to supply the key. |
| `base_url` | str \| None | no | Endpoint override for self-hosted / [on-prem](/docs/deployment/on-prem). Python refuses a non-HTTPS URL (`CompresrError("insecure_base_url")`) except loopback, unless `COMPRESR_ALLOW_INSECURE=1` is set; TypeScript logs a warning and proceeds. |
| `timeout` | int (seconds) | no | Per-call timeout. **Python takes seconds, TypeScript milliseconds.** |
| `retry_config` | RetryConfig \| None | no | `RetryConfig` (importable from the package root): retries `429`/`503` with exponential backoff, respects `Retry-After`. |
| `llm` | str \| None | no | Provider for the [agent client](#agent-client), e.g. `"anthropic"` or `"anthropic:claude-haiku-4-5"` to pin a default model. |
| `llm_api_key` | str \| None | no | Key for the LLM provider. Falls back to `ANTHROPIC_API_KEY` / `OPENAI_API_KEY` / `GOOGLE_API_KEY` depending on `llm`. |
| `compression` | dict \| CompressionPolicy \| None | no | Middleware policy for agent tool outputs — see [Compression knobs](#compression-knobs). |
| `enable_prompt_cache` | bool | no | Provider-side prompt caching for the agent client (Anthropic `cache_control`, OpenAI `prompt_cache_key`; no-op on Gemini). |
| `prompt_cache_ttl` | "5m" \| "1h" | no | Anthropic cache TTL. On OpenAI, `"1h"` maps to `prompt_cache_retention: "24h"`. |
| `prompt_cache_min_messages` | int | no | Skip caching for very short conversations. |
| `openai_prompt_cache_key` | str \| None | no | Explicit OpenAI `prompt_cache_key`; omitted → provider defaults apply. |
| `llm_http_client / llm_http_async_client` | httpx.Client / httpx.AsyncClient | no | Python only: custom `httpx` client(s) for the downstream LLM call — corporate proxies, custom CA bundles, mTLS. |

## compress

Single-request compression. Parameters map 1:1 to the API — the canonical parameter table, defaults, and `target_compression_ratio` semantics live in the [Models reference](/docs/api-reference/models). TypeScript accepts camelCase (`targetCompressionRatio`); the wire and all response fields stay snake_case.

```python
result = client.compress(
    context=long_document,
    query="What is the diameter of JWST's primary mirror?",
    compression_model_name="{{DEFAULT_MODEL}}",
    target_compression_ratio=0.5,
)

print(result.data.compressed_context)
print(f"{result.data.original_tokens} -> {result.data.compressed_tokens} tokens")
```

**TypeScript**

```typescript
const result = await client.compress({
  context: longDocument,
  query: "What is the diameter of JWST's primary mirror?",
  compressionModelName: '{{DEFAULT_MODEL}}',
  targetCompressionRatio: 0.5,
});

console.log(result.data.compressed_context);
```

**cURL**

```bash
curl -X POST https://api.compresr.ai/api/compress/question-specific/ \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "<your long document>",
    "query": "What is the diameter of JWST'"'"'s primary mirror?",
    "compression_model_name": "{{DEFAULT_MODEL}}",
    "target_compression_ratio": 0.5
  }'
```

### Response

| Field | Type | Description |
| --- | --- | --- |
| `data` | object |  |
| `data.compressed_context` | string | The compressed text, ready to drop into your prompt. |
| `data.original_tokens` | integer | Token count of the input context (tiktoken cl100k). |
| `data.compressed_tokens` | integer | Token count of the compressed output. |
| `data.tokens_saved` | integer | original_tokens − compressed_tokens. |
| `data.actual_compression_ratio` | number | Fraction of input tokens removed (0–1), regardless of how the target was expressed. |
| `data.duration_ms` | integer | Server-side wall-clock time for the compression pass. |

## Async

Python ships `compress_async` — same parameters as `compress`, awaitable. Use `async with` (or call `await client.aclose()` when done) so the connection pool closes cleanly. TypeScript is async-native throughout; there are no `_async` variants — fire calls concurrently with `Promise.all`.

```python
import asyncio
from compresr import CompressionClient

async def main():
    # Key picked up from COMPRESR_API_KEY.
    async with CompressionClient() as client:
        results = await asyncio.gather(
            client.compress_async(
                context=contract_a,
                query="What penalties apply on late delivery?",
                compression_model_name="{{DEFAULT_MODEL}}",
            ),
            client.compress_async(
                context=contract_b,
                query="What is the termination notice period?",
                compression_model_name="{{DEFAULT_MODEL}}",
            ),
        )
        for r in results:
            print(r.data.compressed_context)

asyncio.run(main())
```

**TypeScript**

```typescript
import { CompressionClient } from '@compresr/sdk';

// Key picked up from COMPRESR_API_KEY.
const client = new CompressionClient({});

const [a, b] = await Promise.all([
  client.compress({
    context: contractA,
    query: 'What penalties apply on late delivery?',
    compressionModelName: '{{DEFAULT_MODEL}}',
  }),
  client.compress({
    context: contractB,
    query: 'What is the termination notice period?',
    compressionModelName: '{{DEFAULT_MODEL}}',
  }),
]);

console.log(a.data.compressed_context, b.data.compressed_context);
```

**cURL**

```bash
# The API is synchronous per request — parallelize at the shell level,
# e.g. two curl calls with & and wait, or xargs -P.
```

If every row shares one model and ratio, prefer a single [`compress_batch`](/docs/guides/batch) call over N concurrent singles — one round trip, one rate-limit hit. Streaming is sync-only in Python; `compress_batch_async` is covered in the [batch guide](/docs/guides/batch).

## Everything else

| Capability | Methods | Where it's documented |
|---|---|---|
| Streaming | `compress_stream` / `compressStream` | [Streaming guide](/docs/guides/streaming) — wire protocol, error handling, why plain `compress` usually wins today |
| Batch (≤ 100 contexts) | `compress_batch` / `compressBatch` | [Batch guide](/docs/guides/batch) — input shapes, atomicity, aggregates; endpoint: [`/batch`](/docs/api-reference/compress-qs-batch) |
| Web search tools | `WebSearchTool` (Tavily / Brave / AgentCore) | [Web search guide](/docs/guides/web-search) |
| Research | `client.research.run` / `.search` | [Web search guide § Research facade](/docs/guides/web-search#research-facade-clientresearch) |
| Framework middleware | `CompresrToolMiddleware`, extractors, postprocessors | [LangChain](/docs/framework-integration/langchain), [LangGraph](/docs/framework-integration/langgraph), [LiteLLM](/docs/framework-integration/litellm), [LlamaIndex](/docs/framework-integration/llamaindex) |
| Parameters & models | `target_compression_ratio`, `coarse`, `dynamic`, … | [Models reference](/docs/api-reference/models) — the canonical table; [coarse mode](/docs/guides/coarse-mode) |
| Wire contract | envelope, status codes, rate limits | [API conventions](/docs/api-reference/conventions), [Error codes](/docs/api-reference/errors), [Rate limits](/docs/api-reference/rate-limits) |

## Agent client

Construct the client with `llm=` and you get an agent surface: three call shapes that auto-compress every tool output above `min_tokens` before the LLM sees it (LangChain 1.0 `create_agent` + `CompresrToolMiddleware` under the hood). There is no cURL equivalent — the compression the middleware fires is the same `/api/compress/question-specific/` endpoint.

```python
import os
from compresr import CompressionClient, WebSearchTool

client = CompressionClient(
    api_key=os.environ["COMPRESR_API_KEY"],
    llm="anthropic",                     # or "openai", "google_genai"; pin a default
    llm_api_key=os.environ["ANTHROPIC_API_KEY"],  # model with "anthropic:claude-haiku-4-5"
    compression={"target_compression_ratio": 0.5, "min_tokens": 300},
)

tavily = WebSearchTool.tavily(api_key=os.environ["TAVILY_API_KEY"], max_results=3)
messages = [{"role": "user", "content": "What's the latest AI news?"}]

# Anthropic shape — duck-types anthropic.types.Message
msg = client.messages.create(model="claude-haiku-4-5", max_tokens=512,
                             messages=messages, tools=[tavily])
# OpenAI shape — duck-types openai ChatCompletion
completion = client.chat.completions.create(model="gpt-4o-mini",
                                            messages=messages, tools=[tavily])
# Native shape — returns NormalizedResult (.text, .tool_uses, .citations, .usage)
result = client.run(prompt="What's the latest AI news?", model="claude-haiku-4-5",
                    tools=[tavily], max_tokens=512)
```

**TypeScript**

```typescript
import { CompressionClient, WebSearchTool } from '@compresr/sdk';

const client = new CompressionClient({
  apiKey: process.env.COMPRESR_API_KEY!,
  llm: 'anthropic',                    // or 'openai', 'google_genai'; pin a default
  llmApiKey: process.env.ANTHROPIC_API_KEY!,  // model with 'anthropic:claude-haiku-4-5'
  compression: { targetCompressionRatio: 0.5, minTokens: 300 },
});

const tavily = await WebSearchTool.tavily({ apiKey: process.env.TAVILY_API_KEY!, maxResults: 3 });
const messages = [{ role: 'user' as const, content: "What's the latest AI news?" }];

// Anthropic shape
const msg = await client.messages.create({ model: 'claude-haiku-4-5', maxTokens: 512, messages, tools: [tavily] });
// OpenAI shape
const completion = await client.chat.completions.create({ model: 'gpt-4o-mini', messages, tools: [tavily] });
// Native shape
const result = await client.run({ prompt: "What's the latest AI news?", model: 'claude-haiku-4-5', tools: [tavily], maxTokens: 512 });
```

**cURL**

```bash
# The agent client is SDK-side: it calls your LLM provider itself. The only
# traffic to Compresr is one compress call per tool result above min_tokens,
# with the query resolved for that tool call (its "query" argument by default).
# This is that call for a web-search result:
curl -X POST https://api.compresr.ai/api/compress/question-specific/ \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "<raw web-search results returned by the tool>",
    "query": "latest AI news",
    "compression_model_name": "{{SDK_DEFAULT_MODEL}}",
    "target_compression_ratio": 0.5
  }'
```

Notes:

- **Model lives at the call site.** `llm="anthropic"` sets the provider; every call passes `model=`. Pin a default with `llm="anthropic:claude-haiku-4-5"` (or `/` — both separators work). Neither set → `CompresrError("model is required …")`.
- **Python `run()` / `arun()` are keyword-only** — `client.run("question")` raises `TypeError`.
- **Any LangChain tool works.** A `@tool`-decorated function (Python) or `tool({...})` (TypeScript) gets its string output compressed automatically.
- **Per-call LLM knobs** (`temperature`, `top_p`, `max_tokens`, `stop`, `seed`, …) forward to the chat model; unknown keys are silently dropped. On Gemini the SDK renames `max_tokens` → `max_output_tokens` for you.
- **Async:** Python exposes `acreate` / `arun`. **Agent-layer streaming is not implemented** — Python facades have no `.stream(...)` (raises `AttributeError`); TypeScript throws `CompresrError(code: 'not_implemented')`. [`compress_stream`](/docs/guides/streaming) is unaffected.
- **Web search and research:** `WebSearchTool` (Tavily / Brave / AgentCore) and the `client.research` facade are covered in the [web search guide](/docs/guides/web-search).

### Compression knobs

`compression={...}` at construction applies to every tool output the middleware compresses:

| Key | Default | Effect |
|---|---|---|
| `compression_model_name` | `"{{SDK_DEFAULT_MODEL}}"` | Which model compresses tool outputs. See [Models](/docs/api-reference/models). |
| `target_compression_ratio` | `0.5` | Same semantics as the [`compress` parameter](/docs/api-reference/models#target_compression_ratio). |
| `min_tokens` | `200` | Tool outputs shorter than this skip compression (middleware-side gate, not sent to the API). |
| `coarse` | server default (`true`) | Paragraph-level vs token-level. See [coarse mode](/docs/guides/coarse-mode). |
| `allow_tools` / `ignore_tools` | — | Whitelist / blacklist of tool names to compress. |
| `on_error` | `"passthrough"` | On a Compresr backend error, forward the original tool output; `"raise"` to fail loudly. |

The policy doesn't expose the `dynamic*` knobs; call `client.compress(dynamic=True, ...)` directly if you need adaptive ratios.

## Errors

Every SDK error inherits from `CompresrError` and carries a stable `code` plus structured attributes (`err.retry_after`, `err.field`, …) — HTTP-level details in [Error codes](/docs/api-reference/errors). In Python, import them from **`compresr.exceptions`** (they are not re-exported at the package root). In TypeScript, the classes marked ✓ are importable from `@compresr/sdk`; everything else arrives as `CompresrError` with the corresponding `code`.

| Exception | HTTP | `code` | TS import | Attributes |
|---|---|---|---|---|
| `AuthenticationError` | 401 | `authentication_error` | ✓ | — |
| `ScopeError` | 403 | `scope_error` | ✓ | `required_scope` |
| `NotFoundError` | 404 | `not_found` | ✓ | `resource` |
| `RateLimitError` | 429 | `rate_limit_exceeded` | ✓ | `retry_after` (int; fractional `Retry-After` values read as `None` — guard with `err.retry_after or 1`) |
| `ValidationError` | 422 | `validation_error` | ✓ | `field` |
| `ServerError` | 5xx | `server_error` | ✓ | — |
| `ServiceUnavailableError` | 503 | `service_unavailable` | — | `service`, `retry_after` |
| `CompresrConnectionError` / `ConnectionError` | — | `connection_error` | ✓ | transport failures, including timeouts |
| `CompresrError` | — | (varies) | ✓ | base class — catch as fallback |

```python
import time
from compresr.exceptions import CompresrError, RateLimitError, ValidationError

try:
    result = client.compress(context=doc, query=q,
                             compression_model_name="{{DEFAULT_MODEL}}")
except RateLimitError as e:
    time.sleep(e.retry_after or 1)   # then retry
except ValidationError as e:
    raise ValueError(f"Bad request ({e.field}): {e}") from e
except CompresrError:
    raise
```

**TypeScript**

```typescript
import { CompresrError, RateLimitError, ValidationError } from '@compresr/sdk';

try {
  const result = await client.compress({ context: doc, query: q,
    compressionModelName: '{{DEFAULT_MODEL}}' });
} catch (error: unknown) {
  if (error instanceof RateLimitError) {
    await new Promise((r) => setTimeout(r, (error.retryAfter ?? 1) * 1000));
    // then retry
  } else if (error instanceof ValidationError) {
    throw new Error(`Bad request: ${error.message}`);
  } else {
    throw error;
  }
}
```

**cURL**

```bash
# Branch on HTTP status; the body carries the flat error envelope.
status=$(curl -s -o body.json -D headers.txt -w "%{http_code}" \
  -X POST https://api.compresr.ai/api/compress/question-specific/ \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"context":"...","query":"...","compression_model_name":"{{DEFAULT_MODEL}}"}')

if [ "$status" = "429" ]; then
  retry_after=$(grep -i '^Retry-After:' headers.txt | awk '{print $2}' | tr -d '\r')
  sleep "${retry_after:-1}"   # then retry
elif [ "$status" -ge 400 ]; then
  cat body.json >&2; exit 1
fi
```

> **Always handle 429**
> The free tier has tight per-minute limits. A retry loop with exponential backoff that respects `retry_after` is the single most important piece of production error handling.