# https://compresr.ai/docs/guides/streaming

> Human-readable page: https://compresr.ai/docs/guides/streaming

Use streaming when a human is watching the output land - chat UIs, agent traces, anything where time-to-first-token matters more than the total wall-clock time. The Compresr SDKs expose `compress_stream` / `compressStream` as iterators; the underlying HTTP transport is newline-delimited JSON in SSE-formatted `data:` frames on `/api/compress/question-specific/stream`. It's a subset of SSE - no `event:`, `id:`, or `retry:` lines.

This guide covers the basic streaming pattern in all three languages, the shape of each chunk, how to handle errors that fire mid-stream, and when streaming is the wrong choice.

## Why stream

A non-streaming call buffers the entire compressed output server-side and returns it in one HTTP response. Latency to the first visible character equals the total compression time.

Streaming pushes characters to the client the moment the model produces them. First visible output lands much sooner, which matters when:

- A user is watching a chat response build up
- An agent loop displays per-step traces
- A long-running compression sits on the latency critical path of a UI

For background jobs and pipelines that need the full output before doing anything with it, the synchronous `compress` call is simpler.

## Basic streaming

The Python and TypeScript SDKs expose streaming as an iterator. cURL reads the `data:` frames directly; pass `-N` to disable output buffering and parse them yourself. Watch for the `data: [DONE]` sentinel that marks end-of-stream.

```python
for chunk in client.compress_stream(
    context=long_document,
    query="What was the project's Q3 churn rate?",
    compression_model_name="{{DEFAULT_MODEL}}",
):
    print(chunk.content, end="", flush=True)
    if chunk.done:
        print()
        break
```

**TypeScript**

```typescript
for await (const chunk of client.compressStream({
  context: longDocument,
  query: "What was the project's Q3 churn rate?",
  compressionModelName: '{{DEFAULT_MODEL}}',
})) {
  process.stdout.write(chunk.content);
  if (chunk.done) {
    process.stdout.write('\n');
    break;
  }
}
```

**cURL**

```bash
curl -N -X POST https://api.compresr.ai/api/compress/question-specific/stream \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "<your long document>",
    "query": "What was the project'"'"'s Q3 churn rate?",
    "compression_model_name": "{{DEFAULT_MODEL}}"
  }' \
| while IFS= read -r line; do
    # Each frame arrives as: data: {"content": "..."}
    # The server closes the stream with: data: [DONE]
    case "$line" in
      "data: [DONE]") break ;;
      data:*) echo "${line#data: }" ;;
    esac
  done
```

## The chunk shape

The wire only carries content frames:

```
data: {"content": "string"}
```

No `done`, no `error`. The server signals end-of-stream with a literal `data: [DONE]` sentinel and closes the connection.

The SDK generator (Python) and async iterator (TypeScript) yield each wire frame verbatim as `{content, done: false}`, then yield a final synthetic `{content: "", done: true}` after the underlying iterator exhausts. Concatenate `content` across chunks and stop on `done: true`.

The `error` field on `StreamChunk` is defined in the schema but currently unused - no code path populates it. Server-side aborts propagate as raw `httpx` exceptions (Python) or `ConnectionError` (TypeScript), not as chunks with `error` set.

If you need the token-accounting metadata (`original_tokens`, `compressed_tokens`, `tokens_saved`, `actual_compression_ratio`, `duration_ms`) for billing or telemetry, call the non-streaming [`compress`](/docs/sdks/python#3-compress) endpoint; the streaming endpoint only emits text chunks, not the response envelope.

## Handling errors

Errors split cleanly by when they happen:

- **Before the first byte.** `4xx` / `5xx` responses are mapped to typed errors (`RateLimitError`, `AuthenticationError`, `CompresrError`, ...) by the initial status check. Catch these to backoff or reauth.
- **After the first byte.** Once the SDK is inside the byte-loop, mid-stream drops surface as raw transport exceptions - `httpx.HTTPError` in Python, `ConnectionError` in TypeScript. No typed `RateLimitError` post-first-byte, since headers and status have already been consumed.

Design for partial output: keep what you have, surface a clear error, and decide whether to reissue from scratch. `RetryConfig` is bypassed on streaming - the SDK only retries the initial connect, never a mid-stream failure. `retry_after` / `retryAfter` is nullable; guard the sleep.

```python
import httpx
from compresr import RateLimitError, CompresrError

buffer = []
try:
    for chunk in client.compress_stream(
        context=long_document,
        query="What was the project's Q3 churn rate?",
        compression_model_name="{{DEFAULT_MODEL}}",
    ):
        buffer.append(chunk.content)
        if chunk.done:
            break
except RateLimitError as e:
    # 429 mapped before first byte. retry_after may be None.
    logger.warning("Rate limited, sleeping %ss", e.retry_after or 1)
    time.sleep(e.retry_after or 1)
except CompresrError as e:
    # Other typed errors raised before the byte-loop starts.
    logger.exception("Request rejected: %s", e)
except httpx.HTTPError as e:
    # Mid-stream drop: transport exception, buffer holds partial output.
    logger.exception("Stream interrupted: %s", e)

partial = "".join(buffer)
```

**TypeScript**

```typescript
import { RateLimitError, CompresrError, ConnectionError } from '@compresr/sdk';

const buffer: string[] = [];
try {
  for await (const chunk of client.compressStream({
    context: longDocument,
    query: "What was the project's Q3 churn rate?",
    compressionModelName: '{{DEFAULT_MODEL}}',
  })) {
    buffer.push(chunk.content);
    if (chunk.done) break;
  }
} catch (error: unknown) {
  if (error instanceof RateLimitError) {
    // 429 mapped before first byte. retryAfter may be undefined.
    const wait = (error.retryAfter ?? 1) * 1000;
    console.warn(`Rate limited, sleeping ${wait}ms`);
    await new Promise((r) => setTimeout(r, wait));
  } else if (error instanceof ConnectionError) {
    // Mid-stream drop. Buffer holds partial output.
    console.error('Stream interrupted', error);
  } else if (error instanceof CompresrError) {
    console.error('Request rejected', error);
  } else {
    throw error;
  }
}

const partial = buffer.join('');
```

**cURL**

```bash
# The wire is a subset of SSE: only "data: {\"content\": \"...\"}\n\n" frames,
# terminated by a "data: [DONE]" sentinel. No event: / id: / retry: channels.
#
# 4xx / 5xx failures arrive as a normal HTTP response BEFORE any data frames,
# so -i prints them as headers + JSON body. Mid-stream drops close the socket -
# there's no in-band error frame; the partial output is whatever you printed.
curl -N -i -X POST https://api.compresr.ai/api/compress/question-specific/stream \
  -H "X-API-Key: $COMPRESR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "context": "<your long document>",
    "query": "What was the project'"'"'s Q3 churn rate?",
    "compression_model_name": "{{DEFAULT_MODEL}}"
  }'
```

## SDK caveats

- **Python is sync-only.** `compress_stream` returns a `Generator`, not an `AsyncGenerator`. There is no `compress_stream_async`. On FastAPI or asyncio, offload the loop to a thread with `asyncio.to_thread` or `run_in_executor`, or use the TypeScript SDK which is async-native.
- **TypeScript timeout kills the whole stream.** The client uses a single `AbortController` armed for `timeout` ms from connect - not idle-based. Long streams may need `timeout: 0` or a much larger value at client construction.
- **Agent-layer streaming is not wired up.** `client.messages.stream(...)` and `client.chat.completions.stream(...)` on the TS agent facades throw `CompresrError('streaming not yet implemented', code: 'not_implemented')`; the Python facades don't expose `.stream(...)` at all. It's a Phase 2 work item. The compression-API `compress_stream` / `compressStream` shown above is the working streaming path today.

## When NOT to stream

- **The downstream consumer is a non-streaming LLM call.** You gain nothing by streaming the compressed text just to buffer it before passing it to a synchronous `chat.completions.create`. Use plain `compress` instead.
- **Batch / pipeline jobs over many contexts.** Per-call streaming overhead is wasted when no human is watching. Use `compress_batch` to round-trip once instead.
- **Tight latency budgets at high concurrency.** Streams hold an open HTTP connection per request, so the synchronous `compress` call is cheaper to schedule when you're firing hundreds of compressions per second.