Where Latte sits
- The document and your question go to Compresr in one call; the question is the
query. - Compresr returns the document with the parts that don’t help answer it removed.
- The LLM answers from the shorter document. The left lane skips step 1 and sends the whole document.
Latte SDK examples
from compresr import CompressionClient
compresr = CompressionClient() # reads COMPRESR_API_KEY
kept = compresr.compress(
context=document,
query=question,
compression_model_name="latte_v2.1",
target_compression_ratio=0.4, # remove ~40%, or dynamic=True
).data.compressed_context
answer = llm.responses.create(
model="gpt-5.4-mini",
input=f"DOCUMENT:\n{kept}\n\nQUESTION: {question}",
)More: Quick start · Models & parameters
Concepts
Two runs, one question
Every run happens twice in parallel with the same model and inputs: without Compresr on the left, with it on the right. A judge then compares the two answers.
Query-aware compression
Compresr keeps what matters for a query. For the X agent the query also includes the focus the agent writes per read, so the same timeline keeps different posts for “pricing complaints” than for “launch dates”. Turn query-aware off to compress against the question alone.
Latte: fixed rate or dynamic mode
A fixed rate targets the requested share (rate 0.4 targets 40% removed). Dynamic mode picks the cut per input from how relevance falls off, inside the min/max you set as multipliers (2× keeps half). latte_v2 family only.
Latte: coarse or fine
Fine (the default here) scores individual tokens, so it can trim inside a paragraph. Coarse keeps or drops whole paragraphs (for X, whole posts) and is faster.
Laya semantic budgeting
The demo splits each read into semantic units and asks Laya how useful each unit is for the question. Remove sets an approximate token budget: the most relevant units are kept in source order with their required headings. The first selected unit and its headings may exceed that budget. Laya is demo-only; the public SDK examples here use Latte.
Reading the dropped view
Open a step in the right lane: kept text is shown as is and every removed stretch is folded into a “[N lines dropped]” or “[N words dropped]” marker. Click one to see exactly what the LLM never saw.
Small inputs and failures
Inputs under the pass-through threshold go to the LLM untouched. If a Compresr call fails, the raw text is sent instead, so the model never loses context.
The judge
A separate LLM call reads both answers and the source without knowing which is which (the order is randomised every run). It calls them equivalent unless one gets the core answer wrong or leaves it out; style, length and secondary details don’t count. When it isn’t sure, it says equivalent.