Compress JSON for LLMs: 2026 Guide to Cut Token Costs

TL;DR

When you compress JSON for LLMs, you reduce the token count of structured data before it enters a language model. This spans two dimensions: format compression (stripping syntactic overhead like repeated keys, braces, and whitespace) and content compression (removing data irrelevant to the current query). Format tricks save 10 to 60% depending on data shape, while query aware content compression can cut 50 to 90%+ of tokens by keeping only what the LLM actually needs to answer the question.

Key Takeaway: When you compress JSON for LLMs, you reduce token consumption via two methods:

1. Format Compression (Lossless): Minification, key abbreviation, and converting tabular JSON to alternative formats like TOON or CSV save 10 to 60% on syntactic overhead (braces, quotes, whitespace).
2. Content Compression (Lossy): Query aware filtering drops fields irrelevant to the specific prompt, saving 50 to 90%+ of tokens. Combining query aware content filtering first, followed by format minification, delivers maximum token efficiency and minimizes latency.

If you want to see content compression in action, try the Compresr demo with your own JSON payloads.

What Does "Compress JSON for LLMs" Mean?

To compress JSON for LLMs is to reduce the token count of JSON data before sending it to a language model. The goal is straightforward: spend fewer tokens on the same information, which lowers cost, reduces latency, and frees up context window space for content that matters.

But "compress" is doing double duty here. Most articles on the topic treat it as a format problem: strip whitespace, shorten keys, maybe convert to a different notation. That covers one dimension. The other dimension, largely missing from the conversation, is content compression: removing the fields, records, and values that are irrelevant to the specific task. The difference matters enormously. Format compression might save 30 to 60% on syntax. Content compression can remove 80%+ of the payload by dropping data the LLM never needed to see.

This guide covers both.

---

Why JSON Is Token Expensive

JSON was designed for machines to parse and humans to read. It was not designed for BPE tokenizers to process efficiently. Several mechanics make it uniquely costly when passed to LLMs, and understanding them is essential before you attempt to compress JSON for LLMs in any meaningful way.

Structural Characters Don't Compress Well in BPE

BPE tokenizers build their vocabulary by merging frequently co occurring byte pairs. JSON brackets, colons, URL slashes, and code indentation patterns rarely merge aggressively during training, so those sequences stay expensive in tokens relative to the characters they contain. A single curly brace or quotation mark often consumes an entire token.

Research shows that whitespace and indentation alone account for roughly 24.5% of tokens across programming languages. When you're outputting or consuming structured formats like JSON, where braces, quotes, and indentation stack on top of actual content, that overhead compounds fast.

Repeated Keys Multiply the Problem

Consider an array of 1,000 sensor readings. Field names like device_id, temperature, and timestamp appear 1,000 times each. Every repetition costs tokens. Add the structural punctuation (braces, brackets, colons, commas) and you're burning thousands of tokens on information that could be stated once.

A 500 word JSON payload routinely uses 800+ tokens. A kubectl get pods response in JSON runs 2.5 to 3x more tokens than the same data in a compact format. Studies on agentic and structured benchmarks show that wrapping unstructured text in JSON adds 18% to 30% more tokens on average. JSON doesn't just fail to compress content; it actively bloats it.

Understanding this overhead is the first step toward reducing input token costs across your LLM pipeline.

How Tokenizers See Your Data (And Why It Matters)

Format level compression gets more effective when you understand what the tokenizer actually does with your JSON. This is tokenizer aware data formatting: structuring your data to align with how BPE vocabularies are built, not just how humans read it.

A few patterns that practitioners on Reddit report making a measurable difference:

Camel case vs. snake case keys. A key like device_id often splits into three tokens (device, _, id), while deviceId may merge into two or even one token depending on the vocabulary. For high cardinality arrays where keys repeat thousands of times, this difference compounds. Testing your actual keys against tiktoken or your model's tokenizer before committing to a naming convention is worth the five minutes.

Number formatting matters. Floats like 0.00034 tokenize differently than 3.4e-4. Scientific notation is sometimes cheaper for very small or very large numbers, but not always. Integer IDs tokenize better than UUIDs or base64 strings. A UUID like 550e8400-e29b-41d4-a716-446655440000 costs 4 to 5 tokens; a sequential integer costs 1.

Boolean flag optimization. JSON booleans (true, false) cost 1 token each, but the key name still costs tokens. When you have objects with many boolean flags (feature toggles, permissions, status indicators), a common optimization is to replace multiple boolean fields with a single bitfield or comma separated list of active flags. Instead of {"canRead":true,"canWrite":false,"canDelete":true,"canAdmin":false}, send {"perms":"read,delete"}. This cuts both key repetition and value tokens. For arrays of objects with many boolean fields, the savings are significant.

Date and timestamp compression. ISO 8601 timestamps (2026-07-15T14:30:00Z) tokenize into 7 to 9 tokens. Unix epoch integers tokenize into 2 to 3. If the LLM doesn't need human readable dates, switch formats.

The general principle: run a sample payload through your tokenizer, identify the most expensive patterns, and restructure those first. The savings from tokenizer aware formatting stack on top of standard minification.

Format Level Techniques to Compress JSON for LLMs

These approaches reduce JSON's syntactic overhead while preserving all the original data. Think of them as lossless compression for LLM inputs.

Quick Wins (No Dependencies Required)

• Minification: Strip all whitespace, newlines, and indentation. In one test case, an unminified JSON object tokenized to 10 tokens while minification cut it to 6, a 40% reduction. That's an extreme best case, but 5 to 20% savings are typical. Models do not need indentation. Indentation exists for human developers.
• Key abbreviation: Replace verbose key names with short aliases and include a mapping header. "device_id" becomes "d", "temperature" becomes "t". When keys repeat across hundreds or thousands of records, this adds up quickly.
• UUID shortening: A UUID like "550e8400-e29b-41d4-a716-446655440000" costs 4 to 5 tokens. A short alias like "u-1" costs 1. If the LLM doesn't need the actual UUID to answer the question, replace it.
• Removing nulls and empty values: Fields with null, "", or [] values carry zero information but still cost tokens. Drop them.

These techniques combined typically deliver 10 to 30% savings with minimal implementation effort.

Flatten Nested Structures Before Encoding

Deeply nested JSON is one of the worst offenders for token waste. Each level of nesting adds braces, indentation (if pretty printed), and often repeats context that could be expressed more concisely.

Consider a typical API response with nested address objects:

{"user":{"name":"Alice","address":{"street":"123 Main","city":"Portland","state":"OR"}}}

Flattened with dot notation keys:

{"user.name":"Alice","user.address.street":"123 Main","user.address.city":"Portland","user.address.state":"OR"}

The flattened version eliminates inner braces and colons from the nesting structure. For a single record, the savings are modest. For an array of 500 records with 3 levels of nesting, flattening can cut 15 to 25% of tokens purely from structural character removal.

A few practical considerations:

• Flatten before converting to TOON or CSV. Both formats assume flat, uniform rows. Nested objects break their column model and either force JSON fallback or lose data. Flattening first is a prerequisite, not an optimization.
• Use a consistent separator. Dot notation (user.address.city) is the most common. Avoid characters that conflict with your data values.
• Watch for key collisions. If your JSON has both user.name as a nested path and user.name as a top level key, flattening creates ambiguity. This is rare in practice but worth checking.
• Preserve nesting when the LLM needs hierarchical reasoning. If the prompt asks the model to navigate or compare nested structures (e.g., "compare the billing address to the shipping address"), flattening can remove structural cues the model relies on. Test accuracy before and after.

Practitioners in forum discussions note that flattening combined with key abbreviation is often the highest ROI format change for nested API responses, precisely because it attacks both repeated structural characters and verbose key paths simultaneously.

Schema and Cardinality Declaration in Prompt

Instead of repeating field names and types across every record, state the schema once in your prompt and pass only values. This is a technique that shows up frequently in discussions about optimizing token usage by component.

The pattern:

Schema: [name:str, age:int, city:str, active:bool]
Cardinality: 150 records
Data:
["Alice",32,"Portland",1]
["Bob",28,"Seattle",0]
...

This eliminates key repetition entirely while giving the LLM enough structural context to interpret the values correctly. The cardinality declaration ("150 records") tells the model what to expect, which helps with reasoning tasks like "how many users are in Portland?"

For large uniform arrays, schema declaration plus value only rows can outperform even TOON, because it removes not just repeated keys but also the per record braces and quotes around key names. One developer shared in a YouTube walkthrough that this pattern cut a 4,000 token customer record payload to under 1,200 tokens, roughly 70% savings, while maintaining perfect accuracy on filtering and aggregation queries.

The tradeoff: this format is more fragile. If column order gets scrambled or a value is missing without a placeholder, the LLM loses track of which value maps to which field. Always use consistent column ordering and explicit null placeholders (empty string or a sentinel like _).

Indentation Based Hierarchy Instead of Braces

For nested, non uniform JSON that can't be flattened cleanly, an alternative to braces is indentation based hierarchy, similar to YAML or Python's structural approach.

Instead of:

{"order":{"id":42,"items":[{"sku":"A1","qty":2},{"sku":"B3","qty":1}],"total":59.99}}

Use:

order
  id: 42
  items
    A1, 2
    B3, 1
  total: 59.99

This preserves the tree structure while eliminating braces, brackets, quotes around keys, and colons between key value pairs (replaced with a single colon or whitespace). For moderately nested data, this format saves 20 to 40% of tokens compared to minified JSON.

The caveats are real:

• Model familiarity varies. Most large models (GPT 4 class and above, Claude 3.5+) handle YAML like indentation well because their training data includes massive amounts of YAML, Python, and config files. Smaller models may struggle.
• Ambiguity at the edges. Values containing colons or leading whitespace need escaping or quoting, which partially negates the savings.
• Not suitable for machine parsing on output. If you need the LLM to return structured data that your code parses, JSON with constrained decoding is safer. This technique is for input compression only.

Think of indentation based formatting as a middle ground: it preserves hierarchy better than flattening, costs fewer tokens than JSON, but requires more careful testing than either.

Alternative Formats: TOON, CSV, and Markdown Tables

For tabular or uniform data, converting away from JSON entirely yields bigger gains. Any team looking to compress JSON for LLMs at scale should evaluate these alternatives for their tabular payloads.

TOON (Token Oriented Object Notation) is an alternative designed for LLMs. It states field names once as a header row, then lists values as rows, eliminating the repeated key problem. Benchmarks show impressive results for uniform data shapes:

• Uniform employee records (100 rows): 60.7% fewer tokens than formatted JSON, 36.8% fewer than minified JSON
• Time series analytics data: 59% and 35.8% savings respectively
• Model accuracy: Across four major LLMs, TOON used 39.6% fewer tokens while achieving 73.9% accuracy versus JSON's 69.7%
• Production implementation: Halodoc's engineering team reported 35 to 45% token reduction with roughly 95% LLM compatibility across OpenAI, Anthropic, Google, and AWS Bedrock.

CSV and Markdown tables perform similarly for flat tabular data. One benchmark on MCP server responses found TSV and CSV both achieved 55% token reduction, with Markdown tables at 44%.

Deterministic Round Trip JSON to TOON Conversion

One concern that comes up repeatedly when teams evaluate TOON is whether the conversion is lossless and reversible. If you encode JSON as TOON for the LLM's input but need the LLM's output back in valid JSON for downstream systems, the round trip needs to be deterministic.

For flat, uniform arrays (all objects share the same keys in the same order), the round trip is straightforward: TOON's header declares columns, each row provides values in column order, and reconstruction is mechanical. A simple encoder/decoder pair handles this with no ambiguity.

Where round trips break down:

• Optional fields. If some records have a field and others don't, TOON needs explicit null placeholders to keep column alignment. Missing placeholders silently shift values into wrong columns.
• Mixed types within a column. A column that contains both strings and numbers, or strings that look like numbers, requires type hints or quoting conventions to reconstruct correctly.
• Nested values. TOON doesn't natively represent nested objects. Embedding JSON fragments inside TOON cells is possible but ugly and partially defeats the purpose.

For production pipelines where you need guaranteed round trip fidelity, build validation into your encoder: assert that decoding the TOON output produces byte identical JSON to the original input. If it doesn't, fall back to minified JSON for that payload. This hybrid approach captures TOON's savings on the ~80% of payloads that are clean and uniform while avoiding silent data corruption on edge cases.

Streaming TOON Encoding for Large Datasets

Standard TOON encoding requires knowing all the column names upfront, which means buffering the entire dataset before writing the header. For large datasets (100K+ records from a database query or log export), this buffering adds memory pressure and latency before the LLM receives any data.

Streaming TOON encoding solves this by fixing the schema at stream initialization and encoding records incrementally:

1. Emit the header from the known schema (database table definition, API response schema, or the first N records).
2. Stream rows as they arrive, encoding each against the fixed column list.
3. Handle schema drift by either dropping unexpected fields (with a warning) or falling back to JSON for that record.

This pattern matters most in two scenarios:

• Agent tool outputs from database queries. The agent calls a SQL tool that returns thousands of rows. Streaming TOON encoding lets you pipe compressed rows to the LLM's context as they arrive, rather than waiting for the full result set.
• Log analysis. Structured logs often have a stable schema with occasional extra fields. Streaming TOON handles the stable fields efficiently and can be configured to append extras as a catch all column.

For teams using streaming compression endpoints, combining streaming TOON encoding with query aware content compression creates a pipeline where large JSON datasets are filtered to relevant records, converted to TOON on the fly, and streamed into the LLM context with minimal buffering.

When Alternative Formats Break Down

TOON's limitations are real and often omitted from discussions promoting it.

• Nested data is a problem: For deeply nested or non uniform JSON, TOON can actually cost more tokens than the original. If your data has nested objects, you need to flatten before encoding, which adds complexity and can lose structural information.
• Model compatibility varies wildly: In benchmark evaluations, Mistral Small 24B's accuracy dropped from 89% under JSON to 53% under TOON. That's a severe regression that breaks the model's ability to reason about the data. Always test with your specific model.
• Small payloads don't benefit: For payloads under ~500 tokens, the conversion cost likely exceeds the savings.
• TOON is specialized: It works best for high volume, internally bounded LLM pipelines where both the prompt and the parser are under your control. For a single prompt with a small dataset, JSON with minification is fine.

Format Compression Decision Criteria

Choosing the right way to compress JSON for LLMs depends on your data shape and the compression ratio you need:

• Small payloads (<500 tokens): Don't bother. The optimization overhead isn't worth it.
• Tabular/uniform arrays: TOON or CSV. Expect 30 to 60% savings.
• Deeply nested structures: Stick with minified JSON, or try indentation based hierarchy if your model handles it well.
• Mixed payloads: Minify the JSON and abbreviate keys. Expect 15 to 30%.
• High cardinality uniform data: Schema declaration with value only rows. Expect 50 to 70%.

Rule of thumb: Use TOON for high volume tabular input (40% fewer tokens), JSON with constrained decoding for output (guaranteed valid), and a two step approach (free reasoning then structured formatting) when accuracy matters.

Content Level Compression: The Bigger Win When You Compress JSON for LLMs

Format compression attacks syntax. Content compression attacks relevance. For most real world JSON payloads, irrelevant content is the bigger problem.

When an API returns 50 fields but the user's question only needs 3, format tricks save 30 to 60% on the syntax wrapping those 50 fields. Query aware content compression can remove 80%+ of the payload by dropping the 47 irrelevant fields entirely.

This matters most in three scenarios:

• Tool outputs in agent workflows: An agent calls an API that returns a massive JSON response. Most of that response is irrelevant to what the agent was actually asked. Passing all of it to the downstream model wastes tokens and hurts accuracy.
• RAG retrieved documents: Retrieved chunks wrapped in JSON metadata often carry fields (timestamps, source IDs, embedding scores) that the LLM doesn't need for answering the question.
• Chat history: Long conversation histories serialized as JSON contain many exchanges irrelevant to the current turn.

Passing irrelevant data creates context rot: even powerful models have finite capacity, and long streams of irrelevant information degrade their reasoning.

Query Aware vs. Query Agnostic Compression

Context compression methods fall into two categories:

• Query agnostic methods: Exploit redundancy in the text itself, compressing uniformly regardless of the task.
• Query aware methods: Compress differently depending on the specific question, keeping only the fields and values relevant to the current query.

For JSON payloads, query specific compression is almost always the better choice. A JSON tool output from a financial API might contain dozens of fields, but if the user asked "What was last quarter's revenue?", only a few fields matter. Query aware compression identifies and preserves those fields while aggressively cutting everything else.

The typical savings range is 50 to 80%, with some payloads compressing by 90%+.

Tools like Compresr's query specific API automate this process, taking the query and the JSON payload as inputs and returning only the tokens the LLM needs to produce a correct answer.

Combining Format and Content Methods to Compress JSON for LLMs

Optimal Two Step Compression Pipeline:

Raw JSON Payload → Step 1: Query Aware Content Filter → Step 2: Format Optimization → Token Efficient Prompt

The maximum compression on JSON payloads comes from layering both approaches:

1. Content compression first: Run the JSON payload through query aware compression to strip irrelevant fields and values.
2. Format optimization second: If the remaining data is still large (particularly if it's tabular), apply format level techniques: minify, abbreviate keys, or convert to TOON/CSV.

Why content first? Format compression operates on every byte of data provided. If you minify a 10,000 token JSON payload and then strip 80% of the content, you wasted effort minifying 8,000 tokens of data that got thrown away. Compress content first, then optimize the format of what remains.

This two step approach is where net savings from compression really become clear. The content step does the heavy lifting (removing irrelevant data), and the format step squeezes out the remaining syntactic waste.

Measuring Results

Any effort to compress JSON for LLMs should be validated with concrete metrics, not assumptions. Here's what to track:

• Track tokens per record (TPR): Measures baseline efficiency independent of payload size. A JSON array of 100 user records might start at 45 TPR, drop to 30 TPR after minification, and drop to 6 TPR after query aware compression strips irrelevant fields.
• Measure accuracy: Run your evaluation suite after every compression change. Compression is useless if the LLM cannot answer correctly with the compressed input.
• Compare tokenizer output directly. Use tiktoken (for OpenAI models) or your provider's tokenizer to count tokens before and after each compression step. Don't estimate; measure.
• Track compression across the full pipeline. A compression step that saves 40% on JSON formatting but adds a 200ms API call might not be worth it for latency sensitive applications. Measure end to end.

For teams managing wasted context across pipelines, combining format and content compression yields the highest return on optimization effort. Compresr's pricing runs at $0.10 per 1M tokens compressed, which means the compression step typically pays for itself many times over against the cost of major LLM providers.

Common Mistakes When You Compress JSON for LLMs

• Compressing short payloads: If your JSON is under 500 tokens, the API overhead of a compression step exceeds the savings. Set a minimum token threshold and skip compression for small inputs.
• Assuming all models handle alternative formats equally: The Mistral accuracy drop (89% to 53%) when switching from JSON to TOON highlights the risks. Test with your specific model and data shape before deploying non standard formats.
• Trusting format conversion alone when content is the real problem: If your JSON payload is 10,000 tokens because it contains 200 fields from an API response and the user's question only touches 5 of them, TOON will save ~40% on syntax. Query aware compression will save 90%+ by removing the 195 irrelevant fields.
• Not measuring accuracy after compression: Run structured evaluations. Some compression approaches improve accuracy (by removing distracting information) while others degrade it (by removing context the model needed).
• Forgetting about output costs: Compressing JSON inputs is half the picture. If you're asking the LLM to produce JSON output, evaluate whether you need full JSON or if a simpler format suffices for downstream consumers.
• Ignoring tokenizer differences across models. GPT 4o, Claude 3.5, and Gemini all use different tokenizers. A key name that's 1 token in cl100k_base might be 2 tokens in another vocabulary. If you're optimizing for a specific provider, test against that provider's tokenizer.
• Flattening when hierarchy matters. Flattening nested JSON saves tokens, but if your prompt asks the model to compare nested structures or navigate a tree, flattening removes the structural cues it needs. Always validate accuracy on your actual queries, not just token count.

When Each Approach Makes Sense

The table below summarizes the best way to compress JSON for LLMs based on your specific scenario and data characteristics.

---

FAQ

Does minifying JSON hurt LLM accuracy?

No. Current models process minified JSON just as well as pretty printed JSON, provided the schema definition is clear in the prompt. Minification is a free win that every pipeline should implement.

Is TOON always better than JSON for LLM prompts?

Not at all. TOON excels at uniform, flat, tabular data where field names repeat across many rows. For deeply nested or non uniform JSON, TOON can increase token count. Some models (notably smaller ones like Mistral Small 24B) show significant accuracy drops when switching from JSON to TOON. Always benchmark with your specific model and data.

What's the difference between format compression and content compression for JSON?

Format compression reduces syntactic overhead (whitespace, repeated keys, braces) while preserving all original data. Content compression removes fields and values that are irrelevant to the current query. Format compression is lossless and capped at roughly 60% savings for ideal data shapes. Content compression is lossy but targeted, yielding 50 to 90%+ savings by keeping only what the LLM needs to answer the question.

How do I know if my JSON payloads are worth compressing?

Measure the token count. If a payload is under 500 tokens, compression overhead likely exceeds the savings. If you're sending the same schema repeatedly at high volume, or if your payloads regularly exceed 1,000 tokens, compression will pay for itself quickly. Track tokens per record as your baseline metric.

Can gzip or similar byte level compression help with LLM token costs?

No. Gzip compresses bytes for network transfer, but the LLM still needs to process the decompressed text. LLM costs are based on token count, not byte count. You need to reduce the actual text that reaches the model, not just compress it in transit.

What about using Protocol Buffers or MessagePack instead of JSON?

These are binary formats that reduce network payload size, but LLMs cannot read binary data. You must serialize back to text before sending to the LLM, which defeats the purpose. The compression needs to happen at the text/token level, not the wire protocol level.

How does it affect latency when you compress JSON for LLMs?

Fewer input tokens means less prefill time. LLM inference latency scales with input length, so a 50% reduction in tokens often translates to a measurable latency improvement, particularly on longer inputs. One illustrative example showed a 24% latency reduction alongside an 86% cost decrease on a financial filing.

Should I compress JSON inputs or outputs first?

Start with inputs. Input tokens are where the waste is largest, especially for tool outputs, RAG contexts, and API responses. Output compression is a secondary optimization that matters most when output token costs are a significant share of your bill.

Does flattening nested JSON always save tokens?

Usually, but not always. Flattening eliminates nesting braces and brackets, which saves tokens. But if flattening produces very long dot notation key paths (order.customer.billing.address.zipCode), the savings from removed braces can be offset by longer key strings. Measure both versions with your tokenizer. For most real world API responses with 2 to 3 levels of nesting, flattening saves 15 to 25%.

Can I use schema declaration with non uniform data?

Not cleanly. Schema declaration with value only rows assumes every record has exactly the same fields in the same order. If records have optional or variable fields, you need null placeholders for every missing value, which can negate the savings. For non uniform data, minified JSON with key abbreviation is the safer choice.