# https://compresr.ai/docs/guides/web-search > Human-readable page: https://compresr.ai/docs/guides/web-search `WebSearchTool` is Compresr's drop-in search tool for agent loops. It returns a real LangChain `BaseTool`; pass it to any of the [agent client](/docs/sdks/python#6-agent-client) facades and the tool output flows through `CompresrToolMiddleware` automatically: search results get compressed before they re-enter the model context, with no extra wiring. Three backends ship out of the box: | Provider | Strengths | Bring your own key | |---|---|---| | **Tavily** (default) | LLM-tuned results, strong on recency and citations, `allowed_domains` / `blocked_domains` filtering, `max_results` forwarded verbatim | `TAVILY_API_KEY` | | **Brave** | Generalist web index, independent crawler, native domain filtering uses Goggles (out of scope for v1) | `BRAVE_SEARCH_API_KEY` (fallback: `BRAVE_API_KEY`) | | **AgentCore** | Amazon Bedrock web search over MCP + Cognito OAuth, `max_results` clamped 1..25 | Cognito client credentials (see [AgentCore](#agentcore) below) | ## Why not Anthropic / OpenAI / Gemini server search? Provider-native server search tools (Anthropic's `web_search_20250305`, OpenAI's `web_search_preview`, Gemini's `google_search`) **execute server-side and return opaque or encrypted content** that Compresr cannot read or compress. The model sees the search content but the SDK doesn't; there's nothing to compress because we can't intercept the payload. `WebSearchTool` deliberately calls the search API client-side. The plaintext results reach the SDK first, pass through `CompresrToolMiddleware`, and arrive at the model in compressed form. This is the only path on which Compresr's compression actually fires for web search results. ## Quick start: Tavily ```python import os from compresr import CompressionClient, WebSearchTool client = CompressionClient( api_key=os.environ["COMPRESR_API_KEY"], llm="anthropic", llm_api_key=os.environ["ANTHROPIC_API_KEY"], compression={"target_compression_ratio": 0.5, "min_tokens": 300}, ) tavily = WebSearchTool.tavily( api_key=os.environ["TAVILY_API_KEY"], max_results=5, ) msg = client.messages.create( model="claude-haiku-4-5", max_tokens=512, messages=[ {"role": "user", "content": "What are this week's top stories on AI agents?"}, ], tools=[tavily], ) print(msg.content[0].text) ``` **TypeScript** ```typescript import { CompressionClient, WebSearchTool } from '@compresr/sdk'; const client = new CompressionClient({ apiKey: process.env.COMPRESR_API_KEY!, llm: 'anthropic', llmApiKey: process.env.ANTHROPIC_API_KEY!, compression: { targetCompressionRatio: 0.5, minTokens: 300 }, }); const tavily = await WebSearchTool.tavily({ apiKey: process.env.TAVILY_API_KEY!, maxResults: 5, }); const msg = await client.messages.create({ model: 'claude-haiku-4-5', maxTokens: 512, messages: [ { role: 'user', content: "What are this week's top stories on AI agents?" }, ], tools: [tavily], }); console.log(msg.content[0]); ``` That's the whole integration. The model decides when to call `tavily_search` (or `brave_search` / `agentcore_web_search` for the other providers); the SDK fires the underlying API, hands the raw response to `CompresrToolMiddleware`, gets back a compressed version, and threads that back into the model context. ## Construction options All three backends accept a small, deliberately-narrow option set. Anything provider-specific (e.g. Tavily `search_depth`) is forwarded through the `extra` / `**extra` escape hatch. `max_results` is forwarded verbatim to Tavily and Brave (their own upper limits apply); AgentCore clamps it to `1..25`. ### Tavily ```python WebSearchTool.tavily( api_key=os.environ["TAVILY_API_KEY"], # optional; falls back to TAVILY_API_KEY env max_results=5, # forwarded verbatim to Tavily allowed_domains=["nytimes.com", "reuters.com"], blocked_domains=["example.com"], # Anything else (e.g. search_depth="advanced") goes via **extra: # search_depth="advanced", ) ``` **TypeScript** ```typescript await WebSearchTool.tavily({ apiKey: process.env.TAVILY_API_KEY!, // required; the TS factory does not read TAVILY_API_KEY itself maxResults: 5, // forwarded verbatim to Tavily allowedDomains: ['nytimes.com', 'reuters.com'], blockedDomains: ['example.com'], // Anything else (e.g. searchDepth: 'advanced') goes via extra: // extra: { searchDepth: 'advanced' }, }); ``` > **TypeScript Tavily env-var fallback** > The TypeScript `WebSearchTool.tavily({...})` factory does **not** read `TAVILY_API_KEY` from the environment; pass `apiKey` explicitly. The Python factory and the `client.research(...)` facade string form both do the env fallback. ### Brave ```python WebSearchTool.brave( api_key=os.environ["BRAVE_SEARCH_API_KEY"], # optional; env fallback tries BRAVE_SEARCH_API_KEY first, then BRAVE_API_KEY max_results=5, ) ``` **TypeScript** ```typescript await WebSearchTool.brave({ apiKey: process.env.BRAVE_SEARCH_API_KEY!, // env fallback (Python only): BRAVE_SEARCH_API_KEY, then BRAVE_API_KEY maxResults: 5, }); ``` > **No native domain filtering on Brave** > Brave doesn't expose native `allowed_domains` / `blocked_domains` — filtering flows through Goggles, which is out of scope for v1. Python emits a `UserWarning` and proceeds; TypeScript silently ignores the kwargs (they aren't declared on `BraveOptions`). Route through Tavily if you need real domain filtering. > **TypeScript Brave + Claude tool_use** > `WebSearchTool.brave` in TypeScript returns the upstream `BraveSearch` directly. LangChain.js's `BraveSearch` ships without an `args_schema`, so Anthropic (Claude) tool_use calls can fail with `missing 1 required positional argument: 'query'`. Workaround: wrap it in your own `tool()` factory that declares a `query: z.string()` schema, or use Tavily or AgentCore. Python's `WebSearchTool.brave` already wraps Brave with an explicit schema, so this is a TypeScript-only pitfall. ### AgentCore Amazon Bedrock AgentCore web search over an MCP streamable-HTTP session, authenticated with a Cognito OAuth client-credentials handshake. Runtime deps are optional — install the extra: `pip install compresr[agentcore]` (Python) or `npm install @modelcontextprotocol/sdk` (TypeScript). ```python WebSearchTool.agentcore( gateway_url=os.environ["AGENTCORE_GATEWAY_MCP_URL"], cognito_token_url=os.environ["AGENTCORE_COGNITO_TOKEN_URL"], client_id=os.environ["AGENTCORE_COGNITO_CLIENT_ID"], client_secret=os.environ["AGENTCORE_COGNITO_CLIENT_SECRET"], scope=os.environ["AGENTCORE_COGNITO_SCOPE"], max_results=5, # clamped 1..25 # allowed_domains / blocked_domains accepted for signature parity — # AgentCore has no native domain filter, emits a UserWarning. ) ``` **TypeScript** ```typescript await WebSearchTool.agentcore({ gatewayUrl: process.env.AGENTCORE_GATEWAY_MCP_URL!, cognitoTokenUrl: process.env.AGENTCORE_COGNITO_TOKEN_URL!, clientId: process.env.AGENTCORE_COGNITO_CLIENT_ID!, clientSecret: process.env.AGENTCORE_COGNITO_CLIENT_SECRET!, scope: process.env.AGENTCORE_COGNITO_SCOPE!, maxResults: 5, // clamped 1..25 // allowedDomains / blockedDomains: accepted for parity, silently ignored. }); ``` Every config field resolves from the explicit argument first, then a two-step env-var fallback: | Field | Primary env var | Fallback env var | |---|---|---| | `gateway_url` / `gatewayUrl` | `AGENTCORE_GATEWAY_MCP_URL` | `GATEWAY_MCP_URL` | | `cognito_token_url` / `cognitoTokenUrl` | `AGENTCORE_COGNITO_TOKEN_URL` | `COGNITO_TOKEN_URL` | | `client_id` / `clientId` | `AGENTCORE_COGNITO_CLIENT_ID` | `COGNITO_CLIENT_ID` | | `client_secret` / `clientSecret` | `AGENTCORE_COGNITO_CLIENT_SECRET` | `COGNITO_CLIENT_SECRET` | | `scope` | `AGENTCORE_COGNITO_SCOPE` | `COGNITO_SCOPE` | Runtime behaviour: - **HTTPS-only URLs (TypeScript).** `gatewayUrl` and `cognitoTokenUrl` are validated to start with `https://`; a plaintext URL throws `CompresrError('invalid_config')` before any credential leaves the process. - **`max_results` clamp.** TypeScript enforces `1..25` at build time. Python passes the value through to the underlying client (which applies the same bound). - **Bearer-token cache.** The Cognito token is minted once and cached on the shared client; a `401` from the gateway triggers exactly one automatic re-mint before the call is retried. - **Timeouts and response cap.** Cognito requests time out at 30s, tool calls at 30s, and responses are hard-capped at 1 MB. - **`allowed_domains` / `blocked_domains`.** Accepted for signature parity with Tavily/Brave. Python emits a `UserWarning` and proceeds; TypeScript silently ignores them (fields declared but unread). Use Tavily for real domain filtering. ## Constructor form The classmethod factories above are the recommended path; they're discoverable in autocomplete and statically typed. The constructor form below is equivalent and useful when the provider is a runtime variable: ```python # Equivalent to WebSearchTool.tavily(...) WebSearchTool(provider="tavily", api_key=..., max_results=5) # Equivalent to WebSearchTool.brave(...) WebSearchTool(provider="brave", api_key=..., max_results=5) # Equivalent to WebSearchTool.agentcore(...) WebSearchTool(provider="agentcore", gateway_url=..., cognito_token_url=..., client_id=..., client_secret=..., scope=..., max_results=5) ``` **TypeScript** ```typescript import { createWebSearchTool } from '@compresr/sdk'; // Equivalent to WebSearchTool.tavily({ ... }) await createWebSearchTool('tavily', { apiKey: ..., maxResults: 5 }); // Equivalent to WebSearchTool.brave({ ... }) await createWebSearchTool('brave', { apiKey: ..., maxResults: 5 }); // Equivalent to WebSearchTool.agentcore({ ... }) await createWebSearchTool('agentcore', { gatewayUrl: ..., cognitoTokenUrl: ..., clientId: ..., clientSecret: ..., scope: ..., maxResults: 5 }); ``` ## How compression interacts with web search `CompresrToolMiddleware` runs on every tool return inside the agent loop: 1. The model emits a tool call (e.g. `tavily_search({ query: "..." })`). 2. The SDK invokes the provider and normalises the response via `_flatten_search_results` / `flattenSearchResults` — the raw JSON is reshaped into blank-line-separated plain-text blocks of `title\nurl\ncontent`. This is the shape `latte_v1` can actually compress; JSON input is a no-op for compression. 3. The middleware checks the serialized string length. If it's at or below `compression.min_tokens` (default `200`), it's forwarded untouched. Above the threshold, the middleware calls `client.compress(...)` with the result body as `context` and the user's last message as `query`. 4. The compressed body replaces the original in the agent state. The model never sees the uncompressed search results. Two practical implications: - **Tune `compression.min_tokens` to your search backend.** Tavily returns long content per hit, easy to exceed `200` tokens. Brave returns shorter snippets, so you may want to lower `min_tokens` to e.g. `100` to capture them. - **The `query` used for compression is the user's intent**, not the model's tool-call query string. This keeps the compression query-aware against what the user actually asked, even when the model rewords the search. ## Errors & failure modes | Scenario | Behaviour | Mitigation | |---|---|---| | Missing peer dep (`langchain-tavily` / `langchain-community` / `@modelcontextprotocol/sdk`) | Python raises `ImportError` naming the extra. TypeScript raises `CompresrError` code `missing_peer_dependency`. | Install the extra: `pip install compresr[agents-tavily\|agents-brave\|agentcore]` or `npm install @langchain/tavily @langchain/community @modelcontextprotocol/sdk`. | | Missing API key or config | Python raises `ValueError` naming the arg and env-var fallback. TypeScript raises `CompresrError` with code `missing_api_key` (Brave), `missing_config` (AgentCore), or `invalid_config` (AgentCore non-HTTPS URL), plus `invalid_provider` for the constructor form. | Pass the arg or set the env var. | | Search-provider 401 / 403 / rate limit at call time | Python: the provider's exception propagates inside the agent loop. TypeScript: raised as `CompresrError` — for AgentCore specifically, codes `agentcore_auth_error`, `agentcore_no_tool`, `agentcore_bad_response`, `agentcore_tool_error`. | Rotate the key or back off / lower `max_results`. | | Compresr backend down while the middleware fires | Returns the original (uncompressed) search result by default (`on_error="passthrough"` on the policy). | Set `compression={"on_error": "raise"}` if you want to fail loudly instead. | | No tools fired (model answered without searching) | No compression call. The middleware only runs on tool returns. | Encourage the model to search via the user prompt or a `system` instruction. | ## Research facade For research-style pipelines that combine web search, snippet compression, and citation extraction in one call, use `client.research.run(question, search="tavily" | "brave" | , ...)` instead of wiring `WebSearchTool` into an agent loop by hand. The string form of `search=` currently supports `tavily` and `brave` only; pass a pre-built `WebSearchTool.agentcore(...)` if you want AgentCore under the facade. See the SDK reference pages for the full signature. ## Next steps - [Agent client](/docs/sdks/python#6-agent-client): the full facade surface (Anthropic shape, OpenAI shape, native). - [LangChain integration](/docs/framework-integration/langchain): lower-level `CompresrToolMiddleware`, `wrap_tool_with_compression`, and `CompresrExtractor` for custom retrieval pipelines. - [LangGraph integration](/docs/framework-integration/langgraph): drop-in compression node for `StateGraph`-style agents.