Why Agent Loops Don't Warn — They Bill
How runaway tool traffic drives token cost — and where a gateway helps.
A single "research this topic" prompt can trigger dozens of fetches, searches, and re-reads. Each hop may re-send context to your model. The loop doesn't ask permission — it invoices.
The hidden multiplier
| Stage | What happens |
|---|---|
| Agent plans | Model chooses tools repeatedly |
| Tool output | Large HTML/JSON returns to context |
| Re-prompt | Same content sent again on the next turn |
| No ceiling | Budget guardrails often live too late (at the LLM API) |
SmartGate intervenes at the tool layer: compress prompts, deduplicate repeated chunks, enforce budget checks, and log every call for audit.
Practical first steps
- Connect SmartGate MCP before enabling long-running agents
- Use Activity Logs to see which tools dominate spend
- Set team budget alerts in the Dashboard
Learn the tools: Documentation. Connect in minutes: Quick Start.
What "tool layer" means in practice
SmartGate is an MCP-native intelligence layer—not an LLM proxy. Your host (Cursor, Claude, OpenClaw, or a custom runtime) still plans and chats. The multiplier lives in tool calls: search snippets, fetched pages, re-reads, and pipe steps that re-enter the prompt. Intervening at the gateway means compression, deduplication, budget checks, and audit happen before that material becomes billable model input.
FinOps conversations often start with model unit price. That matters, but agent loops can dominate even when the model is cheap. A research turn that fetches five near-duplicate articles and pastes all of them into context will look like "one task" in a chat UI and like many token-heavy tool rows in Activity Logs.
Controls that change the invoice shape
- smart_context_gate — gateway-local compression with a configurable ratio and optional purpose filter. Savings depend on input structure; the dashboard shows per-request metrics from audit, not a marketing fixed percentage.
- smart_dedup — remove overlapping passages across multi-fetch or multi-search results before they fill the window three times.
- smart_budget_guard — check before expensive jobs and record after. Alerts alone do not stop loops; optional hard-cap behavior on eligible plans can block further tool calls when the monthly pool is reached.
- smart_pipe — run a research template as one audited chain instead of four ad-hoc calls that are easy to lose in a transcript.
How to investigate spend without guessing
Open Activity Logs filtered to your team and time window. Sort attention toward tools with high token_used, repeated identical URLs, and pipe steps that re-fetch. Export when a reviewer needs an artifact. Treat Est. USD saved as a gateway counterfactual at an internal benchmark for tool modules—not as a paste-ready replacement for your vendor LLM invoice.
Fair-use and evaluation hygiene
Free and Playground use strict fair-use rate limits. Bursts soft-fail with HTTP 429, limit_scope, and Retry-After when available. That is intentional for individual evaluation, not a silent multi-user research farm. Upgrade when shared workspace RPM, retention, or pools need to rise.
What not to claim in an internal write-up
Do not invent concurrent capacity figures, customer logos, or "save 80%" language. Do not describe SmartGate as a ChatGPT alternative or model router. Position it as information intelligence governance between hosts and the world: Research, Context, Memory, Control, and Pipeline on seven primitives.
Next reading
Related reading
This post is the cost-control companion to What Is an MCP Gateway (and Why You Need One), which hosts the shared operational playbook, production checklist, product links, and operator closing note for the whole series.
Where the numbers actually come from
The gateway does not assert a fixed savings percentage; it reads unit prices from a table. backend/smartgate/modules/budget_guard/model_prices.json lists per-token costs and context ceilings per model. deepseek-chat is priced at 0.00000014 per input token and 0.00000028 per output token, with a 65,536-token input ceiling and an 8,192-token output ceiling. gpt-4o is 0.0000025 input and 0.00001 output at a 128,000-token input ceiling. gpt-4o-mini is 0.00000015 input and 0.0000006 output. claude-3-5-sonnet-20241022 is 0.000003 input and 0.000015 output at a 200,000-token input ceiling, and claude-3-haiku-20240307 is 0.00000025 input and 0.00000125 output. Claude Haiku costs roughly a twelfth of Sonnet per input token, so identical text can bill very differently depending on which model the host routes to. The spread across those rows is why cost control cannot stop at "the model is cheap."
Token counting is model-aware. In budget_guard/algorithm.py, _get_count_function selects an encoding by model family: names containing gpt-4o or deepseek (case-insensitive) use o200k_base, other names resolve through tiktoken.encoding_for_model, and an unknown model degrades to cl100k_base. cost_per_token then multiplies prompt and completion tokens by the table's input and output prices and returns input, output, and total cost rounded to six decimals. The default model for a count call is deepseek-chat, so an unlabeled estimate still lands in a real price row.
How tool usage is counted
Not every tool reports exact tokens, so backend/smartgate/core/record_usage.py estimates when a module omits result.meta.token_used. Its token-inference helper uses a characters // 4 convention: a fetch is estimated as the length of its markdown (or content) field divided by four; a search sums the character length of each result's content or snippet and divides by four; a dedup is the text length divided by four; and a compress records the original token count reported by the module. Only compress, fetch, search, and dedup are billable, and budget_check, budget_count, and rate_limit are skipped from accounting entirely. Recorded totals land in Redis under the monthly usage key written by budget_guard/__init__.py, with a roughly 35-day expiry so the counter resets on its own.
Plan limits: monthly caps and rate limits
Two different ceilings apply. Per-plan monthly token caps live in config/plan-catalog.ts as monthlyTokenLimit: Free 2,000,000; Pro 20,000,000; Teams 100,000,000; Enterprise 200,000,000. The effective cap for a team is read from the team row by core/team_entitlement.py, which returns the stored monthly token limit and otherwise defaults to 1,000,000. Per-plan request rates live in core/plan_entitlements.py: Free allows 20 REST writes per minute and 30 MCP requests per minute per key with a team ceiling of 30; Pro 120/300/600; Teams 300/600/3000; Enterprise 600/1200/9999. Those values are cached for 60 seconds and fail safe to the Free set if the lookup errors.