Traffic Customs

Govern what agents fetch before it governs your bill

Uncontrolled fetch and search are the upstream cause of token bloat. Aggregate, extract, deduplicate, and compress — as one pipe or step-by-step.

The pain

Agents treat the web as free context. They re-fetch the same URLs, pile overlapping snippets into prompt windows, and pass the cost to your host LLM. Traffic customs means shaping tool output before it becomes model input.

Live pipe: `research` template

search→fetch→dedup→context_gate

From PIPELINE_TEMPLATES · backend/smartgate/core/pipeline.py

smart_search

Multi-engine snippets when no definitive URL exists. Don't use when user already gave a URL.

smart_fetch

HTTP(S) → Markdown-friendly content. Never invent page text.

smart_pipe

Orchestrate research · read · remember — or custom step arrays.

Pipe templates

Template	Steps	Use when
`research`	search → fetch → dedup → context_gate	Open-ended research
`read`	fetch → context_gate	User supplied a URL
`remember`	search → memory add	Persist team facts across sessions

Before / after

Scenario	Before	With SmartGate
Research workflow	4 ad-hoc tool calls	One smart_pipe research template
Page content	Raw HTML in context	Markdown-friendly fetch output
Search overlap	Duplicate passages	dedup step in research pipe

MCP example

{ "tool": "smart_pipe", "arguments": {
  "template": "research",
  "params": { "query": "MCP gateway enterprise adoption 2026" }
}}

Connect MCP in 5 minutes

Read MCP setup docs

Tool rate limits

smart_search and smart_fetch share a 20 requests/minute per-team gateway cap — separate from MCP requests/minute per API key. See Docs for details.

Team memory

Team-scoped memory (add/search/get/delete). Session and team facts — not general world knowledge.