Enterprise AI Gateway Architecture Best Practices: MCP Gateway

Short answer: the MCP endpoint is the visible half of an enterprise gateway; the half that enforces anything is the application around it. Eight decisions carry the weight: a tenant provisioned with a plan and a token limit, an identity resolved per request, canonicalized inputs before any comparison, policy read at runtime, prefix-scoped exclusions, validated tool parameters, constant-time secret comparison, and plan refusals that arrive as typed errors.

Key takeaways

• Limits are provisioned, not computed. newFreeTeamCreateData copies the plan's monthly token limit onto the team at creation, so the quota check always has a number to compare.
• Identity and authorization are two steps, not one. getUserByEmail returns a principal or null and never throws, which keeps the 401-versus-403 decision in a single place.
• Canonicalize before you compare. normalizePath gives one spelling per resource; if the writer and the checker disagree about spelling, a policy has a hole neither test can see.
• Configuration is data with a closed default. getSitemapConfig reads both key lists in one query, normalizes every entry, and falls back to the smallest authority on error.
• Scope is a subtree, not a substring. isPathExcluded is prefix-aware at the slash boundary, which is exactly the rule an agent memory namespace needs.
• Validate shapes, reject loudly, compare secrets in constant time. hexToBytes throws on malformed input instead of zero-filling; timingSafeEqual reads every byte wherever the difference is.
• Do this next: write down, on one page, what your gateway knows about a caller before its first tool call — tenant, plan, token limit, principal, and the scope it may reach. Whatever is missing from that page is missing from your enforcement.

---

Tenancy before traffic: what an enterprise AI gateway provisions first

# lib/team.ts — source lines 17–30 (newFreeTeamCreateData)
function newFreeTeamCreateData(
  user: { name?: string | null; email?: string | null },
  userId: string,
) {
  const free = getPlanCatalogEntry("FREE");
  return {
    name: `${user.name || user.email || "User"}'s Team`,
    slug: slugify(user.name || user.email || "User"),
    plan: "FREE" as const,
    monthlyTokenLimit: free.monthlyTokenLimit,
    features: free.features as object,
    members: { create: { userId, role: "OWNER" as const } },
  };
}

A gateway can only enforce a limit on something it has already described, so three facts have to exist before the first tool call: an identity, a tenant and a plan. Here the tenant is written with its plan and its monthly token limit in the same function that names it — getPlanCatalogEntry("FREE") supplies the limit and the feature flags, and the creator is inserted as an OWNER member in the same nested write.

That ordering is the decision. A limit provisioned at creation is data the quota check can compare against; a limit computed on the first request leaves a state where nobody decided anything, and the default reading of that state — no limit configured, so no ceiling — is the one that appears on an invoice a month later. Reading the value from the plan catalog also makes a plan change a migration of one row instead of a new branch in the enforcement path.

The failure mode this avoids is the common one: minting the agent's credentials before the tenant exists. The key works, the tools run, and every audit row afterwards is unattributable to a plan. Provision the tenant, write the limit, then issue the credential — that stored limit is what the monthly check later reads, and the arithmetic of holding it per team is worked through in enforcing a token quota per team.

The identity layer in an AI gateway architecture

# lib/user.ts — source lines 3–19 (getUserByEmail)
export const getUserByEmail = async (email: string) => {
  try {
    const user = await prisma.user.findUnique({
      where: {
        email: email,
      },
      select: {
        name: true,
        emailVerified: true,
      },
    });

    return user;
  } catch {
    return null;
  }
};

An AI gateway architecture has four separable jobs in its request path — parse the credential, resolve the principal, authorize the action, enforce the limit — and the cleanest thing you can do is keep them in different functions. This resolver is the second job: it returns the user record, or null. It catches its own failure and hands the caller a value rather than an exception.

The shape pays off twice. A lookup that returns a fact lets the transport layer own the status codes, so "no principal" becomes 401 and "principal without membership" becomes 403 in one file instead of being re-derived by every caller that catches a generic error. And a resolver with no side effects is safe to call on any path, including the ones an audit has to describe after the fact.

The query is scoped to two fields, name and emailVerified — least privilege in a select clause is cheap, and it is the reason this function cannot leak a credential hash into a log line. authentication for MCP traffic covers the credential side; the architectural point is that OAuth tokens, API keys and a development bypass should resolve to one internal principal, because per-key rate limiting and per-actor audit rows only compare when the actor is the same kind of thing. Skipping the resolver — and letting a client assert X-User-Id — moves an authorization decision from your gateway to your caller.

Canonicalize input before any comparison: the enterprise LLM gateway rule

# lib/sitemap-config.ts — source lines 11–14 (normalizePath)
function normalizePath(path: string): string {
  let normalized = path.startsWith("/") ? path : `/${path}`;
  return normalized.replace(/\/+$/, "") || "/";
}

Every enforcement decision in an enterprise LLM gateway is a comparison: this route against that policy entry, this model alias against that allow-list, this tool name against that plan. A comparison is only as good as the two strings, so canonicalization belongs at the boundary where a value enters — request parsing, configuration load, policy save — and every internal comparison works on canonical values.

The function here is four lines and makes three decisions. A path without a leading slash gets one, so a resource has a single spelling. Trailing slashes are stripped, so /v1/models/ and /v1/models cannot be two different resources in a policy table that lists only one of them. An empty result collapses to / rather than the empty string, so "normalized to nothing" cannot slip past a lookup.

One implementation is what prevents an asymmetry you cannot test your way out of. If policy is stored normalized but requests are matched raw — or the reverse — a rule written for /v1/models is bypassed by /v1/models/, and both functions are individually correct, so neither unit test fails. The same bug appears wherever identifiers cross a boundary: aliases differing by case or registry prefix, tool names carrying a namespace, slugs with a trailing character. And normalizing is not validating — a canonical path can still be a forbidden one. AI gateway vs API gateway is the same equality question one level up: which hop a request goes through at all.

Configuration: what a Model Context Protocol gateway reads at runtime

# lib/sitemap-config.ts — source lines 21–40 (getSitemapConfig)
async function getSitemapConfig(): Promise<SitemapConfig> {
  try {
    const configs = await prisma.systemConfig.findMany({
      where: {
        key: { in: ["SITEMAP_EXCLUDED_PATHS", "SITEMAP_ADDITIONAL_PATHS"] },
      },
    });
    return {
      excludedPaths: parseConfigList(
        configs.find((c) => c.key === "SITEMAP_EXCLUDED_PATHS")?.value,
      ).map(normalizePath),
      additionalPaths: parseConfigList(
        configs.find((c) => c.key === "SITEMAP_ADDITIONAL_PATHS")?.value,
      ).map(normalizePath),
    };
  } catch (err) {
    console.error("Failed to fetch sitemap config from DB:", err);
    return { excludedPaths: [], additionalPaths: [] };
  }
}

Configuration is data, and the loader is where its shape gets fixed for everyone else. This one reads two keys from a systemConfig table in one query, splits each stored string into a list, normalizes every entry, and returns the excluded paths and the additional ones. One naming note before the code: what the MCP acronym means is worth pinning down first, because this loader reads configuration the protocol itself never defines.

Four properties transfer to a Model Context Protocol gateway's policy. One read for the whole document: fetching each key on demand multiplies the round trips a request pays for and makes a partially loaded policy possible. Parsing at the boundary: the stored string becomes a list exactly once, so no consumer has to know the storage format. Normalization as it loads (.map(normalizePath)): two consumers can never disagree about spelling.

The fourth is the interesting one. The error path returns empty lists, and that is conservative only after you have decided which direction "empty" falls: for an allow-list, empty means deny, which is the safe branch, while for a deny-list empty means allow everything, which is not. If the same loader serves the control that decides which tools, models or memory namespaces are reachable, store it as an allow-list, so a database error narrows the gateway instead of opening it. Logging the failure, as this loader does, keeps that trade visible. The token half of the same control plane is set out in token optimization techniques, and the placement of the gateway in the request path is where the gateway sits.

Scope, not just permission: what an agent memory may touch

# lib/sitemap-config.ts — source lines 42–51 (isPathExcluded)
function isPathExcluded(path: string, excludedPaths: string[]): boolean {
  const normalized = normalizePath(path);
  return excludedPaths.some((ex) => {
    const normalizedEx = normalizePath(ex);
    return (
      normalized === normalizedEx ||
      normalized.startsWith(`${normalizedEx}/`)
    );
  });
}

Permission answers "may this caller run this tool". Scope answers the harder question an agent memory forces: which data may this run read, write or remember. The matcher here is one function, and the line that matters is the boundary in its prefix test — a path is excluded when it equals the excluded path or starts with the excluded path plus a slash.

Without that slash, an exclusion for /teams/acme would also exclude /teams/acme-prod and every other path sharing the prefix. It is the classic prefix bug, and in a memory system it is a cross-tenant bug. With the slash, an exclusion names a subtree — exactly the construction a memory namespace needs: a run may write under teams/<id>/, and teams/<id>-staging/ is a different subtree that has to be granted separately. Two agents sharing a prefix must never share a store by accident, and the guarantee is a delimiter, not a string comparison. Normalization happens inside the matcher, on both sides, which is why a rule from a settings form and data from the memory tool can be compared at all — at the cost of O(n) work per check over a list that only grows, so past a few hundred entries the first path segment wants to become the index.

The last decision is the direction of the list. An exclusion list is a deny-list whose default is "everything else is allowed", so every new namespace is born visible; for agent memory and retrieval corpora, invert it into an allow-list of the namespaces a workflow may touch. Quota bounds how much a run costs, scope bounds what it can reach — and only one of them can be enforced by counting. agent memory architecture covers what belongs in that store.

Input validation: where the MCP spec stops and the gateway starts

# lib/crypto.ts — source lines 27–37 (hexToBytes)
function hexToBytes(hex: string): Uint8Array {
  const normalized = String(hex || "").trim().toLowerCase();
  if (!/^[0-9a-f]*$/.test(normalized) || normalized.length % 2 !== 0) {
    throw new Error("Invalid hex string");
  }
  const out = new Uint8Array(normalized.length / 2);
  for (let i = 0; i < out.length; i++) {
    out[i] = parseInt(normalized.slice(i * 2, i * 2 + 2), 16);
  }
  return out;
}

The decoder here is eleven lines long and contains the three decisions every gateway's input validation needs. It normalizes first — trim, lowercase. It validates the shape before it allocates anything: a character-class test plus an even-length check, both of which reject rather than repair. And it throws on a malformed value instead of returning a partially filled buffer, because a decoder that silently zero-fills turns a caller's bug into a security event.

The MCP spec illustrates why this is the implementer's problem. It defines message shapes, method names and a transport; it does not say what your server has to reject, how large a payload may be, or which fields a plan bounds. Those are your decisions, and the cheapest place to make them is the boundary where a tool's arguments arrive.

Three rules make that boundary earn its place. Validate once, at entry, and pass typed values inward — validation repeated in two places drifts, and the drift is found by whichever caller hits the other copy. Choose reject-or-coerce per field class: coerce enumerations and optional filters so a typo narrows a request, and reject identifiers, hex values, keys and cursors, because a typo there addresses a different object. And bound the expensive dimensions before any work starts — text length, list depth, page size, steps in a chain — since an unbounded parameter is a denial-of-service vector that costs tokens as well as latency. An error should carry the offending field and the shape that arrived; the MCP tools reference is where those shapes belong, so validation and documentation come from one description.

Secret comparison: the MCP specification says bearer token, not how to compare it

# lib/crypto.ts — source lines 39–48 (timingSafeEqual)
function timingSafeEqual(a: ArrayLike<number>, b: ArrayLike<number>): boolean {
  const lenA = a.length;
  const lenB = b.length;
  if (lenA !== lenB) return false;
  let diff = 0;
  for (let i = 0; i < lenA; i++) {
    diff |= ((a[i] as number) ^ (b[i] as number)) & 0xff;
  }
  return diff === 0;
}

The MCP specification is explicit about the credential — an HTTP transport may act as an OAuth 2.1 resource server and accept bearer tokens — and silent about the comparison that follows. That is the right division for a standard and a careless place for an implementation, because the ordinary equality operator returns as soon as it finds a difference, and an attacker who controls the input and can measure response time can recover a secret one byte at a time.

This function is the answer in nine lines. It checks length first, which reveals only the length and is rarely a secret. Then it walks the whole buffer, XOR-accumulating each byte into one integer and masking with & 0xff, and compares once at the end. The work is identical wherever the first difference sits, and the result is a boolean, so no caller can branch on a partial match.

Two details matter. The inputs are ArrayLike<number>, so the same helper verifies a hex-decoded header — the decoder above produces exactly that — and an array of character codes: one comparison primitive for every secret in the request path. And the length check is a deliberate early return; the rule to keep is that content-dependent work stays branch-free. Its place in a gateway is a short list: API-key comparison, webhook and HMAC signature verification, and anything answering "is this the token you issued". Compare digests rather than raw secrets when the secret is stored, so a timing leak exposes a hash. And keep the comparison on one code path — a cache that returns early for known-good keys re-introduces the branch you just removed. The contract this sits inside is walked through in the specification walkthrough.

Plan gates: what a Model Context Protocol server returns when the plan says no

# lib/exceptions.ts — source lines 2–4 (constructor)
constructor(message = "This action requires a pro plan") {
    super(message)
  }

A plan gate is the gateway failure a client meets most often, and its shape decides whether an operator fixes the problem in five minutes or files a ticket. The class here carries a default message — this action requires a pro plan — so every raising site in a Model Context Protocol server produces the same actionable sentence, and a subclass changes the message only to be more specific. Three properties are worth requiring of the path.

It has to be typed. A distinct error class lets the request layer map the refusal onto a stable code and a 4xx status. If plan refusals arrive as a generic 500, clients retry, the retries count against the rate limit, and your dashboard shows an outage whose real cause is a misconfigured plan. Typed errors are also what let an audit tell "asked and was refused" from "crashed". And it has to name the way forward — the plan that would unlock the tool, the capability involved, the plan the caller is on. "Forbidden" is not actionable; "smart_pipe is available on Teams and above, and this team is on Free" is.

The third property is timing: the gate belongs at tool entry, before retrieval, before compression, before a model call or a memory write, so a refused call costs nothing and leaves no audit row that later reads as usage. MCP logging and observability is where telling those two rows apart becomes a habit rather than an incident review. Whether the tool list should also be filtered per plan is the open question: hiding smart_pipe from a Free client's tools/list makes the refusal rare, but the client has to refresh its cached list after an upgrade and the server-side check stays necessary — the tool list is discovery, not enforcement.

The short version for whoever signs the invoice

Agents do not warn before they spend: an unsupervised loop calls tools, grows context and bills until something external stops it. That is why the gateway, not the model, is the control point a security review asks about — who called, what they were allowed to call, what it cost, and can you prove it six weeks later.

SmartGate is an MCP-native algorithm gateway for token control, traffic shaping, and agent audit. It is not a model host: you keep your model and your agents, each client gets one MCP endpoint, and the gateway holds seven metered tools — smart_fetch, smart_search, smart_context_gate, smart_dedup, smart_budget_guard, smart_memory and smart_pipe. The plan ladder is the enforcement vocabulary rather than a price list: Free covers 2 million tokens a month with 120 requests per minute per key and 7-day logs; Pro, from $18/month with the first month at $5, raises that to 300 and 30 days; Teams, at $55/month, to 600 and 90 days; Enterprise is contractual, with 200M+ pooled tokens, 1,200 requests per minute per key and 180 days. Billing follows "Pay for the platform. Share only when you save." — the savings share begins after $15 of measured savings, so the platform fee is the only line item you can forecast (pricing).

What the standards already require

The MCP specification is deliberately thin here. Authorization is optional; where a transport supports it, the server acts as an OAuth 2.1 resource server using bearer tokens (MCP authorization), and the transports page defines one Streamable HTTP endpoint with sessions optional (MCP transports). Nothing obliges a server to bound how often a client calls a tool, cap what those calls cost, or keep a record.

The risk literature names the same gaps without closing them. OWASP's Top 10 for LLM applications lists Excessive Agency (LLM06:2025) and Unbounded Consumption (LLM10:2025) (OWASP LLM Top 10) — the tool-scope problem and the bill problem, neither fixed by choosing a different model. NIST's AI Risk Management Framework frames the work as govern, map, measure and manage (NIST AI RMF). All three describe the obligation and leave the mechanism to the implementer, which is why a review asks where enforcement lives.

How SmartGate compares

The category splits by what sits in the request path and what it can stop. SmartGate's scope is deliberately narrow: it governs tool traffic.

	What it governs	How you run it	What you pay
SmartGate	MCP tool calls: fetch, search, compression, dedup, budgets, memory, pipelines	Hosted gateway; one MCP endpoint per client	Free covers 2M tokens/mo with 120 req/min/key and 7-day logs; Pro from $18/mo (first month $5) (pricing)
LLM routers and proxies (e.g. LiteLLM, Portkey)	Model calls: provider routing, keys, model-level spend	Self-hosted or hosted, in front of the model API	Usage-based on model traffic; tool calls sit outside their scope (LiteLLM, Portkey)
Cloudflare AI Gateway	Model provider traffic: caching, analytics, provider limits	Hosted at the edge, per provider route	Free tier plus usage; a different hop from your tool calls (Cloudflare)
Kong AI Gateway	Model and API traffic inside your own gateway estate	Self-hosted; you operate and upgrade it	Platform cost plus the engineering time to run it (Kong)
In-house middleware or MCP bridges	Whatever you build, one capability at a time	You own the runtime and the retries	Engineering time; no budget, rate-limit or audit layer until you write it

Leaving a proxy you already operate is a migration rather than a shopping decision, and the five criteria that decide it - protocol, credentials, record, limits, carry-over - are set out in the LiteLLM alternative framework.

Two shapes come up in every scoping conversation, and both are worth reading in full before you pick one: a gateway that ships inside the data platform and borrows that platform's own permission model (Databricks AI Gateway as a case), and a lightweight self-hosted gateway you run beside your own services (the self-hosted shape and when it pays).

If your problem is model routing, a router is the right tool and this gateway is not a substitute for it. If agents call tools with credentials nobody tracks, a team can spend past its plan, or nobody can reconstruct last month, those are tool-side concerns a router will not answer. If what you actually searched for was an OpenRouter alternative, the layer question is the whole answer, and the router-versus-tool-gateway comparison takes it apart row by row.

How to get started

1. Create the team and mint a key. Sign in at smartgate.network, create the team, and copy the key for the environment you are wiring — the plan and its token limit are written at that moment.
2. Set the monthly token limit before connecting anything. The quota check falls back to the plan default when no explicit limit is configured; an explicit limit is what makes the exceeded flag mean something.
3. Mount one client at a time. Point one agent at the Streamable HTTP endpoint, call a tool, and confirm the audit row carries the actor you expect. Running the gateway inside your own network changes the deployment contract but not the enforcement order — see the private-cloud deployment. A worked single-client setup, down to the config key and the two auth headers a host sends, is connecting OpenClaw over MCP.
4. Check the plan numbers against your workload. Compare expected requests per minute with your tier's per-key limit and the team ceiling above it — clients that share a key share its limit.
5. Wrap loops with smart_budget_guard, and move multi-step work to smart_pipe. One audited trace per workflow is what keeps the log readable a month later.

Start on Free — 2 million tokens a month, all seven tools, 120 requests per minute per key, no card: start free, then compare the caps and retention windows your workload needs on the pricing page.

Limitations and what this does not do

• This is the application side of a gateway, not a network design. Subnets, TLS termination, secret distribution and backups are yours. A component that forwards the connection and decides nothing is a proxy, not this layer, and the line between the three names — proxy, router, gateway — is drawn in MCP proxy versus router versus gateway.
• It governs tool traffic, not model traffic. Provider routing, model caching and prompt-level spend belong to a router or to the model provider — a managed gateway, dissected is where that half of the stack is worked through.
• Scope and quota answer different questions. A budget cap is not a sandbox: capping spend does not make an unsafe tool safe, and the gateway cannot judge whether a fetched page should have been fetched. OWASP's excessive-agency risk shrinks by scoping tools, not by metering them.
• Limits are per key plus a team ceiling, not per agent role. Two agents sharing a key share its limit; independent budgets need independent keys.
• Retention is an entitlement and the job prunes on schedule. Export before the window closes if your compliance requirement outlives the plan's window.

One thing this page assumes rather than argues: what the category itself is, and where its edge sits against proxies, orchestration and observability. That definitional ground is worked through on LLM gateways; read the two together before scoping a deployment, because a private-cloud rollout of the wrong category is an expensive way to learn the difference.

Sources

• Model Context Protocol — specification index: https://modelcontextprotocol.io/specification/2026-07-28
• Model Context Protocol — authorization (OAuth 2.1 resource server, optional): https://modelcontextprotocol.io/specification/2026-07-28/basic/authorization
• Model Context Protocol — transports (Streamable HTTP, stateless servers): https://modelcontextprotocol.io/specification/2026-07-28/basic/transports
• OWASP Top 10 for LLM applications — Excessive Agency (LLM06:2025), Unbounded Consumption (LLM10:2025): https://genai.owasp.org/llm-top-10/
• NIST — AI Risk Management Framework: https://www.nist.gov/itl/ai-risk-management-framework
• SmartGate — audit and compliance: https://smartgate.network/features/audit-compliance
• SmartGate — pricing and plan caps: https://smartgate.network/pricing
• SmartGate — sign in / start free: https://smartgate.network/login?from=/dashboard
• LiteLLM — proxy documentation (model routing): https://docs.litellm.ai/
• Portkey — gateway documentation (model routing): https://portkey.ai/docs
• Cloudflare — AI Gateway documentation: https://developers.cloudflare.com/ai-gateway/
• Kong — AI Gateway documentation: https://developer.konghq.com/ai-gateway/

Method note

The code in this article is not transcribed. Each block was cut out of the slice body returned by the SmartGate slice API and re-asserted byte-for-byte as a substring of that body before publication; the first line inside every fence records the file and the exact source lines. Symbols were pinned with whole-name containment (rule A level 2) and confirmed by the service's slot-proof endpoint. One excerpt is three lines long: the plan-gate example is a constructor, so its class and the site that raises it are described in prose rather than copied.

Slice provenance

#	SERP keyword	Symbol	File	Source lines	How it was pinned	sha256(12)
1	enterprise ai gateway	newFreeTeamCreateData	lib/team.ts	17–30	rule A L0 → remote symbol candidates → slot-proof	4d810a8b9613
2	ai gateway architecture	getUserByEmail	lib/user.ts	3–19	rule A L0 → remote symbol candidates → slot-proof	d4be0d4cb56f
3	enterprise llm gateway	normalizePath	lib/sitemap-config.ts	11–14	rule A L0 → remote symbol candidates → slot-proof	2eb5193f7ae1
4	model context protocol mcp	getSitemapConfig	lib/sitemap-config.ts	21–40	rule A L0 → remote symbol candidates → slot-proof	181ec0c1fb02
5	agent memory	isPathExcluded	lib/sitemap-config.ts	42–51	rule A L3 → slot-proof	8f1268b57f71
6	mcp spec	hexToBytes	lib/crypto.ts	27–37	rule A L0 → remote symbol candidates → slot-proof	b52a5e2f7dd3
7	mcp specification	timingSafeEqual	lib/crypto.ts	39–48	rule A L0 → remote symbol candidates → slot-proof	d08fe27e2863
8	model context protocol server	constructor	lib/exceptions.ts	2–4	rule A L0 → remote symbol candidates → slot-proof	acd7732e5804

Every fenced block above was cut from the slice body and re-asserted against it byte-for-byte before publication. 8 of 8 sections pinned, 0 abstentions, 0 misses.