TrueFoundry MCP Gateway: An Enterprise Control Plane View

Short answer: An enterprise TrueFoundry MCP gateway is bought for its control plane, not for its connector list. What a large organisation needs is one place to author policy for LLM traffic and MCP tool calls, a mechanism that pushes that policy to every gateway plane in every cluster, enforcement that runs inside the request path without calling a central service, and an audit trail that spans environments. TrueFoundry documents exactly that split — a control plane, stateless gateway planes kept in sync over a queue, and, across network boundaries, one plane per cloud with gateway-to-gateway proxying. A single-point MCP gateway in one environment gives you none of the four.

Key takeaways

• A gateway is two systems: a control plane that holds configuration (models, teams, virtual accounts, rate and budget rules, routing) and gateway planes that serve traffic. Buy the first; the second is stateless and replaceable.
• The control plane is what makes policy consistent: the same rule reaches every plane, including a plane that cannot reach a private MCP server.
• Enforcement must be in the request path and in memory — a policy check that calls a central service on every tool call is a latency and availability problem, not a control plane.
• A single-point MCP gateway solves discovery and credential concentration for one environment; it does not give you policy distribution, cross-cluster consistency, or an audit trail that spans clusters.
• Start by writing down which policy domains must be identical everywhere (identity, tool access, budget, retention); that list, not the tool catalogue, decides whether you need a control plane.

---

truefoundry mcp gateway: what the control plane actually is

The most useful thing to understand about the TrueFoundry MCP gateway is that it is not one service. It is a control plane plus a fleet of gateway planes, and the documentation is explicit about the split. The control plane "manages the configuration for the AI Gateway like models, virtual models, users, teams, virtual accounts, rate and budget limiting config, legacy routing configuration, and related settings," and that configuration "is synced to the AI Gateway via the NATS queue." The gateway plane "handles all the requests from the users/applications to the LLM/MCP servers" (Gateway Plane Architecture).

The enforcement model is where this stops being architecture and becomes a purchasing fact. The gateway plane is written so that "there are no external calls in the pathway of executing a request from the client to the LLM provider (unless we are using cache)"; rate and budget checks, load balancing, authentication and authorization "are done in memory," and logs and metrics "are written to a queue in an async manner." The page goes further: the gateway "never fails a request even if the external queue is down." That is the operational definition of a control plane doing its job — policy is pushed down and evaluated locally, while the observability trail is pulled back up out of band. A gateway that had to ask a central service for permission on every tool call would make the control plane a single point of failure for the whole estate instead of a source of truth.

The control plane itself is documented as named components: a UI, backend microservices, PostgreSQL for configuration, blob storage, a queue (NATS) that carries data to the compute and gateway planes, an OpenTelemetry collector, an ingestor, and a controller (Control Plane Architecture). Two of those matter when you compare vendors. The database is where the registry and policies live, so who owns that database owns your governance state; the queue is the distribution mechanism, so its availability sets how fast a policy change becomes true everywhere. TrueFoundry ships the whole thing as SaaS, hybrid, or fully self-hosted in your own VPC (AI Gateway introduction) — the deployment question most enterprise buyers ask before any feature question.

The federated case is where the design stops being abstract. Enterprise MCP servers are routinely unreachable from each other: an analytics server in one cloud, a Microsoft 365 server in another, legacy tools on premises. TrueFoundry's answer is to deploy "gateway planes in each cloud environment, with all gateway planes connected to a central control plane," where "each gateway plane can only access MCP servers within its own environment" and cross-environment access "is achieved through gateway-to-gateway proxying" (Federated MCP Gateway). When a server is registered you record its internal URL and a proxy URL naming the plane that can reach it; a request that lands on the wrong plane is forwarded to the right one. A developer's client talks to one address while the traffic is routed, authenticated and logged across clouds under a single registry. That is the baseline any honest comparison starts from — the ordinary case of an what an MCP gateway is is a single environment, whereas the control-plane case is an estate.

llm gateway aws: when model traffic and tool traffic get separate control planes

"llm gateway aws" is a distinct phrase from "ai gateway aws" for a historical reason: teams put a gateway in front of language models first, long before agents started calling tools. That first generation of gateway is a reverse proxy in front of a model endpoint, and on AWS it is usually assembled from existing parts — an API layer for keys and throttling, an identity layer for callers, a logging path, and a rule about which model a caller may reach. None of that is wrong, but it governs model calls and stops at the tool boundary.

AWS now names the separation in its own documentation. Amazon Bedrock AgentCore publishes a control plane API reference and a data plane API reference as two separate surfaces (Amazon Bedrock AgentCore documentation), the same two-plane vocabulary. The lesson is not that one vendor copied another; it is that the two-plane shape is the correct shape, and that it must cover both kinds of traffic. If LLM requests are governed by one control plane and MCP tool calls by another, you have two policy authorities, two audit streams, and an inevitable drift between them — the model call is rate-limited by one system while the tool call it triggers is authorised by a second that has never heard of the first rule.

The failure mode is concrete. Suppose finance enforces a per-team token budget at the LLM gateway while security requires explicit approval for a write-capable MCP tool. If those rules live in different systems, nothing stops an agent from staying inside its token budget while invoking an unapproved write tool, because the system that knows the budget never evaluated the tool. A control plane holds both rule sets and applies them on the same request — token accounting and tool authorisation are two checks inside one decision, not two gates in two products.

That is why the open-source route deserves a fair hearing rather than a reflexive "no". A self-hosted LLM gateway such as the LiteLLM route can absolutely provide a control plane for model traffic, and if your MCP servers already sit behind your own identity provider you can extend it. The question is not "open source or commercial" but "does one control plane hold authority over both model calls and tool calls, and can it place a plane where my private MCP servers live?" If yes, a smaller footprint is a feature; if no, you have two authorities and will reconcile them by hand.

mcp server gateway: one entry point, many servers — and where that stops

Strip away the marketing and the minimal MCP server gateway is a well-understood machine: one endpoint in front of N MCP servers, presenting their tools as a single catalogue. AWS describes the aggregation behaviour precisely — MCP targets "operate in aggregation mode — the gateway acts as an MCP server whose capabilities combine those of all its MCP targets," so clients see "a single consolidated tools/list response" (MCP targets, Amazon Bedrock AgentCore). A client that would otherwise need a URL, a credential and a version check per server needs one of each. That is real value, and the AWS-native MCP gateway is the clearest example of it.

What it does not do is govern. Discovery, credential concentration and one URL are all data-plane conveniences. They answer "how does the agent reach the tools?" They do not answer "who may reach which tool, under what budget, with what record — and is that true in the next environment?" A gateway derived from an API-gateway product is stronger here because it brings a policy engine it already had, but the same boundary applies: a policy engine configured per process is a policy domain per process.

The limits of the single-point pattern show up in four places. Blast radius: one process fronting every server is one process whose failure affects every tool at once, with no second plane to fall back to. Environment asymmetry: the same catalogue is rarely correct for production, staging and a regulated region, so differences get managed outside the gateway. Identity: enterprise tool access usually has to follow the identity provider's groups rather than a shared key pasted into a client, and a single-process gateway holding one credential set cannot express per-user access. Audit: a gateway that logs where it lives produces a trail scoped to that environment, and a question that spans clusters cannot be answered from it.

Curation features deserve a fair reading, because they are often sold as governance. Combining tools from several servers into one virtual server is a genuine control — it narrows what an agent can see — but it curates the catalogue, not the caller. A virtual server exposing three tools to a team still has to answer whether this user may call them, and that answer lives in the access layer. The contrast is between a gateway that can only publish a catalogue and a tool-integration-first gateway that leans on its connector model: both can be excellent at getting tools connected, and neither is, by itself, a control plane.

microsoft ai gateway: a comparator built on an API-management foundation

Microsoft's entry is instructive precisely because it did not build a new gateway. "The AI gateway, including MCP server capabilities, extends API Management's existing API gateway; it's not a separate offering" (AI gateway capabilities in Azure API Management). The AI Gateway tier (in public preview) is "a fully managed gateway for AI workloads" that gives teams "one place to publish, secure, govern, and observe access to AI models and Model Context Protocol (MCP) tools," where a single MCP server endpoint "federates one or more backends from three sources: a remote MCP server, an OpenAPI spec, or a built-in connector" (AI Gateway tier overview).

Read against the control-plane frame, two things stand out. The first is the policy engine as the control plane: token rate limits, request rate limits, content safety and IP filters are configured as policies on the gateway, not in each application, and the gateway holds backend credentials so "applications never handle provider keys." That mirrors TrueFoundry's in-memory enforcement — the policy is evaluated at a boundary the organisation controls. The second is the inheritance cost. Because AI capability rides on API Management, the buyer inherits that product's tiers, scale-unit planning, upgrade calendar and preview status; the tier has no service-level agreement during preview and preview quotas can cap the number of models, tools and keys. That inheritance is a benefit if you already run API Management, and the real decision if you do not.

The design also raises a question to ask every vendor: is MCP governance first-class, or a capability of the surrounding product? Here the MCP side is real and documented — one governed endpoint per MCP server, federating remote servers, OpenAPI specs and connectors — but it is part of a wider API platform whose primary job is API governance. TrueFoundry's framing is the mirror image: MCP and LLM traffic share a purpose-built gateway, and the surrounding platform is the control plane. Neither is automatically right. The test is whether policy authored once reaches every plane that can touch a tool, and whether the record of a tool call is retrievable beside the record of the model call. The Microsoft-stack route is the right first question if API Management is already your estate's front door.

openrouter vs vercel ai gateway: a vendor-operated control plane

This comparison is really two managed routing services, and comparing them honestly means naming what they are. OpenRouter "provides a unified API that gives you access to hundreds of AI models through a single endpoint, while automatically handling fallbacks and selecting the most cost-effective options" (OpenRouter quickstart). Vercel's AI Gateway "gives applications and coding agents shared access to models across providers," routes "requests across providers and fallback models, then records status, provider, latency, token usage, cost, and every routing attempt," and supports budgets, access policies and Bring Your Own Key with zero markup on token prices (Vercel AI Gateway).

Both are excellent at what they were built for, and Vercel states the trade explicitly: use AI Gateway "when you want these controls without operating your own proxy, database, and routing control plane." That sentence is the whole argument. You are not buying your control plane; you are buying a control plane, operated by the vendor, and accepting its terms about where configuration lives, where logs live, which providers are reachable, and how policy is expressed. For a small team shipping fast that is often correct — the alternative is a database, a queue and a fleet you have to run.

For an enterprise with MCP servers inside private networks, the calculus changes. A managed routing control plane governs the model call; it does not deploy a plane inside your VPC to reach a database tool addressable only from your own network, and it does not give you a registry whose access policies you can point at your identity provider. A tool-integration-first gateway bets differently: it concentrates connectivity and credentials so agents have fewer moving parts, while leaving open where the authorisation decision for a regulated internal tool is recorded. The question to carry into any evaluation is narrow: for one tool call to one private server, name the system that holds the rule, the one that enforces it, and the one that records it. If those are not the same control plane, you have a managed routing layer, not a governed one.

ai gateway aws: the fully managed single entry point

Where "llm gateway aws" describes the assembled proxy, "ai gateway aws" points at what AWS now ships as a product. Amazon Bedrock AgentCore Gateway is "a fully managed AI gateway that provides a single, secure entry point for agentic traffic — connecting agents to tools, to other agents, and to large language models (LLMs)" (AgentCore Gateway). It is a stronger shape than the proxy: it fronts tools as well as models, aggregates MCP targets into one consolidated catalogue, and supports inbound and outbound authorisation.

The control-plane question still applies, and AWS answers part of it well. The gateway is configured through a control plane and serves through a data plane — the two API references are published separately — so registry and traffic path are distinct, which is correct. The gateway is also a managed resource, so availability and regional placement are AWS's responsibility. What a managed single-vendor gateway does not do is place planes in environments AWS does not operate. If a tool lives in another cloud or on premises behind a boundary, the federated pattern — a plane per environment, all subscribed to one control plane — is something you assemble around the gateway rather than something it gives you.

That is why the control-plane lens keeps paying off. The question is never whether a managed gateway is good — it usually is; it is scope. How many environments does the control plane reach, and can the same policy and audit trail span them? A managed gateway covering every environment you actually have is a control plane; one covering a single environment while your tools live in three is a very good edge.

What an enterprise is really buying

Pull the vendor names away and a pattern appears. The properties that separate a control-plane gateway from a single-point MCP gateway are not features on a comparison sheet; they are the five things an organisation must be able to do once it runs agents at scale.

Property	Single-point MCP gateway	Control-plane gateway
Policy authorship	Rules configured per process, often per environment	One place holding models, teams, budgets and tool policy together
Distribution	Each process updated by hand or by its own deployment	Configuration synced to every plane over a queue or equivalent
In-path enforcement	Whatever the process can check locally	Identity, rate, budget and tool rules evaluated in memory on the request path
Cross-environment consistency	Catalogue and rules drift between clusters and clouds	One registry; a plane that cannot reach a server still enforces its policy via a peer plane
Audit	A log scoped to the environment the gateway runs in	One record of who called what, retrievable across environments

The first two rows are what make the last three possible. A policy authored in one place and distributed automatically can be assumed consistent; a policy configured per process can only be hoped consistent, and the gap widens with every new environment. That is why "how many environments does one control plane cover?" is a better opening question than "how many MCP servers does it support?" The server count is a data-plane capacity number; the environment count is the governance question.

A second-order benefit buyers underestimate: a control plane lets a plane that cannot see a private server still enforce the rule written for it. With registry and policy held centrally, a tool call proxied from one environment to another is evaluated against the same access rule and lands in the same audit stream. Without that, every cross-boundary tool call is a governance exception, and exceptions are where compliance programmes quietly fail.

TrueFoundry-class layering: how our own gateway resolves settings

One codebase becomes several deployments through three layers, and naming them is the difference between configuration you can reason about and configuration you debug. The first layer is the file: backend/smartgate/core/config.py loads config.yaml, falling back to config.yaml.example when it is absent, and resolves environment placeholders that carry an inline default at read time — so one image ships many topologies and the differences live in the environment rather than in the artifact. The pydantic Settings class is the second layer and owns the secrets: DATABASE_URL has no default and must arrive from the environment or the .env file, while SMARTGATE_API_KEY_SALT falls back to a name that says it is unsafe, so a missing salt is visible rather than silent.

The third layer is per request. core/effective_settings.py parses an X-SmartGate-Settings JSON header arriving from the front door and merges it section by section over PLATFORM_DEFAULTS_V1 — compression ratio, budget model, playground cap, governance switches, analytics flags — carrying a schema version so an older caller is not silently reinterpreted. Invalid or absent JSON falls back to the platform defaults, and each consumer resolves in the order request body, then header override, then platform default. Same binary, three places a value can come from, and a stated precedence for every one of them.

Where SmartGate fits

SmartGate is an MCP-native algorithm gateway, and it sits on the same axis this page has drawn: the difference between plumbing that connects tools and a control surface that governs them. Its operational limits belong in the control-plane conversation rather than a feature list — monthly token caps of 2M, 20M, 100M and 200M-plus; requests per minute per key of 120, 300, 600 and 1200; audit-log retention of 7, 30, 90 and 180 days; and team-key limits of 2, 10, 30 and effectively unlimited on the top tier. Audit-log retention is the column that answers to a compliance deadline rather than a preference, so read it against your own obligation before committing a date to anyone. Treat those figures as directional and read the current numbers from the pricing page.

Those limits are the visible edge of the control plane. A token cap and a requests-per-minute limit are policy; an audit-retention window is the record; a key count is the unit of identity. Comparing gateways on those four numbers is comparing control planes, whether or not a vendor uses the word. Where SmartGate's model differs from a managed routing service is that the budget and audit surface is the product rather than an add-on: the platform is priced so its cost is meant to be self-funding against the savings it produces, with a share taken only after a saving floor is cleared. A gateway whose own incentives track the saving defaults to enforcing limits rather than reporting them after the invoice.

How to get started

The first three steps need no purchase and no vendor.

1. Write down the policy domains that must be identical everywhere — identity and tool access, budget, data retention, and the record of who called what. This list, not the tool catalogue, tells you whether a control plane is mandatory.
2. Map your MCP servers by network boundary: which are reachable from where, and which only from inside a specific cloud or VPC. Every boundary you cross is a place where a single-point gateway stops and a federated plane starts.
3. For one representative private tool call, name the system that holds the rule, the system that enforces it, and the system that records it. If those are not one control plane, you have found your gap.
4. Only then choose a shape: a managed single-environment gateway if that is genuinely all you have; a control plane with planes per environment if your tools already span boundaries.
5. To test the enforcement model rather than the marketing, connect a client and run one real tool call end to end, then read the audit record it produced. Start free with an MCP-speaking client, and read the pricing page once real call volume tells you which tier you need.

Limitations

This page is a selection frame, not a benchmark: no product is ranked, because the right answer depends on where an organisation's identity, data and record obligations already live. The control-plane and single-point categories are a useful distinction, not a claim that any named product belongs cleanly to one side: real systems overlap, and a single-vendor managed gateway can be a control plane for some estates and a very good edge for others.

The vendor facts quoted here are what each vendor's own public documentation states, checked on 2026-10-03; several are preview features whose names, tiers and quotas can change without notice, and this page restates no pricing, because a plan figure from a secondary source is worse than no figure at all. The demand figures are our own paid measurements for the United States over a twelve-month window and are recorded in this project's measurement files; they describe how many people search, not the topic's worth to a team, and they will age.

Because this page carries no code excerpt — the reason is recorded in the Method note below — it makes no line-numbered or implementation-level claim about any product, including ours. The five control-plane properties describe what to check; they are not a promise that any specific gateway passes them.

Sources

• TrueFoundry — Gateway Plane Architecture and Control Plane Architecture, for the control-plane / gateway-plane split, NATS-synced configuration and in-memory enforcement.
• TrueFoundry — Federated MCP Gateway, for one plane per cloud environment, proxy URLs and gateway-to-gateway proxying under a central control plane.
• TrueFoundry — MCP Gateway overview and the AI Gateway introduction, for the registry, unified token management, tool-level access control and deployment options.
• Amazon Web Services — Amazon Bedrock AgentCore Gateway and MCP targets, for the managed single entry point and MCP aggregation mode.
• Microsoft — AI gateway capabilities in Azure API Management and the AI Gateway tier overview, for policy-based governance of models and MCP tools and its preview status.
• Vercel — AI Gateway, for managed routing, recorded routing attempts and the "without operating your own control plane" framing.
• OpenRouter — Quickstart, for the unified model endpoint and automatic fallback behaviour.
• The Model Context Protocol specification — modelcontextprotocol.io/specification, for the tool-call interface every gateway in this comparison speaks.
• Demand figures come from our own paid measurement run and are recorded in this project's search_volume.json; the vendor facts above were read from the linked pages on 2026-10-03.

Method note

This page carries no code excerpt, and that is a recorded finding rather than an omission. The slice matcher pinned 0 of 6 sections for this page (0 abstention(s), 6 no-slice verdict(s)), because every keyword in this lane is a product or comparison phrase rather than a code symbol, and the remote candidate fallback returned only generic or unrelated containers. Rather than quote a container that says nothing about any section, the page is written from the vendors' own published documentation, with a link for every claim. No code, batch fingerprints, auction data or internal hosts are transcribed.

The slice run for this page recorded 0 of 6 sections pinned, 0 abstention(s) and 6 no-slice verdict(s); BLOCKS is empty because those sections are product- and comparison-phrase lanes with no unique symbol to pin, as the Method note above explains.