Production AI Agents Need an MCP Gateway

Two MCP servers in a developer notebook is fine. Forty MCP servers across production agents, with auth tokens, audit trails, and cost ceilings, is not. The Model Context Protocol won the wire format war; the deployment story is still open. Treating MCP like every other L7 protocol and putting a gateway in front is the only path that survives the second quarter of production.

What MCP actually is in 2026¶

The Model Context Protocol (MCP) is the JSON-RPC protocol that gives an LLM agent standardised access to tools, resources, and prompts. The spec is at version 2026-07-28, released 2026-07-28, and it's now what every major agent framework speaks.¹

A typical agent today uses an OpenAI Agents SDK v0.22.0 harness, an Anthropic SDK v1.4.0 client, or a LangGraph 1.x orchestrator, and they all share one thing: an MCP client. Each agent connects to N MCP servers (filesystem, GitHub, Slack, Postgres, internal APIs) over stdio, HTTP, or SSE.²

One agent talking to a handful of MCP servers works fine. Twenty agents talking to forty servers, sharing credentials, fighting over rate limits, and getting audited, does not.

Why MCP needs a gateway¶

The same forces that made Envoy, nginx, and Kong inevitable for HTTP are repeating for MCP. Five problems show up the moment MCP leaves the laptop:

None of this is novel. We solved it for HTTP, gRPC, and GraphQL. The fact that the wire format is JSON-RPC over stdio doesn't change the operational problems.

The two open-source gateways worth taking seriously in 2026¶

Docker MCP Gateway - v0.43.3, 2026-07-16¶

Docker's MCP Gateway is built into the Docker Desktop MCP Toolkit (Docker Desktop 4.59+). It runs MCP servers as Docker containers, manages their lifecycle, and exposes a single Gateway URL to every MCP client.⁴

# Configure a server from the catalog
docker mcp server configure --server github --env GITHUB_TOKEN_FILE=/run/secrets/gh

# Run the gateway
docker mcp gateway run --port 8811 --secrets-from docker-desktop

# Every MCP client connects to http://localhost:8811

What it does well:

• Container isolation for each MCP server.
• Secrets management through Docker Desktop (no tokens in env vars).
• OAuth flows built in for MCPs that need them.
• Dynamic tool discovery — agents see the union of capabilities across all configured servers.
• Catalog model — install from the Docker MCP Catalog like installing packages.

What it doesn't solve well:

• Local-first: production deployment requires lifting the pattern into a cluster.
• No native multi-tenant RBAC; assumes one user, one machine.
• Lacks a policy-as-code surface (OPA, Cedar, etc.).

Agentgateway - v1.5.0, 2026-08-27, Apache-2.0¶

Agentgateway is a CNCF-friendly proxy purpose-built for agentic protocols: MCP and Google's A2A. It's framework-agnostic and explicitly positioned as the missing connectivity layer for AI agents.⁵

The README states its purpose directly:

Agentgateway is an open source proxy built on AI-native protocols (MCP and A2A) that provides drop-in security, observability, and governance for agent-to-LLM, agent-to-tool, and agent-to-agent communication across any framework and environment.

It's effectively "Envoy for AI agents" — same control-plane-as-data philosophy (Gateway API CRDs, kgateway-style), same familiar observability story, but the resource model understands tools and resources, not HTTP paths.

The rough decision rule: Docker MCP Gateway for local development and single-team production. Agentgateway when you have multiple agent workloads, multiple tenants, and a Gateway API-first operations culture.

A reference architecture¶

The pattern that holds up across most production teams in 2026:

                    ┌──────────────────────┐
                    │  Agent Runtimes      │
                    │  - OpenAI Agents SDK │
                    │  - Anthropic SDK     │
                    │  - LangGraph         │
                    └──────────┬───────────┘
                               │ MCP-over-HTTP/SSE
                               ▼
                    ┌──────────────────────┐
                    │   MCP Gateway        │
                    │  ┌────────────────┐  │
                    │  │ Authn / Authz  │  │
                    │  │ Rate limit     │  │
                    │  │ Tool registry  │  │
                    │  │ Policy engine  │  │
                    │  │ OTLP exporter  │  │
                    │  └────────────────┘  │
                    └──────────┬───────────┘
                               │
        ┌──────────────┬───────┴────────┬──────────────┐
        ▼              ▼                ▼              ▼
   ┌────────┐    ┌────────┐       ┌────────┐      ┌────────┐
   │ GitHub │    │ Slack  │       │ Postgres│     │ Custom │
   │ MCP    │    │ MCP    │       │ MCP     │     │ MCP    │
   └────────┘    └────────┘       └────────┘      └────────┘

The gateway is the only thing every agent talks to. Every MCP server is reachable only through the gateway. That gives you one place to enforce policy, one place to collect telemetry, and one place to swap or version a server.

What to actually enforce at the gateway¶

Five things are worth the engineering in 2026. These are the policies that catch real production incidents, not theoretical ones.

1. Authentication and authorization per tool¶

The MCP wire format exposes three primitives: tools, resources, and prompts. Each should be a first-class subject in your policy. A practical spec:

# gateway-policies.yaml
policies:
  - name: hr-tools-rbac
    match: { tool: "hr.*" }
    allow: { teams: ["people-ops", "engineering-managers"] }
    deny: { teams: ["contractors"] }
    audit: true

  - name: github-write-rate-limit
    match: { tool: "github.create_issue", tool: "github.merge_pr" }
    rate_limit: { rps: 5, burst: 10 }
    require_approval: true

  - name: postgres-read-only
    match: { tool: "postgres.*" }
    allow_methods: ["SELECT", "EXPLAIN"]
    deny_methods: ["INSERT", "UPDATE", "DELETE", "DROP"]

The Anthropic security guidance for MCP is explicit: implementations MUST enforce consent and authorization flows, access controls, and data protection at the protocol boundary.³ That boundary is your gateway.

2. Per-tenant rate limits and cost ceilings¶

A model in a tight loop can call a single tool 10,000 times in a session. The cost is not the LLM tokens; it's the API cost of the tool. A 1,000-RPM run on a third-party API is a $5,000 bill in a day. The gateway is the only place to cap that.

# OpenTelemetry counter for cost per tool
from opentelemetry import metrics
meter = metrics.get_meter("mcp-gateway")
cost = meter.create_counter("mcp.tool.cost_usd")

@tool.invocation
async def track_and_call(req, call):
    est_cost = estimate_cost(req.tool, req.arguments)
    cost.add(est_cost, {"tool": req.tool, "tenant": req.tenant})
    if cost_above_quota(req.tenant, req.tool):
        raise QuotaExceeded()
    return await call(req)

3. Tool result filtering¶

MCP servers return whatever the upstream API returns. That's fine for GET /users/me. It's a problem when the tool is "look up customer record" and the agent is from a different team.

Filter at the gateway, not at the model. Strip PII fields, redact secrets, drop rows that don't match the requesting tenant's policy scope.

4. Audit logging¶

Two questions always come up:

• "Why did the agent do X yesterday at 14:23?"
• "What data did this agent's tools see?"

A gateway-level audit log answers both with one query. Every tool call gets:

• Caller identity (which agent, which run ID, which user the agent is acting on behalf of).
• Tool name and version.
• Arguments (with secret fields redacted at ingestion).
• Result size, status, latency.
• Trace correlation ID for the parent OTel span.

Push to your existing observability backend. Don't build a new tool.

5. Prompt-injection defense at the boundary¶

The MCP spec warns explicitly that protocol-level guarantees are not enough:

While MCP itself cannot enforce these security principles at the protocol level, implementors SHOULD build robust consent and authorization flows, access controls, and data protections into their applications.

MCP tool results come back as model context. Anything an upstream tool returned is now part of the prompt. The gateway should treat tool results as untrusted input and apply the same redaction and validation policies that you'd apply to web-fetched text in any other LLM pipeline.

A practical pattern:

def guard_tool_result(tool_name, result):
    # Strip anything that looks like an injected instruction
    sanitized = redact_injection_attempts(result)
    # Cap result size to prevent context stuffing
    return truncate(sanitized, max_tokens=2000)

Self-hosted vs hosted: the deployment decision¶

Three deployment shapes are realistic in 2026:

The central gateway is the only one that scales. The first time you have an auth change, you'll understand why.

A reasonable production deployment on Kubernetes:

# gateway-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata: { name: mcp-gateway }
spec:
  replicas: 3
  selector:
    matchLabels: { app: mcp-gateway }
  template:
    metadata:
      labels: { app: mcp-gateway }
      annotations:
        prometheus.io/scrape: "true"
    spec:
      containers:
        - name: gateway
          image: ghcr.io/agentgateway/agentgateway:v1.5.0
          args: ["run", "--config", "/etc/gateway/config.yaml"]
          ports:
            - { containerPort: 8811 }
          volumeMounts:
            - { name: cfg, mountPath: /etc/gateway }
          env:
            - name: OTEL_EXPORTER_OTLP_ENDPOINT
              value: "http://otel-collector.observability:4317"
            - name: OTEL_SERVICE_NAME
              value: "mcp-gateway"
          resources:
            requests: { cpu: "1", memory: 2Gi }
            limits: { cpu: "4", memory: 4Gi }
        - name: oauth-proxy
          image: ghcr.io/oauth2-proxy/oauth2-proxy:v7.6.0
          # OIDC validation for agent identity
      volumes:
        - name: cfg
          configMap: { name: gateway-config }

Three observations:

• Three replicas for HA behind a service. The gateway is now in your critical path; treat it like one.
• OTLP exporter as a first-class env. The whole point is unified observability; wire it from day one.
• OAuth proxy sidecar for agent identity. Agents present a service-account JWT; the proxy validates against your IdP.

A practical first pass¶

If you're starting today:

• Inventory MCP usage. Ask every team: which MCP servers, which agents, which credentials. Most shops discover 3-4x more usage than they knew about.
• Pick one gateway. Docker MCP Gateway for dev/early prod; agentgateway when you have ≥3 agent workloads or ≥2 tenants.
• Front the gateway with an identity-aware proxy. OIDC or mTLS. Per-agent identity, not per-pod.
• Wire OTLP from day one. Three signals: tool call rate, tool call latency, tool call result size. Alert on the first two; chart the third.
• Define a default-deny policy. New MCP servers and tools require explicit approval before they're reachable through the gateway.
• Roll one tenant first. Pilot one team's MCP traffic through the gateway before flipping everyone over.
• Audit-log everything. Pipe to your existing SIEM or observability backend; don't invent a new one.

What NOT to put behind the gateway¶

Three categories don't belong inside an MCP gateway:

1. Anything latency-critical in single-digit milliseconds. MCP gateways add tens of milliseconds of overhead. If your trading strategy needs sub-10ms tool calls, run them on a direct path, not through the gateway.
2. Authenticated browser session state. MCP is for backend agent-to-tool. Don't try to retrofit it for browser automation; use the appropriate browser-control primitives.
3. One-off internal scripts. If only one agent talks to one MCP server and you're not going to scale it, the gateway is overhead. Add it when the second agent shows up.

Summary¶

• MCP won the wire format. The deployment story is the open problem.
• A central MCP gateway is the only way to enforce auth, rate limits, audit, and policy across many agents and many servers.
• Docker MCP Gateway v0.43.3 fits local + single-team prod. Agentgateway v1.5.0 fits multi-tenant, Kubernetes-native prod.
• Wire OTLP from day one. Don't invent a new observability backend for this.
• Treat tool results as untrusted input at the gateway. The MCP spec is explicit that protocol-level guarantees are not enough.
• Start with inventory, not architecture. Most teams discover they have 3-4x more MCP usage than they know about.
• The gateway is in the critical path. HA from day one; identity-aware proxy from day one.

MCP without a gateway is HTTP without nginx. It works until it doesn't, and by then you've lost control of who calls what.

Questions or discussion? Connect on LinkedIn, X, or email.

Discussion

Have thoughts on this post? Share them below — questions, corrections, or your own experience are all welcome.