AI¶

MCP Servers in Production: Patterns for Connecting AI Agents to Kubernetes and Cloud APIs

The first wave of AI agents talked to infrastructure through bespoke wrappers: a Python script that called kubectl, a Lambda that polled CloudWatch, a chat plugin that hard-coded an OpenAPI spec. That worked for demos. It collapsed the moment an agent needed to talk to two systems at once, or get rotated credentials, or appear in a SOC 2 audit.

Model Context Protocol (MCP) is the answer that won, and by 2026 it has the ecosystem to prove it. The Python SDK alone shipped v2.2.0 on 2026-09-07.¹ The spec was last revised 2025-06-18 and is now stable enough that AWS Labs, Microsoft, Google, Cloudflare, and Alibaba all ship vendor MCP servers against it.² OWASP published an MCP-specific Top 10 in 2025 because the attack surface is real enough to warrant one.³

This post is the field guide I wish someone had handed me twelve months ago. It covers the transport choices, the Kubernetes deployment patterns, the cloud-vendor landscape, authentication, security, multi-server gateways, observability, and the operational checklist that turns a working prototype into a production control plane.

Self-Host an AI Gateway on Kubernetes: Routing, Caching, and Cost Control with Envoy

One LLM provider is a weekend project. Five providers, two self-hosted inference clusters, three regions, and a per-team budget is a platform. The path between them is an AI gateway — and Envoy is the only data plane with a first-class AI filter shipped upstream. Run it on Kubernetes the way you already run any other gateway, and you get a single control plane for every model call in the company.

Disaggregated LLM Inference on Kubernetes with LLM-D: Prefill/Decode Split, Expert Parallelism, and Real Autoscaling

LLM inference has two phases with different bottlenecks. Prefill is FLOPs-bound, decode is memory bandwidth-bound. Most production stacks shuffle both through the same GPU pool, which costs you time-to-first-token and tethers decode latency to whoever next requests a long prompt. LLM-D — the CNCF Sandbox project co-founded by Red Hat, Google Cloud, IBM Research, CoreWeave, and NVIDIA — splits those phases onto separate pods, schedules expert parallelism for MoE models, and replaces CPU/GPU autoscaling with EPP-derived signals that actually predict load.

Graph Engineering: From Karpathy's Loops to Shared Knowledge Graphs

A short PDF cross-references three pieces of 2026 work — Karpathy's autoresearch loop, his AgentHub sketch, and Anthropic's Knowledge Graph Construction Cookbook — and walks through the missing layer that turns single-agent loops into collaborative swarms. It's an independent synthesis, not a Karpathy paper, but the architectural progression is precise and worth taking seriously. The pipeline has six steps, and the step that matters most is the one your loop is currently skipping.

Running Hermes Agent in Production Behind Traefik

I deployed Hermes Agent to my Hostinger VPS with the one-click Docker Compose template. The build log said Project deployed successfully. It lied — the container came up, but its HTTPS was broken and its admin dashboard was published to the public internet over plain HTTP.

This is the walkthrough of what the template actually gives you, the three defects I found on my own box, and the compose file I run now.

LLM Observability with OpenTelemetry: Instrumenting AI Workloads the Cloud-Native Way

If you already run OpenTelemetry for your services, you don't need a new telemetry stack for LLMs. You need the same tracer, the same collector, the same Grafana dashboard — pointed at a new model of work. The recent GenAI semantic conventions let spans, metrics, and logs describe what an LLM actually did: which model, which prompt, how many tokens, how long, what it cost, and which tool it called.

Production AI Agents Need an MCP Gateway

Two MCP servers in a developer notebook is fine. Forty MCP servers across production agents, with auth tokens, audit trails, and cost ceilings, is not. The Model Context Protocol won the wire format war; the deployment story is still open. Treating MCP like every other L7 protocol and putting a gateway in front is the only path that survives the second quarter of production.

The Quant Developer Stack in 2026: Self-hosted Market Data, GPU Backtests with Nix and Slurm, and the Path from Notebook to Live Signal

A quant team has two completely different problems to solve. One is research: lots of analysts, lots of notebooks, lots of Parquet, slow human-in-the-loop iteration. The other is production: a small trading desk, kernel-bypass networking, microsecond budgets, almost no humans. The mistake is trying to run both on the same infrastructure. The right answer is two stacks, one boundary.

From YAML to AI: How Platform Engineers Are Embedding LLMs into Internal Developer Platforms

Platform engineering spent years turning tribal knowledge into YAML: catalog descriptors, scaffolder templates, scorecards, policies, and deployment workflows. Now LLMs are becoming the conversational layer over that machinery. The useful shift isn't replacing YAML with chat. It's giving developers a safer way to discover and invoke the platform capabilities already encoded underneath.

Scaling LLM Inference: DP, PP, and TP with vLLM

You sized the model and it doesn't fit on one GPU — or it fits but can't keep up with traffic. Both problems are solved by spreading the model across cards, but with different knobs. Pick the wrong one and you'll waste GPUs, melt your latency, or both. This post is the decision tree.