Skip to content

September 2026

Scaling LLM Inference: DP, PP, and TP with vLLM

You sized the model and it doesn't fit on one GPU — or it fits but can't keep up with traffic. Both problems are solved by spreading the model across cards, but with different knobs. Pick the wrong one and you'll waste GPUs, melt your latency, or both. This post is the decision tree.

Context engineering for production AI agents

Prompt engineering is the discipline of writing the right words for a single turn. Context engineering is the discipline of curating everything that ends up in the model's context window across an entire agent run — system prompt, tool definitions, retrieved documents, conversation history, scratchpad notes, sub-agent outputs. Anthropic named this discipline in September 20251 because production agents kept hitting the same wall: the model was smart enough, but the context it saw was wrong.

This post covers what context engineering is, the four strategies you actually use, the failure modes that show up in production, and the tooling that does each one well.

GPUs on Kubernetes: From Bare Metal to Schedulable in One Operator

A fresh Kubernetes cluster has no idea your nodes have GPUs. kubectl describe node shows CPU, memory, and pods — nothing else. To make a pod request a GPU you need a driver, a container runtime hook, and a device plugin advertising the hardware to the scheduler, all version-matched across every GPU node. Do it by hand and you'll re-do it on every kernel bump. This post wires it up the way you actually want — one operator — and works on any Kubernetes, not a specific vendor's distro.

Stacking MIG and Time-Slicing on One GPU Operator values.yaml

MIG carves a GPU into hardware-isolated slices. Time-slicing oversubscribes each slice so more pods can share it. Stack them and one physical GPU advertises far more schedulable units than it has silicon — useful when you have more workloads than GPUs and most of them sit idle. Here's the exact values.yaml, wired into the kommander-applications GPU Operator 26.3.0 app1, and the labels that switch a node between layouts.

Multi-agent engineering with Claude Code: the dev loop, demonstrated

The previous post walked through a single Claude Code session — one engineer, one context window, 14 turns. That's the right shape for most work. But some tasks don't fit in one brain. A migration across 12 services, a security audit of 200 endpoints, a refactor that touches every file in the repo — these want a small team, not one engineer working 14-hour days.

Claude Code has a Task tool that spawns sub-agents with isolated context windows. The main session is the lead engineer; sub-agents are specialists. Each specialist runs to completion, returns a summary, and the lead stitches the work together. That's the multi-agent loop, in Claude Code.

This post is a full session that uses it. I'll do a real task — port a Python CLI from one config library to another across 8 files — by having Claude Code delegate parts to sub-agents. You'll see the prompts, the sub-agent invocations, where handoffs went well, and where they didn't.

The session is reconstructed from real Claude Code runs. The patterns work; the exact transcripts are illustrative.

Build Your Own Token-as-a-Service: A Self-Hosted OpenAI-Compatible AI Gateway on Kubernetes

Public LLM APIs bill you per token and leak your prompts off-prem. The fix isn't "run Ollama on a box" — it's a centralized, OpenAI-compatible inference platform your whole org consumes like a utility: one API key, token-based quotas, models that stay in your datacenter. Call it token-as-a-service. This post is the architecture — model tiering, GPU math, MIG partitioning, vLLM tuning, and KV-cache-aware routing — generic enough to run on any Kubernetes cluster with NVIDIA GPUs.

The full Claude Code dev loop, demonstrated end-to-end

Most posts about Claude Code show a single claude command and a single impressive output. That's not how you actually use it. A real session is 30+ turns: plan, write, run, fail, read the error, fix, test, commit, repeat. The interesting parts are the failures and the recovery.

This post is one full session, reconstructed from real Claude Code sessions. Every command is one you'd actually run, every code block is code you'd actually see, every bug is the kind of bug that does appear. The wall time and the exact bug-discovery sequence are approximate — the patterns are real.

The session took 14 minutes of wall time. The transcript below is annotated.

GPU DRA on Kubernetes: migrating off the NVIDIA device plugin, end-to-end

Dynamic Resource Allocation went GA in Kubernetes 1.351, and NVIDIA's DRA driver for GPUs shipped v0.4.1 on 2026-06-302. The nvidia.com/gpu extended resource model — the one the device plugin has used since 2017 — now has a sanctioned successor. The migration is non-trivial because the resource model is completely different (claim-based, not extended-resource-based), but the payoff is real: structured GPU allocation, native NVLink topology awareness, and a single API for MIG, MPS, time-slicing, and Multi-Node NVLink.

This post is the migration path: install the driver, write the first claim, swap one workload at a time, and run device plugin + DRA side-by-side until you're done.

Cilium and eBPF: the Networking Layer Under Your Cluster

Your cluster routes every packet through iptables, and at a few thousand Services that chain becomes a linear scan the kernel walks per connection. Cilium throws that out and runs networking, policy, and observability as eBPF programs in the kernel instead — no sidecars, no iptables, no agent in the data path. This post walks the whole stack: how a packet even finds another pod, installing Cilium as a kube-proxy replacement, watching real traffic with Hubble, locking the cluster down with identity-based policy, stretching it across clusters with Cluster Mesh, and dropping the service-mesh sidecars — all copy-paste.

Centralize Log Solution with the Elastic Stack

Use case: ship Kubernetes ingress logs off the cluster into a self-managed Elastic Stack on Ubuntu servers, parse them, and keep them searchable for 90 days without blowing the disk budget. The Elastic Stack handles this cleanly, and the 2026 build leans on four current features that older tutorials skip: LogsDB, ILM, data streams, and a Kubernetes-native shipper.