You sized the model and it doesn't fit on one GPU — or it fits but can't keep up with traffic. Both problems are solved by spreading the model across cards, but with different knobs. Pick the wrong one and you'll waste GPUs, melt your latency, or both. This post is the decision tree.
Prompt engineering is the discipline of writing the right words for a single turn. Context engineering is the discipline of curating everything that ends up in the model's context window across an entire agent run — system prompt, tool definitions, retrieved documents, conversation history, scratchpad notes, sub-agent outputs. Anthropic named this discipline in September 20251 because production agents kept hitting the same wall: the model was smart enough, but the context it saw was wrong.
This post covers what context engineering is, the four strategies you actually use, the failure modes that show up in production, and the tooling that does each one well.
Public LLM APIs bill you per token and leak your prompts off-prem. The fix isn't "run Ollama on a box" — it's a centralized, OpenAI-compatible inference platform your whole org consumes like a utility: one API key, token-based quotas, models that stay in your datacenter. Call it token-as-a-service. This post is the architecture — model tiering, GPU math, MIG partitioning, vLLM tuning, and KV-cache-aware routing — generic enough to run on any Kubernetes cluster with NVIDIA GPUs.
Dynamic Resource Allocation went GA in Kubernetes 1.351, and NVIDIA's DRA driver for GPUs shipped v0.4.1 on 2026-06-302. The nvidia.com/gpu extended resource model — the one the device plugin has used since 2017 — now has a sanctioned successor. The migration is non-trivial because the resource model is completely different (claim-based, not extended-resource-based), but the payoff is real: structured GPU allocation, native NVLink topology awareness, and a single API for MIG, MPS, time-slicing, and Multi-Node NVLink.
This post is the migration path: install the driver, write the first claim, swap one workload at a time, and run device plugin + DRA side-by-side until you're done.
Your cluster routes every packet through iptables, and at a few thousand Services that chain becomes a linear scan the kernel walks per connection. Cilium throws that out and runs networking, policy, and observability as eBPF programs in the kernel instead — no sidecars, no iptables, no agent in the data path. This post walks the whole stack: how a packet even finds another pod, installing Cilium as a kube-proxy replacement, watching real traffic with Hubble, locking the cluster down with identity-based policy, stretching it across clusters with Cluster Mesh, and dropping the service-mesh sidecars — all copy-paste.
Use case: ship Kubernetes ingress logs off the cluster into a self-managed Elastic Stack on Ubuntu servers, parse them, and keep them searchable for 90 days without blowing the disk budget. The Elastic Stack handles this cleanly, and the 2026 build leans on four current features that older tutorials skip: LogsDB, ILM, data streams, and a Kubernetes-native shipper.
Run Kong so that a database outage can't take your APIs down, every config change goes through a merge request, and one gateway serves both public and internal traffic without mixing them. That's hybrid mode plus GitOps on Kubernetes. This post builds it on NKP (Nutanix Kubernetes Platform), with a companion project you can run on a laptop first.
"Will this model fit on my GPU?" has one honest answer: do the arithmetic. It's three numbers added together, and you can pull every input from a model's config.json in about two minutes. This post turns a sizing spreadsheet into a method you can run by hand.
You can build a production RAG platform on your own hardware for the price of two years of API bills, and it fits in half a rack. The trick is sizing the GPUs to the models instead of buying the biggest card on the truck. This is the full design for a 100-user, highly available, hybrid-plus-graph RAG stack, with every GPU choice backed by the VRAM math.
LLM workloads are not "regular workloads that happen to need more RAM." They are memory-bandwidth-bound during decode, compute-bound during prefill, and topology-bound the moment a model can't fit on one GPU. The hardware spec sheet, the VM config, the container runtime, the Kubernetes device plugin, and the multi-GPU pattern you pick are all the same decision at different layers. Get one wrong and the others stop mattering.
This is a reference for the whole stack — from the HBM3e bandwidth of a Blackwell GPU to the nvidia.com/gpu resource advertised by a DaemonSet to the tensor-parallel group size that decides whether your 70B model serves or stalls. Every number is verified against a primary source, dated 2026-06-16.