Stacking MIG and Time-Slicing on One GPU Operator values.yaml¶

MIG carves a GPU into hardware-isolated slices. Time-slicing oversubscribes each slice so more pods can share it. Stack them and one physical GPU advertises far more schedulable units than it has silicon — useful when you have more workloads than GPUs and most of them sit idle. Here's the exact values.yaml, wired into the kommander-applications GPU Operator 26.3.0 app¹, and the labels that switch a node between layouts.

Assumes the GPU Operator is already installed. For the from-scratch enablement — driver, toolkit, device plugin, GFD — start with GPUs on Kubernetes: from bare metal to schedulable. For when to choose MIG vs time-slicing vs MPS in the first place, see the sharing comparison. This post is the deep recipe for one combo: MIG + time-slicing stacked. The platform that consumes these slices is in Self-hosted token-as-a-service.

Where these values plug in¶

The kommander-applications repo ships the GPU Operator as a Flux HelmRelease, not a raw helm install. The chain looks like this¹:

AppDeployment (workspace)
   └─ HelmRelease  nvidia-gpu-operator
        ├─ chartRef → OCIRepository  oci://ghcr.io/mesosphere/charts/gpu-operator : v26.3.0
        └─ valuesFrom:
             1. ConfigMap  nvidia-gpu-operator-26.3.0-config-defaults   ← shipped defaults
             2. ConfigMap  nvidia-gpu-operator-overrides                ← YOUR values (wins)

valuesFrom entries merge in order, so your override ConfigMap is layered last and wins. You don't fork the app or edit the defaults — you drop a values.yaml into an override ConfigMap and point the AppDeployment at it. Everything below is the content of that override.

The full values.yaml¶

This is the values.yaml you'll embed in the override ConfigMap. It sets the MIG strategy to mixed, defines a custom time-slicing config, and ships three named MIG layouts the mig-manager can apply on demand.

devicePlugin:
  config:
    name: custom-time-slicing-config
    create: true
    data:
      no-sharing: |-
        version: v1
        flags:
          migStrategy: mixed
      nai-timeslice: |-
        version: v1
        flags:
          migStrategy: mixed
        sharing:
          timeSlicing:
            resources:
            - name: nvidia.com/mig-1g.24gb
              replicas: 3
            - name: nvidia.com/mig-2g.48gb
              replicas: 3

driver:
  enabled: true

mig:
  strategy: mixed

migManager:
  enabled: true
  env:
    - name: WITH_REBOOT
      value: "true"
  config:
    name: custom-mig-config
    create: true
    data:
      config.yaml: |-
        version: v1
        mig-configs:
          all-disabled:
            - devices: all
              mig-enabled: false
          mig-4x-per-card:
            - devices: all
              mig-enabled: true
              mig-devices:
                "1g.24gb": 4
          mig-2x-per-card:
            - devices: all
              mig-enabled: true
              mig-devices:
                "2g.48gb": 2
          mig-4x-2x-combine:
            - devices: [0]
              mig-enabled: true
              mig-devices:
                "1g.24gb": 4
            - devices: [1]
              mig-enabled: true
              mig-devices:
                "2g.48gb": 2

What each block does¶

Time-slicing replicas: 3 doesn't add memory or compute. It tells the scheduler one mig-1g.24gb slice can be claimed by 3 pods, which then share that slice's time. Great for bursty or idle-heavy workloads, bad for anything that wants the whole slice.

Apply it¶

0. Wire the values into the app¶

Wrap the values.yaml above in an override ConfigMap (key must be values.yaml) in your workspace namespace, then reference it from the AppDeployment's configOverrides²:

apiVersion: v1
kind: ConfigMap
metadata:
  name: nvidia-gpu-operator-overrides
  namespace: <workspace-namespace>
data:
  values.yaml: |
    # ← paste the full values.yaml from above here, indented
    mig:
      strategy: mixed
    devicePlugin:
      config:
        name: custom-time-slicing-config
        create: true
        # ...rest of the values
---
apiVersion: apps.kommander.d2iq.io/v1alpha3
kind: AppDeployment
metadata:
  name: nvidia-gpu-operator
  namespace: <workspace-namespace>
spec:
  appRef:
    name: nvidia-gpu-operator-26.3.0
    kind: ClusterApp
  configOverrides:
    name: nvidia-gpu-operator-overrides

Apply both, then let Flux reconcile. The AppDeployment controller appends your ConfigMap to the HelmRelease valuesFrom after the shipped defaults, so your settings win.

kubectl apply -f nvidia-gpu-operator-overrides.yaml

# Force a reconcile instead of waiting for the interval
kubectl annotate helmrelease nvidia-gpu-operator -n <workspace-namespace> \
  reconcile.fluxcd.io/requestedAt="$(date +%s)" --overwrite

1. Set the physical MIG layout¶

Pick one layout per node and label it. The mig-manager watches this label and reconfigures the hardware.

# Four 1g.24gb slices across all GPUs
kubectl label node <node-name> nvidia.com/mig.config=mig-4x-per-card --overwrite

# Two 2g.48gb slices across all GPUs
kubectl label node <node-name> nvidia.com/mig.config=mig-2x-per-card --overwrite

# Combined: four 1g.24gb on GPU 0, two 2g.48gb on GPU 1
kubectl label node <node-name> nvidia.com/mig.config=mig-4x-2x-combine --overwrite

2. Turn on time-slicing¶

This points the device plugin at the nai-timeslice profile, multiplying each slice into its replicas.

kubectl label node <node-name> nvidia.com/device-plugin.config=nai-timeslice --overwrite

3. Verify¶

# Watch the reconfigure reach "success"
kubectl get node <node-name> -L nvidia.com/mig.config.state

# Confirm the advertised MIG resources
kubectl describe node <node-name> | grep -i nvidia.com/mig

Trace it through GitOps (Flux)¶

When the values don't seem to take, walk the chain AppDeployment → HelmRelease → ConfigMaps → pods and confirm each link rendered before you blame the labels.

# AppDeployment present and your override ConfigMap exists?
kubectl get appdeployment nvidia-gpu-operator -n <workspace-namespace>
kubectl get cm nvidia-gpu-operator-overrides -n <workspace-namespace>

# HelmRelease reconciled? Check it actually references BOTH ConfigMaps in valuesFrom
kubectl get helmrelease nvidia-gpu-operator -A
kubectl get helmrelease nvidia-gpu-operator -n <namespace> \
  -o jsonpath='{.spec.valuesFrom}'

# Chart pulled from the OCI registry?
kubectl get ocirepository -A | grep gpu-operator

# Operator pods up?
kubectl get pods -A | grep -i nvidia

# mig-manager logs — where reconfigure failures show up
kubectl logs -l app=nvidia-mig-manager -n <namespace-found-above>

If valuesFrom lists only ...-config-defaults and not your -overrides ConfigMap, the AppDeployment didn't pick up configOverrides — fix that before touching node labels.

Summary¶

• Put your values.yaml in an override ConfigMap (key values.yaml) and reference it from the AppDeployment's configOverrides — don't fork the app.
• Set mig.strategy: mixed so one node can expose multiple MIG profiles.
• Define MIG layouts under migManager.config and label the node with nvidia.com/mig.config=<layout>.
• Define a time-slicing profile under devicePlugin.config and label the node with nvidia.com/device-plugin.config=nai-timeslice.
• Keep WITH_REBOOT: "true" only if the node can tolerate a reboot on MIG reconfigure.
• Verify with nvidia.com/mig.config.state and kubectl describe node | grep nvidia.com/mig.
• If the values don't apply, check the HelmRelease valuesFrom lists your -overrides ConfigMap after -config-defaults.

The mental model: MIG splits the silicon, time-slicing oversubscribes the splits. One physical GPU, two layers of multiplication — 2g.48gb → mig-2x-per-card (2 slices) → replicas: 3 = 6 schedulable units the cluster can hand out.

Discussion

Have thoughts on this post? Share them below — questions, corrections, or your own experience are all welcome.