Kubernetes on Metal or a Single VPS: Request/Limit Math for Small Game Backends

Kubernetes solves a problem most small game backends do not have: running many services across many machines with independent scaling and rolling deploys. If you run one session service, one matchmaker, and one database for a few hundred to a few thousand concurrent players, the fixed overhead of a control plane — etcd, API server, scheduler, controller manager, kubelet, CNI, ingress — is often larger than the workload it manages. This article does the arithmetic: what HPA and KEDA actually cost, how requests and limits interact, and at what player concurrency orchestrating becomes the cheaper option.

What the Control Plane Costs Before You Run Anything

A k3s or kubeadm single-node install idles at a surprisingly specific footprint:

ComponentIdle CPU (cores)Idle RAMDisk
etcd (single node)0.05–0.15150–300 MB1–4 GB, fsync-sensitive
kube-apiserver0.05–0.20250–500 MB—
controller-manager + scheduler0.05–0.10150–300 MB—
kubelet + containerd0.05–0.15200–400 MB—
CNI + kube-proxy/IPVS0.02–0.1050–150 MB—
ingress controller0.05–0.30150–400 MB—
metrics-server (for HPA)0.02–0.1050–120 MB—
Total idle0.3–1.11.0–2.2 GB~5 GB
Single-node k3s with Flannel and ingress; a three-node control plane triples the etcd and API server figures and adds inter-node traffic.

KEDA adds an operator, a metrics adapter, and a scaler pod per ScaledObject class: budget another 0.15–0.3 cores and 300–600 MB. On a 4-core/16 GB box the total is roughly 25% of CPU and 15% of RAM before a single player connects; on 8 cores it is 10–14% of CPU. That tax is paid continuously, whether or not anyone is playing.

Requests, Limits, and the Ratios That Matter

  • Requests are what the scheduler reserves. They do not cap usage. They determine bin-packing and the HPA’s utilisation denominator.
  • Limits are enforced at runtime: CPU gets throttled via cfs_quota, memory gets the container OOM-killed.

The asymmetry is the crux: a CPU limit throttles, a memory limit kills. Under-provision memory by 10% and you get a restart loop; under-provision CPU by 10% and you get tail latency. Be generous with memory limits relative to measured peak, and relatively tight with CPU requests.

Ratio (limit ÷ request)EffectUse when
1.0 (Guaranteed)No throttling headroom; predictableLatency-critical game session pods
1.2–1.5Small burst absorbed, mild throttling possibleMost steady-state services
2.0–3.0Significant burst capacity, throttling under loadBatch workers, cold-start-tolerant jobs
> 3.0 (Burstable-heavy)Node oversubscribed; noisy neighboursRarely justified for game backends

For a game session service, start with CPU request 0.5 and limit 0.75 per pod, memory request 512 Mi and limit 1 Gi. Measure the p99 CPU throttling ratio (container_cpu_cfs_throttled_periods_total / container_cpu_cfs_periods_total): anything above 5% under normal load means the limit is too low or the request is understated and the HPA denominator is wrong.

The HPA Denominator Problem

HPA computes utilisation as current usage ÷ requested. That means the replica count it produces depends entirely on the request value, not on what the pod actually needs. Set a request of 0.2 cores for a pod that reliably consumes 0.6 and HPA sees 300% utilisation and over-scales. Set it at 1.0 for the same pod and HPA sees 60% and never scales, even while the pod is thrashing.

HPA’s default scaleUp behaviour is also slower than a game backend wants: it checks every 15 s and then waits for new pods to become ready — 2–5 seconds for a Go session service, 20–40 seconds for a JVM one including the startup probe. A 30-second spike is largely over by the time the extra replicas are serving.

Where KEDA Genuinely Helps

HPA scales on resource utilisation. KEDA scales on external metrics — a queue length, a Kafka consumer lag, a Redis list depth, or a Prometheus query. For game backends, the honest cases where KEDA earns its overhead:

  • Queue-driven matchmaking workers — scale on queue depth rather than CPU, because the workers have known per-job cost.
  • Post-session analytics — bursts that must drain quickly, scale to zero between bursts. Scale-to-zero is KEDA’s real advantage; HPA cannot go below one replica.
  • Scheduled warm-up — KEDA’s cron scaler can pre-scale for a known event time, so pods are ready before the spike rather than 30 seconds into it.

What KEDA does not fix: a bottleneck that is a shared database, a single-threaded game loop, or a connection pool. Scaling pods past the pool’s capacity produces contention, not throughput.

Node Allocatable Math

The 4 cores on your box are not 4 cores of capacity. Reserve system, kubelet, and eviction headroom:

  • kube-reserved + system-reserved: typically 0.15 cores and 400 Mi on a 4-core node, scaling up with size.
  • evictionHard memory: 100–200 Mi held back.
  • DaemonSets (CNI, kube-proxy, node exporter): 0.1–0.2 cores and 200–400 Mi.
  • Unallocated slack you should keep: 10–15% of remaining capacity, to avoid a node at 100% request where nothing can be scheduled or evicted cleanly.
NodeRawAllocatableAfter 12% slackUsable for app pods
4 vCPU / 8 GB4 / 8192 Mi3.85 / 7.4 Gi3.39 / 6.5 Gi~3.0 / 5.5 Gi
8 vCPU / 32 GB8 / 32768 Mi7.7 / 31.4 Gi6.78 / 27.6 Gi~6.3 / 26 Gi
16 vCPU / 64 GB16 / 65536 Mi15.6 / 63.2 Gi13.7 / 55.6 Gi~13 / 53 Gi

On the 4 vCPU node, after the control plane’s 0.3–1.1 cores, you have roughly 2 cores for application pods. That is two to four session-service replicas at the 0.5–0.75 request above. The box that looked like a 4-core machine is functionally a 2-core machine with a management layer.

Memory Limits and the OOMKill Risk

Container memory accounting includes page cache that the container touched, which is why a pod can be killed long before its heap looks full. Two mitigations that matter:

  • Set the heap ceiling below the limit. For a JVM, -XX:MaxRAMPercentage=70 with a 1 Gi limit leaves 300 MB for metaspace, thread stacks, and off-heap buffers. For Go, set GOGC and GOMEMLIMIT to ~80% of the limit so the runtime collects before the kernel intervenes.
  • Never set a memory limit equal to the request unless you also set the request correctly. Guaranteed QoS is good for latency but leaves no flexibility; Burstable with request 70% and limit 100% is usually the right shape for a session service.

Load-test to the limit and watch container_memory_working_set_bytes, not container_memory_usage_bytes — the former excludes reclaimable cache and is what the eviction decision uses. If working set at p99 peak is more than 85% of the limit, raise the limit.

Cost Comparison: Three Deployments of the Same Backend

DeploymentMonthly costIdle overheadTime to add capacityFailure isolation
Single dedicated box, systemd units$90–$150~0Hours (provisioning)None — one box, one blast radius
Single box, k3s + HPA + KEDA$90–$150 + 0.3 cores, 1.5 GB10–25% CPUSeconds (pods), hours (nodes)Per-pod; box failure still total
3-node managed K8s cluster$450–$1,2002 nodes’ worth of control planeMinutes (autoscaler)Real — node failure tolerated

The single-box K8s option costs the same in dollars and 10–25% of its capacity in overhead, buying faster deploys, per-pod isolation, and a scale-out path that does not require re-architecting. Whether that is worth a quarter of a box is a judgment call — but it is not free.

Decision Thresholds by Concurrency

These are starting points, not laws — the shape of the workload matters more than the count, so assume a session service with roughly 10–20 KB of state per player and a state write every 15–30 seconds.

  • Under 200 concurrent players: one dedicated box, systemd or Compose, no orchestrator. A deploy is systemctl restart and a 5-second blip.
  • 200–1,000: one bigger box, or two with a load balancer and session affinity, if you need per-service isolation or zero-downtime deploys. If you already know Kubernetes well, k3s here is defensible; if you are learning it, learn in staging.
  • 1,000–5,000: multi-node orchestration starts earning its cost. Two or three game-session nodes plus managed database, HPA on CPU or connection count, KEDA for any queue-driven component.
  • Above 5,000 across regions: orchestration is the only practical option; the hard problem becomes placement and cross-region state, not pod sizing.

Between 200 and 1,000 players the deciding factor is team familiarity rather than mathematics: the break-even is really the point where distinct services and deploy targets exceed what one shell script can manage.

If You Do Run It on One Box: Non-Negotiable Settings

  • Reserve the control plane’s memory with system-reserved, or the kubelet and your game pods will compete during a GC pause.
  • etcd on NVMe, fsync enabled. etcd fsync latency above 10 ms produces API server timeouts and flapping controllers — a failure mode that looks like a networking bug.
  • podDisruptionBudget on the session service even with one replica, so an upgrade does not take everything down at once.
  • Startup and readiness probes that reflect real readiness, not just “port open”. A session pod that accepts connections before it has loaded state will drop the first players that reach it.
  • Limit range on the namespace so a pod without a limit cannot be deployed — an unlimited pod on a single box can consume all memory and take the control plane with it.

None of this argues against Kubernetes — it argues for pricing it accurately. Set requests from measured p99 usage rather than a round number guessed at deploy time; guess round numbers and HPA will scale on a fiction.

Leave a Reply