Choosing Storage Tiering for Game Asset CDNs: NVMe vs SATA SSD vs HDD Placement Rules

Not All Game Data Deserves the Same Disk

A game asset CDN serves three completely different kinds of data: hot assets read thousands of times per minute, warm assets read on join or map change, and cold archive data read almost never but kept forever. If all three live on the same NVMe array, you are paying the most expensive storage tier in the machine to hold data that will never be read again. If all three live on the same spinning disk, joins take forty seconds and players leave. Storage tiering is the practice of assigning each data class to the media that matches its access pattern. This guide gives you the access-frequency thresholds, the media comparison, and a working tiering layout for a dedicated game asset server.

Storage is one of the three specs that decide whether a dedicated host can serve assets at scale — the dedicated server comparison breaks out drive bays and NVMe capacity per plan, which sets the ceiling on your tiering design.

Tiering by Access Pattern, Not by Media Hype

Assign every asset to a tier using its read frequency and its latency tolerance. The numbers below are the thresholds that actually separate tiers in a game CDN workload:

Tier Media Access pattern Latency Cost/TB (relative)
T0 — Hot NVMe Gen4/Gen5 (local) >1,000 reads/hour, latency-critical 50-100 µs 10×
T1 — Warm SATA/SAS SSD 10-1,000 reads/hour 100-250 µs 4×
T2 — Cold 7,200-rpm HDD (RAID 10) <10 reads/hour, large sequential 4-12 ms 1×
T3 — Archive Object storage / offsite Rarely, or never after season end 50-500 ms 0.3×

The important insight is that tier boundaries are defined by read frequency, not by file size. A 4 GB map pak file read once per server restart belongs in T2 even though it is huge. A 200 KB weapon skin texture read by every player on every round belongs in T0 even though it is tiny.

Why NVMe vs SATA Matters More Than Raw Throughput

The headline sequential numbers (NVMe Gen4 at 7,000 MB/s, SATA SSD at 550 MB/s) are misleading for a CDN, because asset serving is overwhelmingly random small reads from many concurrent connections, not one big sequential stream. What matters is IOPS at a queue depth of 1-4 per connection:

  • NVMe Gen4: ~600,000 random 4K read IOPS, sub-100 µs latency. Handles thousands of concurrent asset requests with headroom.
  • SATA SSD: ~90,000-100,000 random 4K IOPS, ~120 µs latency. Bottleneck appears around 1,500-2,000 concurrent requests.
  • 7,200-rpm HDD: 120-180 random IOPS. Fine for sequential streaming of one large file, catastrophic for parallel small reads.

This is why a CDN with two NVMe drives will outperform the same box with eight HDDs by an order of magnitude even though the HDD array has more total capacity. Do not put hot assets on spinning disks and expect RAID to save you — RAID improves throughput and redundancy, not seek latency.

A Working Tiering Layout

A concrete layout for a dedicated game asset server with 2× NVMe Gen4 (2 TB each) and 4× 8 TB SATA HDD:

# Drives
/dev/nvme0n1  2 TB  T0 hot + T2 write cache
/dev/nvme1n1  2 TB  T0 hot (mirror of nvme0n1 via RAID 1)
/dev/sda..sdd 4×8TB T2 cold, RAID 10 → /mnt/cold (16 TB usable)

# Mount layout
/mnt/hot      (nvme, RAID1)   hot assets, thumbnails, manifests, indexes
/mnt/cold     (hdd, RAID10)   maps, paks, larger content packs
/mnt/cache    (nvme, tmpfs-backed journal)    write-through cache for cold

# Fstab
/dev/md0  /mnt/hot   ext4  noatime,nodiratime,discard  0 2
/dev/md1  /mnt/cold  xfs   noatime,allocsize=1m        0 2

Use ext4 on NVMe and XFS on the HDD array. XFS handles large sequential files (pak/map files) better and its delayed allocation reduces fragmentation on a cold-tier volume. Mount everything with noatime to eliminate the metadata write on every read — on an asset server, atime updates alone can consume 5-10% of IOPS.

Promotion and Eviction Rules

Tiers only save money if data moves between them automatically. Implement the following rules on the CDN edge:

  1. Promote on second read within 10 minutes. If an asset in T2 is requested twice in a short window (e.g. a map change everyone loads), copy it into T0 and serve subsequent requests from NVMe. This is the single highest-impact rule for join-time latency.
  2. Evict from T0 by LRU with a size cap. Keep T0 under 80% capacity. A full hot tier loses the ability to promote during a traffic spike — exactly when you need it.
  3. Never promote files larger than a threshold. A 4 GB map file promoted to T0 evicts thousands of small genuinely-hot assets. Set the promotion size cap at 64-128 MB and serve larger files from T2 with readahead.
  4. Demote on season rollover, not on age. Assets from a completed season belong in T3 archive regardless of how recently they were read. A clean seasonal boundary is the cheapest demotion trigger you have.

On nginx a promotion rule for the cold tier looks like this — cache to NVMe on second hit within the window, serve from cache thereafter:

proxy_cache_path /mnt/cache levels=1:2 keys_zone=assetcache:512m
                 max_size=1400g inactive=10m use_temp_path=off;

location /assets/ {
    proxy_cache assetcache;
    proxy_cache_min_uses 2;          # promote on 2nd request
    proxy_cache_valid 200 206 7d;
    proxy_cache_lock on;             # collapse thundering herd on map change
    proxy_cache_use_stale updating error timeout;
    proxy_pass http://origin_cold_tier;
}

proxy_cache_lock on is critical during a map change: without it, 200 players requesting the same new map file simultaneously generate 200 origin reads and saturate the HDD tier. With it, one read hits the origin and the rest wait for that single response.

Sizing Check: When Tiering Stops Helping

  • If your entire asset set fits in NVMe with 30% headroom, do not tier. Complexity costs more than the drives save below roughly 2 TB of hot data.
  • If your hot set exceeds 1 TB, add NVMe capacity before adding HDDs — the HDD tier only pays off when there is genuinely cold data to put on it.
  • Track the promotion rate. If more than 20% of T2 reads result in a promotion, your hot tier is undersized or your classification is wrong.
  • Measure join latency, not disk utilization. The goal is players in the world in under 15 seconds, and tiering is only justified when it moves that number.

Storage tiering is a data-placement discipline, not a hardware purchase. Get the classification right and a two-drive NVMe plus four-disk HDD machine will beat an all-NVMe server on cost per player served — while matching it on join latency. For the bandwidth side of the same CDN, see the commit vs burst bandwidth guide.

If you want the drive layout above without buying hardware, a price-locked dedicated server with dual NVMe plus configurable HDD bays gives you both tiers in one machine. InterServer’s dedicated servers keep the same monthly rate for the life of the plan, and promo code TRYINTERSERVER drops the first month to $0.01 so you can benchmark join latency across a real two-tier layout before migrating production.

Leave a Reply