IOMMU, SR-IOV, and GPU Passthrough on Rented Dedicated Hardware

Running a KVM guest with a passed-through GPU on rented dedicated hardware is possible, but whether it works depends less on your software stack and more on what the provider’s BIOS exposes. If IOMMU groups are not properly separated, or if the host was built without ACS, you can spend a week on configuration and still see the guest hang on the first frame. This page covers the checks to run before you sign and the tickets to file when something is missing. If you are choosing a host for this, review the dedicated server hosting plans and ask about IOMMU support first.

What the provider must expose in BIOS

Four firmware settings decide whether passthrough is possible: IOMMU enablement (VT-d on Intel, AMD-Vi on AMD), ACS at the PCIe root complex, Above 4G Decoding, and Resizable BAR where GPUs need it. On AMD EPYC, IOMMU is usually on by default in server BIOS but ACS can be disabled to reduce latency; on Intel, VT-d is often disabled in the “server balanced” profile.

Ask the provider to confirm all four in writing to the specific server build. A provider that says “we do not allow BIOS changes” can still have IOMMU enabled by default, but you will have no way to enable ACS or enable SR-IOV if the firmware lacks the toggle. Request the exact BIOS version as well, since some EPYC 7002 boards shipped with broken IOMMU grouping in early firmware.

Checking IOMMU and ACS groups

Boot Linux, add intel_iommu=on iommu=pt (or amd_iommu=on iommu=pt) to the kernel command line, and check dmesg for “DMAR: IOMMU enabled” or “AMD-Vi: Interrupt remapping enabled”. Then list groups with a short shell loop over /sys/kernel/iommu_groups/. Each group should contain the GPU and its audio function together, and nothing else that you cannot rebind.

Print the groups with a one-liner over /sys/kernel/iommu_groups/ and check each group against the table below. Anything in a group with a device you cannot afford to move is a blocker.

• A clean GPU group contains only the VGA function and its HD-audio function.
• A dirty group contains the GPU plus a PCIe root port, a NIC, or an NVMe controller.
• Root ports in the same group usually indicate ACS is disabled or unsupported on that slot.
• Groups that contain a bridge force you to pass the whole bridge, which breaks other devices.

Check ACS control with lspci -vvv and look for “ACSCtl: SrcValid+ TransBlk+ ReqRedir+ CmpltRedir+ UpstreamFwd+ EgressCtrl+ DirectTrans+”. Missing plus signs mean the slot’s PCIe switch or root complex does not support ACS isolation. If only “SrcValid-” is missing, you may still be able to use the pcie_acs_override kernel patch, but it is unsafe for production because it can route DMA between devices in the same group.

VFIO binding and the guest configuration

Bind the GPU and its audio function to vfio-pci at boot using driver_override, not by blacklisting the host driver, so you can rebind without a reboot. Verify with lspci -knn that both functions show “Kernel driver in use: vfio-pci”. Then pass them to QEMU with -device vfio-pci,host=01:00.0. Include the audio function in the same group; some GPUs will hang if audio is left on the host.

For the guest, use UEFI OVMF with a dedicated OVMF vars file per VM, and set the machine type to q35 with the GPU in a root port that supports PCIe hotplug. Set the CPU model to host-passthrough so AVX2 and AVX-512 reach the guest, and add kvm.ignore_msrs=1 on the host if the guest driver raises MSR faults.

Add -cpu host,kvm=off if the guest is a consumer Windows build that refuses to load a driver when it detects a hypervisor. Expect 2 to 5 percent overhead versus bare metal.

SR-IOV virtual function limits on 10/25GbE

SR-IOV NICs split a physical function into virtual functions that each get their own PCIe function and queue set. Intel X710 and E810 support 128 VFs per port with the right firmware; Mellanox ConnectX-4 and later support up to 127 VFs per port, and ConnectX-6 goes higher. The limit is usually not the NIC but the number of PCIe functions the BIOS or the host kernel can enumerate.

For 10GbE, plan one VF per guest with 2 to 4 queues and confirm that the firmware has SR-IOV enabled. For 25GbE, a single VF with 8 queues can often saturate the port, so one VF per guest is enough. Watch the host for “too many memory regions” errors once a guest uses more than 64 VFs; the IOMMU has a finite number of DMA mapping entries.

Consumer GPU passthrough pitfalls

Consumer GeForce cards work, but they generate most of the tickets. The card’s VBIOS may not initialise when it is not the primary display adapter, so add a dummy HDMI plug or keep the host console on an integrated GPU. Most GeForce cards also cannot reset cleanly after a guest crash without a host reboot.

Watch for the “Code 43” error in Windows guests, which indicates hypervisor detection, and for driver installs that fail when the card sits behind a PCIe switch with limited BAR space. Enterprise cards such as the T4 or L4 avoid most of these issues, but cost more and need passive cooling. Consumer cards rarely support SR-IOV, so each VM needs its own physical GPU.

Nested virtualisation flags

If the guest itself runs KVM, Docker with a runtime that needs KVM, or WSL2, the host must expose nested virtualisation. On Intel this is kvm_intel nested=1 and requires VMX to be passed to the guest with -cpu host. On AMD it is kvm_amd nested=1 and requires SVM. Load the module with the flag and persist it in /etc/modprobe.d/.

Verify inside the guest with grep -c vmx /proc/cpuinfo (Intel) or grep -c svm /proc/cpuinfo (AMD). Nested KVM works but adds 5 to 15 percent overhead and does not support all PMU features, so do not run a nested hypervisor in production unless you control the entire stack. Nested virtualisation also does not help with IOMMU isolation: any device passed to an L2 guest must already be cleanly grouped at L1.

Testing before you commit

Run the whole check in a two-hour trial before signing an annual contract. Install a minimal Debian, enable IOMMU, list the groups, and bind the GPU to vfio-pci. Create a Windows guest with the GPU passed through and install the driver. If the driver installs and a benchmark such as FurMark or Blender renders without a host reset, the build is viable. Record the full lspci -vvv output for the slot.

Repeat the test after a guest hard reset to confirm the card returns to a working state. Many consumer cards fail this step and require a host reboot, which is unacceptable for a hosting product. If the trial server lacks IOMMU, ask for EPYC or Xeon Scalable instead; both have more predictable grouping than desktop chipsets.

The support tickets to file

Be specific in each ticket, and cite the firmware setting you need. For IOMMU: “Please enable VT-d / AMD-Vi and ACS in the BIOS for this server, and confirm the BIOS version.” For SR-IOV: “Please enable SR-IOV in the NIC firmware and confirm the maximum VF count per port.” For the GPU: “Please confirm that above 4G decoding is enabled and that the GPU slot’s root port supports ACS.”

If the provider refuses BIOS changes, ask whether they will swap the server for one that already has IOMMU enabled, or whether they offer a dedicated GPU plan with passthrough pre-configured. A few providers publish the setting on the product page, which is the easiest case. Compare dedicated server hosting options with GPU support and filter by providers that list IOMMU or GPU passthrough as a feature, then run the two-hour test before you sign.