How Much Performance Do You Lose with QEMU/KVM GPU Passthrough?

One of the most common questions we hear from organizations migrating away from VMware is surprisingly simple: How much GPU performance do I lose if I move to KVM?

It’s a fair question.

For AI inference, machine learning, HPC, and CUDA workloads, GPUs are often the most valuable component in a server. Even a small performance penalty can matter when workloads run continuously or when infrastructure is heavily utilized.

The expectation is usually that a virtual machine must introduce some overhead. After all, there is another software layer between the application and the hardware. Fortunately, modern PCIe passthrough (VFIO) works differently.

Rather than emulating a GPU, the hypervisor simply maps the physical PCIe device directly into the guest operating system. Once the guest driver has taken ownership of the device, almost all CUDA commands execute directly on the hardware with minimal hypervisor involvement.

Academic research has shown this for years. Walters et al. measured GPU passthrough across KVM, Xen, VMware ESXi and LXC, finding KVM achieved 98-100% of native performance for CUDA and OpenCL workloads. VMware ESXi also performed well, generally achieving 96-99% of native performance¹.

Still, every environment is different. So we decided to measure it ourselves on Pextra CloudEnvironment, using a pair of NVIDIA Tesla V100S GPUs and a mix of basic AI inference and CUDA compute workloads.

Test Environment

Both the host and guest used an identical software stack to eliminate variables.

Host

• Ubuntu 24.04.4 LTS
• Linux 6.8.0-117-generic
• Dell PowerEdge R7525
• 2× AMD EPYC 7542 (128 logical CPUs)
• 512 GiB RAM
• 2× NVIDIA Tesla V100S PCIe 32GB
• NVIDIA Driver 580
• CUDA 12.5

Guest

• Ubuntu 24.04.4 LTS
• Linux 6.8.0-117-generic
• QEMU 8.2.2
• 64 vCPUs
• 128 GiB RAM
• Same NVIDIA Driver (580)
• Same CUDA version (12.5)

Both Tesla V100S GPUs were attached using VFIO PCI passthrough, giving the guest direct ownership of the hardware. Because the benchmarked workloads execute almost entirely on the GPU, allocating additional vCPUs or memory would have negligible impact on the measured results.

Benchmark Methodology

Rather than relying on a single benchmark, we evaluated several aspects of GPU performance:

• llama.cpp using CUDA 12.5 with Llama 3 70B GGUF
  • Prompt processing (PP512)
  • Token generation (TG128)
• gpu_burn for compute throughput
• CUDA bandwidth tests (bandwidthTest) for memory transfer rates:
  • Host → Device
  • Device → Host
  • Device → Device

These workloads represent a mixture of modern AI inference and traditional CUDA compute.

Results

Across every benchmark we observed approximately 0.4–1.6% performance degradation. For most workloads, this is well within the margin of error for repeated runs, and is unlikely to be noticeable in production. The largest difference appeared during host-device memory transfers, while compute-heavy workloads such as gpu_burn and Llama inference stayed extremely close to bare metal.

Why Is The Overhead So Small?

PCI passthrough is fundamentally different from device emulation.

With VFIO, QEMU is responsible for assigning ownership of the PCIe device during VM startup. After that point, the NVIDIA driver inside the guest communicates directly with the physical GPU. Tensor Core execution, CUDA kernels, GPU memory management and scheduling all occur on the hardware itself.

The hypervisor is largely removed from the critical execution path, leaving only small costs from interrupt delivery, PCIe virtualization, memory address translation, and VM scheduling.

How Do These Results Compare To Existing Research?

The 2014 IEEE Cloud Computing paper GPU Passthrough Performance: A Comparison of KVM, Xen, VMware ESXi, and LXC reported that KVM achieved between 98% and 100% of native GPU performance across CUDA and OpenCL benchmarks. VMware ESXi similarly delivered 96–99% of bare-metal performance depending on workload¹.

VMware has published similar findings over the years². Internal vSphere testing of GPU passthrough and DirectPath I/O consistently showed only a few percent difference from bare metal for machine learning and HPC workloads, with some benchmarks falling within normal run-to-run variance³.

Our test environment used Pextra CloudEnvironment, which is built on KVM/QEMU. From a GPU passthrough perspective, the same fundamental principle applies: once the GPU is assigned directly to the guest, there is very little virtualization remaining in the execution path. So long as the host and guest are configured correctly, performance should be nearly identical to bare metal.

Conclusion

GPU passthrough has matured significantly over the past decade.

Our benchmarks show that modern KVM/QEMU with VFIO delivers essentially native GPU performance for CUDA workloads, with measured overhead between 0.4% and 1.6% across AI inference, compute throughput and memory bandwidth tests.

Organizations leaving VMware often expect GPU workloads to suffer after moving to KVM. In practice, that simply isn’t what we observe. For AI inference, CUDA applications, LLM serving, rendering, scientific computing and other accelerator-heavy workloads, GPU passthrough behaves almost identically to running directly on the host¹.

References