GPU passthrough
Also called: PCI passthrough, vfio, GPU virtualization
GPU passthrough gives a virtual machine direct hardware access to a physical GPU, so it can run accelerated workloads at close to bare-metal speed.
GPU passthrough hands a physical graphics card directly to a virtual machine using the host’s IOMMU, bypassing the hypervisor’s device emulation. The guest sees real hardware and can use the vendor driver and CUDA or ROCm at close to native speed.
For local AI this is often the difference between usable and not: without it a VM falls back to CPU inference, which can be an order of magnitude slower.
The requirements are fiddly. The CPU and motherboard must support IOMMU (Intel VT-d or AMD-Vi) and it must be enabled in firmware. The GPU needs to sit in its own IOMMU group, which some consumer boards make awkward. The host must not be using the card, so a second GPU or headless operation is usually required.
Apple Silicon has no equivalent — but its unified memory means a VM can often be given enough memory to run models directly.