Proxmox IOMMU Group Split for PCIe devices (ACS patch)

Tested on

• AMD Ryzen 5 1600
• Gigabyte AB350 Gaming 3, BIOS F51B
• Proxmox VE 7.4-3

• The reason why you might want this is if your motherboard has crap IOMMU groups by default but you want to pass two PCIe devices separately into a VM, or only one device in a group.
• In my case, the only PCIe slot with it's own group was the second 8x slot in the middle. Which meant I could only pass through my HBA or my GPU.
• Fortunately, you can split those groups in software.
  • Keep in mind this doesn't actually split anything. It will just tell the OS to pretend they are split. This can cause issues when hardware in the same physical IOMMU group shares drivers.
    • For example, if you have two GPUS in the same IOMMU group and you apply the override and pass through each GPU to a separate VM, drivers from each VM will be able to access both.
  • But for my usecase, a GPU and a disk controller, it works without issues.

• Open the following file

 nano /etc/default/grub

• At the start should be a section with following content

GRUB_DEFAULT=0
GRUB_TIMEOUT=5
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT=""
GRUB_CMDLINE_LINUX=""

• Add the following line to the GRUB_CMDLINE_LINUX_DEFAULT parameter

 quiet amd_iommu=on pcie_acs_override=downstream,multifunction

• CTRL + O to save in Nano
• CTRL + X to exit Nano
• Run

  update-grub

• reboot

• Add the following line to the GRUB_CMDLINE_LINUX_DEFAULT parameter

quiet intel_iommu=on pcie_acs_override=downstream,multifunction

• CTRL + O to save in Nano
• CTRL + X to exit Nano
• Run

  update-grub

• reboot