pfSense in a VM
pfSense works great in a VM, but there are a few extra steps that need to be taken first.
1. Create VM as normal.
- When creating the VM, choose the
other install mediaVM template
- Continue through the pfSense installer like normal
2. Install Guest Utilities
Now that you have a pfSense VM running, we need to install guest utilities and tell them to run on boot. SSH (or other CLI method) to the pfSense VM and perform the following:
pkg install xe-guest-utilities echo 'xenguest_enable="YES"' >> /etc/rc.conf.local ln -s /usr/local/etc/rc.d/xenguest /usr/local/etc/rc.d/xenguest.sh service xenguest start
Guest Tools are now installed and running, and will automatically run on every boot of the VM.
3. Disable TX Checksum Offload
Now is the most important step: we must disable tx checksum offload on the virtual xen interfaces of the VM. This is because network traffic between VMs in a hypervisor is not populated with a typical ethernet checksum, since they only traverse server memory and never leave over a physical cable. The majority of operating systems know to expect this when virtualized and handle ethernet frames with empty checksums without issue. However
pf in FreeBSD does not handle them correctly and will drop them, leading to broken performance.
NOTE: Disabling checksum offloading is only necessary for virtual interfaces. When using PCI Passtrough to provide a VM with direct access to physical or virtual (using SR-IOV) devices it is unnecessary to disable tx checksum offloading on any interfaces on those devices.
The solution is to simply turn off checksum-offload on the virtual xen interfaces for pfSense in the TX direction only (TX towards the VM itself). Then the packets will be checksummed like normal and
pf will no longer complain. SSH to dom0 on your XCP-NG hypervisor and run the following:
First get the UUID of your pfSense VM:
Find your pfsense VM in the list, and copy the UUID. Now stick the UUID in the following command:
xe vif-list vm-uuid=08fcfc01-bda4-21b5-2262-741da6f5bfb0
This will list all the virtual interfaces assigned to the VM:
uuid ( RO) : 789358b4-54c8-87d3-bfb3-0b7721e4661b vm-uuid ( RO): 57a27650-6dab-268e-1200-83ee17ee3a55 device ( RO): 1 network-uuid ( RO): 5422a65f-4ff0-0f8c-e8c3-a1e926934eed uuid ( RO) : a9380705-8da2-4bf7-bbb0-f167d8f0d645 vm-uuid ( RO): 57a27650-6dab-268e-1200-83ee17ee3a55 device ( RO): 0 network-uuid ( RO): 4f7e43ef-d28a-29bd-f933-68f5a8f36241
For each interface, you need to take the UUID (the one at the top labeled
uuid ( RO)) and insert it in the
xe vif-param-set uuid=xxx other-config:ethtool-tx="off" command. So for our two virtual interfaces the commands to run would look like this:
xe vif-param-set uuid=789358b4-54c8-87d3-bfb3-0b7721e4661b other-config:ethtool-tx="off" xe vif-param-set uuid=a9380705-8da2-4bf7-bbb0-f167d8f0d645 other-config:ethtool-tx="off"
That's it! For this to take effect you need to fully shut down the VM then power it back on. Then you are good to go! Do not forget: If you ever add more virtual NICs to your pfSense VM, you will need to go back and run the above command for them as well.
NOTE: Many guides on the internet for pfSense in Xen VMs will tell you to uncheck checksum options in the pfSense web UI, or to also disable RX offload on the Xen side. These are not only unnecessary, but some of them will make performance worse. The above is all that's required.