Skip to content

KVM Virtualized Router

Jump to bottom
Will edited this page Jun 3, 2019 · 9 revisions

Fedora Server 30 (yes it's taken this long, but now SFP+ works.)

  1. Get everything up to date

  2. Install VIM (because that's how it goes)

  3. Add user to kvm and libvirt groups

  4. Ensure VM store is accessible to the libvirt group

    • chmod g+w /vm

  5. Add remote user SSH keys to authorized_keys

  6. Enable IOMMU

    • sudo grubby --args="intel_iommu=on" --update-kernel /boot/vmlinuz-4.16.6-302.fc28.x86_64

  7. Create VM

    1. Use OpenWRT 18.06.0

      1. Get the combined-ext4 image and decompress it

      2. Use dd to extend the size of the image file out to ~1GB

      3. Mount the image file via a loopback device with kpartx

        kpartx -v -a ./openwrt-18.06.2-x86-64-combined-ext4.img

      4. Using fdisk, because it's a sector-level stickler, note the starting sector of the rootfs then delete it.

      5. Again using fdisk, re-create the partition at the exact same sector, with the extent reaching to the end of the space in the image file.

      6. Run fsck on the file system to ensure its integrity

      7. Run resize2fs to extend it, then re-run fsck to check.

      8. Convert the image to qcow2 format

        qemu-img convert -f raw -O qcow2 openwrt-18.06.2-x86-64-combined-ext4.img openwrt-18.06.2-x86-64-combined-ext4.qcow2

    2. Configure network

    3. eth0 is attached to the bridge. Later, all other physical interfaces except WAN will be as well. Configure the bridge to use 10.0.0.1

    4. eth1 or whatever is "WAN" and set to use DHCP

    5. On the host:

      $ sudo virsh net-edit <network>

      <network ipv6='yes'>
      <name>vmnet</name>
      <uuid>56273237-b81e-4385-a559-4956722dbf9a</uuid>
      <bridge name='virbr0' stp='on' delay='0'/>
      <mac address='52:54:00:3f:57:6b'/>
      <domain name='vmnet'/>
      <ip address='10.0.0.2' netmask='255.255.255.0'>
      </ip>
      </network>

    6. On the host:

      $ sudo virsh net-destroy <network>
      $ sudo virsh net-start <network>

    7. You will need to shut down and restart the VM to make the networking interface happy.

End Result

VM NIC Configuration

eth0 - 52:54:00:d5:d8:25 - LAN virtual nic

ethX - ac:1f:6b:b0:02:70 - LAN 1GbaseT (switch)

eth1 - ac:1f:6b:b0:02:71 - WAN

ethY - ac:1f:6b:b0:02:72 - LAN 1GbaseT (misc)

ethZ - ac:1f:6b:b0:02:73 - LAN 1GbaseT (misc)

eth2 - ac:1f:6b:b0:79:64 - LAN 10GbaseT (main desktop)

eth3 - ac:1f:6b:b0:79:65 - LAN 10GbaseT (misc)

eth4 - ac:1f:6b:b0:79:66 - LAN 10Gbe SFP+ (NAS)

eth5 - ac:1f:6b:b0:79:67 - LAN 10Gbe SFP+ (misc)

IP Ranges

10.0.0.1 - OpenWRT instance with all NICs routed into it.

10.0.0.2 - NAS

10.0.0.3 - VM Host system (Supermicro A2SDi-TP8F

10.0.0.4 - VM Host BMC (dedicated interface -> switch -> VM)

10.0.0.5 - Network power control box

10.0.0.6 - Ubiquiti AP

10.0.0.7 - Managed switch

10.0.0.50 - Start of wired DHCP range

10.0.0.99 - End of wired DHCP range

10.0.0.0.100 - Main desktop

10.0.0.0.101 - RB-89 (NUC 1)

10.0.0.0.102 - Nano (NUC 2)

10.0.0.0.103 - PS4

10.0.0.150 - Start of wifi DHCP range

10.0.0.199 - End of wifi DHCP range

Clone this wiki locally