Skip to content
/ guest-machine-guide Public

Guide for setup of a Virtual Machine (VM) on the Linux desktop. Includes general overview, optimizations for Windows VMs and hardware-passthrough (VFIO), and more.

License

GPL-3.0 license
1 star 0 forks Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

portellam/guest-machine-guide

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

65 Commits
LICENSE
LICENSE
 
 
README.md
README.md
 
 

Repository files navigation

Guest Machine Guide

Status: Work-In-Progress

Guide for setup of a guest Libvirt/QEMU Virtual Machine (VM). Includes general overview and references, and optimizations for Windows guests and hardware-passthrough (VFIO).

Table of Contents

Contents

Why?

The purpose of this document is to inform a new or returning user how to optimize a Guest machine, without demanding greater research and time.

This document does not serve to replace existing knowledge-bases. If you have any unexpected questions, wish to fact-check, or want to expand your knowledge, then please visit these places!

Copy and paste what you need from here and/or any example XML files, to your Guest XML file.

Related Projects

Project Codeberg GitHub
Deploy VFIO link link
Auto X.Org link link
Generate Evdev link link
Guest Machine Guide link link
Libvirt Hooks link link
Power State Virtual Machine Manager link link

Documentation

Host Optimizations

TODO: add here.

Guest Optimizations

TODO: add here.

Guest XML Layout

Below is an incomplete layout for building a Guest machine. The lines include additional features, of which are absent when creating a Guest XML (with the virsh CLI command or virt-manager GUI application).

1. Syntax

<parent_tag_name attribute_name="attribute_value">
  <child_tag_name>child_tag_value</child_tag_name>
</parent_tag_name>

2. First Lines in XML

<domain> Tag Attribute Value Description
xmlns:qemu "http://libvirt.org/schemas/domain/qemu/1.0" Enable QEMU command lines and overrides.
type "kvm" Enable QEMU command lines and overrides.
<name/>1 none text Name of the Guest.
2.a. <name/> Best practice:

Note: The following formatting examples are a personal preference of the Author.

Format: purpose_vendor*_operating system_architecture_chipset_firmware_topology

* Optional, if Host machine contains two (2) or more video devices (GPU/VGA).

  • Example systems and names:

    • Modern gaming machine:      game_nvidia_win10_x64_q35_uefi_6c12t
    • Older 2000s gaming machine: retro_amd_winxp_x86_i440fx_bios_2c4t
    • Retro 1990s gaming machine: retro_3dfx_win98_x86_i440fx_bios_1c1t
    • Intel MacOS workstation:      work_macos_amd_x64_q35_uefi_6c12t

  • Purpose of the Guest (and suggested names):

    • Gaming PC:      game
    • Legacy/Retro PC:  retro
    • Server:           server
    • Workstation PC:  work
    • etc.

  • Vendor name of the Video device:

    • AMD:      amd

    • Intel:      intel

    • NVIDIA:    nvidia

    • emulated: virtgpu

    • non-mainstream or legacy:

      • 3DFX: 3dfx.

  • Short name of the Operating System (OS):

    • Apple Macintosh:   macos
    • Linux:              arch, debian, redhat, ubuntu
    • Microsoft Windows: win98, winxp, win10
    • etc.

  • Short name of the CPU architecture:

    • AMD/Intel 32-bit: x86
    • AMD/Intel 64-bit: x64
    • ARM 32-bit:       aarch32
    • ARM 64-bit:       aarch64
    • etc.

  • Virtualized chipset:

    • I440FX: i440fx

      • Emulated, older chipset by Intel.
      • Does support legacy Guests (example: Windows NT 5 and before, XP/2000, 9x).
      • PCI bus only; expansion devices will exist on a PCI topology.
      • Will accept PCIe devices.

    • Q35:    q35

      • Virtual, newer platform.
      • Native PCIe bus; expansion devices will exist on a PCIe topology.
      • Does not support legacy Guests.

  • Firmware:

  • Short-hand of Core topology: 4c8t

    • Given the amount of physical cores (example: 4).
    • Given the amount of logical threads per core (2 * 4 = 8).

3. Memory

To gather information of system memory, execute: free --kibi --total --wide For more information on this topic, click here.

<memory> Tag Attribute Value Description
parent a number The total amount of memory.
unit "KiB" Amount of memory in Kibibytes (1024 bytes, Kilo is 1000 bytes).
<currentMemory> Tag Attribute Value Description
parent a number The current amount of memory.
unit "KiB" Amount of memory in Kibibytes (1024 bytes, Kilo is 1000 bytes).
<memoryBacking> Tag Attribute Value Description
<allocation/> mode immediate Specifies how memory allocation is performed.
<discard/> none none TODO: define, what is this?
<hugepages/> none none Enable Huge memory pages.
<nosharepages/> none none Prevents the Host from merging the same memory used among Guests.
3.a. <hugepages/>
  • Static allocation of Host memory pages into Guest memory pages.
  • Huge: Memory page size greater than 4K bytes (2M or 1G bytes). The greater the size, the lower the Host overhead.
  • Dynamic Host memory page allocation is more flexible, but will require defragmentation before use as Guest memory pages (before a Guest machine may start).
  • Warning: If the specified Guest memory pages exceeds the allocated Host memory pages, then the Guest machine will fail to start.

For more information on this topic, click here.

4. CPU Topology (1 / 2)

To gather information about your CPU, execute: lscpu | grep --extended-regexp --ignore-case "per core|per socket|socket"

<vcpu> Tag Attribute Value Description
parent a number Specify number of Host threads for Guest (same as <cpu> tag).
placement "static" Static allocation of Guest CPU threads.
<iothreads> Tag Attribute Value Description
none a number Specify number of Host threads to manage storage block devices (see as <iothreadpin> tag).
<cputune> Tag Attribute Value Description
<vcpupin> vcpu a number Guest CPU: the Guest thread ID number.1
<vcpupin> cpuset a number Guest CPU: the Host thread ID number.2
<emulatorpin> cpuset a number Guest IRQ: the Guest IRQ thread ID number.3
<iothreadpin> iothread a number Guest IO: the Guest IO thread ID number.
<iothreadpin> cpuset a number Guest IO: the Host thread ID numbers.4
4.a. <vcpupin vcpu>
  • Count does not exceed value as defined in <vcpu placement>.
4.b. <vcpupin cpuset>
  • Threads should not overlap Host process threads.
4.c. <emulatorpin cpuset>
  • Emulator threads handle Interrupt Requests for Guest hardware emulation.
  • Threads should not overlap Guest CPU threads as defined in vcpupin cpuset.
4.d. <iothreadpin cpuset>
  • IO threads handle IO processes for Guest virtual drives/disks.
  • Threads should not overlap Guest CPU threads as defined in vcpupin cpuset.
4.e. Example XML:
  <!-- Given a 4-core, 8-thread CPU... -->
  <vcpu placement="static">4</vcpu>           <!-- Statically allocate four (4) cores to Guest. -->
  <iothreads>1</iothreads>                    <!-- Define one (1) thread to IO. -->
  <cputune>
    <vcpupin vcpu="0" cpuset="2"/>            <!-- Guest CPU: use the third core, first thread. -->
    <vcpupin vcpu="1" cpuset="6"/>            <!-- Guest CPU: use the third core, second thread. -->
    <vcpupin vcpu="2" cpuset="3"/>            <!-- Guest CPU: use the fourth core, first thread. -->
    <vcpupin vcpu="3" cpuset="7"/>            <!-- Guest CPU: use the fourth core, second thread. -->
    <emulatorpin cpuset="1,4"/>               <!-- Guest IRQ: use the second core, two threads. -->
    <iothreadpin iothread="1" cpuset="1,4"/>  <!-- Guest IO: use the second core, two threads. -->
  </cputune>

5. System Information Spoofing

To gather information about your BIOS, execute:

 sudo dmidecode --type bios | grep --extended-regexp --ignore-case "vendor|version|release date"

To gather information about your system, execute:

 sudo dmidecode --type system | grep --extended-regexp --ignore-case "manufacturer|product name|version|serial number|sku number|family"

  <!-- BIOS and System spoofing (you may copy your actual info). -->
  <sysinfo type="smbios">                                       <!-- This line is necessary! -->
    <bios>
      <entry name="vendor">American Megatrends Inc.</entry>     <!-- AMI is the industry standard BIOS vendor. -->
      <entry name="version">version_of_bios_firmware</entry>
      <entry name="date">MM/DD/YYYY</entry>
    </bios>
    <system>
      <entry name="manufacturer">vendor_of_motherboard</entry>
      <entry name="product">product_name</entry>
      <entry name="version">Default string</entry>
      <entry name="serial">Default string</entry>
      <entry name="sku">Default string</entry>
      <entry name="family">Default string</entry>
    </system>
  </sysinfo>

6. Features

TODO: make the following inline XML into chart, describe each feature.

    <hyperv mode="custom">
      <relaxed state="on"/>
      <vapic state="on"/>
      <spinlocks state="on" retries="8191"/>
      <vpindex state="on"/>
      <runtime state="on"/>
      <synic state="on"/>

      <stimer state="on">
        <direct state="on"/>
      </stimer>

      <reset state="on"/>
      <vendor_id state="on" value="1234567890ab"/>
      <frequencies state="on"/>
      <reenlightenment state="on"/>
      <tlbflush state="on"/>
      <ipi state="on"/>
      <evmcs state="on"/>
    </hyperv>
    <kvm>
      <hidden state="on"/>
    </kvm>
    <vmport state="off"/>
    <ioapic driver="kvm"/>
  </features>

7. CPU Topology (2 / 2)

To gather information about your CPU, execute: lscpu | grep --extended-regexp --ignore-case "per core|per socket|socket"

7.a. Example output:
Thread(s) per core:                 2
Core(s) per socket:                 8
Socket(s):                          1

TODO: make the following inline XML into chart, describe each feature.

  <cpu mode="host-passthrough" check="none" migratable="on">  <!-- Spoof the CPU info, with the actual CPU info. -->
    <topology sockets="1" dies="1" cores="6" threads="2"/>
    <cache mode="passthrough"/>
    <feature policy="disable" name="hypervisor"/>
    <feature policy="disable" present="yes"/>
    <timer name="tsc" present="yes" mode="native"/>
  </clock>
<cpu> Tag Attribute Value Description
parent mode "host-passthrough" Spoof the Guest CPU info, with the actual Host CPU info.
check "none" TODO: add here.
migratable "on" TODO: add here.
<topology> sockets a number The number of CPU sockets, or maxmium number of dies.
dies a number The number of CPU dies (typically one).
cores a number The number of CPU cores.
threads a number The number of CPU threads per core (typically a factor of two).
<cache> mode "passthrough" Passthrough the Host CPU cache.
<feature> policy "disable" Disables policy.
name "hypervisor" TODO: add here.
<feature> policy "disable" Disables policy.
present "yes" TODO: add here.
<timer> name "tsc" TODO: add here.
present "yes" TODO: add here.
mode "native" TODO: add here.

8. Power Management

  <!-- Power Management -->
  <pm>
    <suspend-to-mem enabled="yes"/>   <!-- Enable S3 Suspend (Sleep) -->
    <suspend-to-disk enabled="yes"/>  <!-- Enable S4 Suspend (Hibernation) -->
  </pm>

9. Devices

For more information on this topic, click here.

TODO: make the following inline XML into chart, describe each feature???

  <!-- Emulated, Paravirtual, Passed-through Real PCI/e, and Shared Memory devices -->
  <devices>
  ...
  </devices>
9.a. Emulated Devices

TODO: make this inline XML?

lorem ipsum

9.b. Real/Passthrough Hardware Devices

TODO: make this inline XML?

  • AMD
    • Integrated
    • Dedicated
    • Reset bug
  • Intel
    • Integrated
    • Dedicated
      • Note: Not much research done regarding this topic. Please refer to any mentioned guides, the Reddit forum, or use an Internet search engine with the keywords intel gpu vfio.
  • NVIDIA
    • Problems:
      • Boot bug (Solution: Use known-good copy of given GPU's video BIOS or VBIOS).
      • TODO: add references, programs used, instructions, and XML here.
9.c. Memory Devices

TODO: make this inline XML?

  • Looking Glass
  • Scream
  • ???

10. QEMU Command Line

TODO: make the following inline XML into chart, describe each feature.

  • Evdev
  <qemu:commandline>...</qemu:commandline>  <!-- Add Evdev here -->

11. QEMU Overrides

TODO: make the following inline XML into chart, describe each feature.

  <qemu:override>...</qemu:override>

Benchmarking Guest Performance

TODO: add here.

Contact

Do you need help? Please visit the Issues page (Codeberg, GitHub).

References

1.

8.2. Memory Tuning on Virtual Machines Red Hat Enterprise Linux 7. Red Hat Customer Portal. Accessed June 15, 2024. https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/virtualization_tuning_and_optimization_guide/sect-virtualization_tuning_optimization_guide-memory-tuning.

2.

AARCH64. Wikipedia, May 30, 2024. https://en.wikipedia.org/wiki/AArch64.

3.

ARM Architecture Family. Wikipedia, May 31, 2024. https://en.wikipedia.org/wiki/ARM_architecture_family#AArch32.

4.

BIOS. Wikipedia, June 7, 2024. https://en.wikipedia.org/wiki/BIOS.

5.

Hugepages. Hugepages - Debian Wiki. Accessed June 15, 2024. https://wiki.debian.org/Hugepages.

6.

I440fx PC (PC-I440fx, PC). i440fx PC (pc-i440fx, pc) - QEMU documentation. Accessed June 15, 2024. https://www.qemu.org/docs/master/system/i386/pc.html.

7.

Intel 440FX. Wikipedia, August 20, 2023. https://en.wikipedia.org/wiki/Intel_440FX.

8.

PCI Overview. QEMU Wiki. Accessed June 15, 2024. https://wiki.qemu.org/images/f/f6/PCIvsPCIe.pdf.

9.

PCI passthrough via OVMF. ArchWiki. Accessed June 14, 2024. https://wiki.archlinux.org/title/PCI_passthrough_via_OVMF.

10.

Q35. QEMU Wiki. Accessed June 15, 2024. https://wiki.qemu.org/images/4/4e/Q35.pdf.

11.

r/VFIO. Accessed June 14, 2024. https://www.reddit.com/r/VFIO/.

12.

UEFI. Wikipedia, June 7, 2024. https://en.wikipedia.org/wiki/UEFI.

13.

VFIO - ‘Virtual Function I/O’ - The Linux Kernel Documentation. The linux kernel. Accessed June 14, 2024. https://www.kernel.org/doc/html/latest/driver-api/vfio.html.

14.

X64. Wikipedia, July 27, 2023. https://en.wikipedia.org/wiki/X64.

15.

X86. Wikipedia, May 18, 2024. https://en.wikipedia.org/wiki/X86.

TODO:

  • add About.
  • add Documentation for:
    • guest XML layout
      • name syntax
      • introduction
      • memory
      • cpu topology (1/2)
      • system information spoofing
      • features
      • cpu topology (2/2)
      • power management
      • qemu command line
      • qemu overrides
    • host optimizations
    • guest optimizations
    • benchmarking performance
  • add References.
  • use collapsable dropdowns (https://gist.github.com/pierrejoubert73/902cc94d79424356a8d20be2b382e1ab).
  • add Guest XML file with all lines referenced in "Guest XML Layout".
  • add pictures of virt-manager.
  • flesh out "Guest XML Layout".
  • how to setup virtio disks (virtio, virtio-scsi).

About

Guide for setup of a Virtual Machine (VM) on the Linux desktop. Includes general overview, optimizations for Windows VMs and hardware-passthrough (VFIO), and more.

Topics

documentation virtual-machine guide virtualization kvm guest qemu libvirt vfio vfio-pci passthrough hyperv pci-passthrough hugepages vfio-setup vfio-passthrough

Resources

Readme

License

GPL-3.0 license
Activity

Stars

1 star

Watchers

1 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

No packages published