Permalink
Browse files

xhyve

  • Loading branch information...
mist64 committed Jun 10, 2015
1 parent 04180a5 commit 6322d865314a4cd1db72834c51bde292341c8f07
Showing with 18,349 additions and 24,519 deletions.
  1. +103 −0 Makefile
  2. +184 −0 README.md
  3. +0 −50 bhyve/Makefile
  4. +0 −325 bhyve/bhyve.8
  5. +0 −892 bhyve/bhyverun.c
  6. +0 −283 bhyve/pci_emul.h
  7. +0 −790 bhyve/pci_passthru.c
  8. +0 −104 bhyve/spinup_ap.c
  9. +0 −230 bhyve/xmsr.c
  10. +0 −16 bhyvectl/Makefile
  11. +0 −2,142 bhyvectl/bhyvectl.c
  12. +0 −13 bhyveload/Makefile
  13. +0 −157 bhyveload/bhyveload.8
  14. +0 −746 bhyveload/bhyveload.c
  15. +73 −0 config.mk
  16. +18 −20 {bhyve → include/xhyve}/acpi.h
  17. +8 −11 {bhyve → include/xhyve}/ahci.h
  18. +24 −25 {bhyve → include/xhyve}/block_if.h
  19. +3 −7 bhyve/mptbl.h → include/xhyve/dbgport.h
  20. +89 −0 include/xhyve/firmware/kexec.h
  21. +34 −34 {bhyve → include/xhyve}/inout.h
  22. +3 −6 {bhyve → include/xhyve}/ioapic.h
  23. +26 −26 {bhyve → include/xhyve}/mem.h
  24. +8 −12 {bhyve → include/xhyve}/mevent.h
  25. +3 −5 bhyve/spinup_ap.h → include/xhyve/mptbl.h
  26. +278 −0 include/xhyve/pci_emul.h
  27. +10 −13 {bhyve → include/xhyve}/pci_irq.h
  28. +23 −22 {bhyve → include/xhyve}/pci_lpc.h
  29. +2 −4 {bhyve → include/xhyve}/rtc.h
  30. +2 −7 {bhyve → include/xhyve}/smbiostbl.h
  31. +64 −0 include/xhyve/support/acpi_hpet.h
  32. +509 −0 include/xhyve/support/apicreg.h
  33. +643 −0 include/xhyve/support/ata.h
  34. +443 −0 include/xhyve/support/atomic.h
  35. +215 −0 include/xhyve/support/bitset.h
  36. +150 −0 include/xhyve/support/cpuset.h
  37. +81 −0 include/xhyve/support/i8253reg.h
  38. +83 −0 include/xhyve/support/i8259.h
  39. +96 −0 include/xhyve/support/linker_set.h
  40. +52 −0 include/xhyve/support/md5.h
  41. +64 −0 include/xhyve/support/misc.h
  42. +194 −0 include/xhyve/support/mptable.h
  43. +242 −0 include/xhyve/support/ns16550.h
  44. +945 −0 include/xhyve/support/pcireg.h
  45. +89 −0 include/xhyve/support/psl.h
  46. +111 −0 include/xhyve/support/rtc.h
  47. +277 −0 include/xhyve/support/segments.h
  48. +845 −0 include/xhyve/support/specialreg.h
  49. +47 −0 include/xhyve/support/timerreg.h
  50. +751 −0 include/xhyve/support/tree.h
  51. +153 −0 include/xhyve/support/uuid.h
  52. +7 −10 {bhyve → include/xhyve}/uart_emul.h
  53. +106 −90 {bhyve → include/xhyve}/virtio.h
  54. +55 −70 { → include/xhyve}/vmm/intel/vmcs.h
  55. +16 −62 { → include/xhyve}/vmm/intel/vmx.h
  56. +94 −0 include/xhyve/vmm/intel/vmx_controls.h
  57. +14 −8 vmm/intel/ept.h → include/xhyve/vmm/intel/vmx_msr.h
  58. +10 −8 { → include/xhyve}/vmm/io/vatpic.h
  59. +10 −7 { → include/xhyve}/vmm/io/vatpit.h
  60. +14 −12 { → include/xhyve}/vmm/io/vhpet.h
  61. +18 −15 { → include/xhyve}/vmm/io/vioapic.h
  62. +5 −4 { → include/xhyve}/vmm/io/vlapic.h
  63. +49 −50 { → include/xhyve}/vmm/io/vlapic_priv.h
  64. +5 −5 { → include/xhyve}/vmm/io/vpmtmr.h
  65. +9 −7 { → include/xhyve}/vmm/io/vrtc.h
  66. +314 −0 include/xhyve/vmm/vmm.h
  67. +113 −0 include/xhyve/vmm/vmm_api.h
  68. +117 −0 include/xhyve/vmm/vmm_callout.h
  69. +322 −0 include/xhyve/vmm/vmm_common.h
  70. +44 −0 include/xhyve/vmm/vmm_host.h
  71. +3 −28 { → include/xhyve/vmm}/vmm_instruction_emul.h
  72. +7 −4 { → include/xhyve}/vmm/vmm_ioport.h
  73. +20 −17 { → include/xhyve}/vmm/vmm_ktr.h
  74. +6 −5 { → include/xhyve}/vmm/vmm_lapic.h
  75. +7 −9 vmm/vmm_util.h → include/xhyve/vmm/vmm_mem.h
  76. +36 −11 { → include/xhyve}/vmm/vmm_stat.h
  77. +3 −4 bhyve/dbgport.h → include/xhyve/vmm/vmm_util.h
  78. +3 −17 { → include/xhyve}/vmm/x86.h
  79. +43 −18 bhyve/bhyverun.h → include/xhyve/xhyve.h
  80. +5 −6 {bhyve → include/xhyve}/xmsr.h
  81. +0 −13 libvmmapi/Makefile
  82. +0 −1,201 libvmmapi/vmmapi.c
  83. +0 −173 libvmmapi/vmmapi.h
  84. +0 −345 libvmmapi/vmmapi_freebsd.c
  85. +39 −34 {bhyve → src}/acpi.c
  86. +16 −25 {bhyve → src}/atkbdc.c
  87. +164 −143 {bhyve → src}/block_if.c
  88. +24 −25 {bhyve → src}/consport.c
  89. +17 −17 {bhyve → src}/dbgport.c
  90. +262 −0 src/firmware/kexec.c
  91. +82 −61 {bhyve → src}/inout.c
  92. +5 −11 {bhyve → src}/ioapic.c
  93. +284 −0 src/md5c.c
  94. +30 −30 {bhyve → src}/mem.c
  95. +25 −30 {bhyve → src}/mevent.c
  96. +16 −5 {bhyve → src}/mevent_test.c
  97. +32 −32 {bhyve → src}/mptbl.c
  98. +226 −170 {bhyve → src}/pci_ahci.c
  99. +183 −189 {bhyve → src}/pci_emul.c
  100. +8 −10 {bhyve → src}/pci_hostbridge.c
  101. +42 −49 {bhyve → src}/pci_irq.c
  102. +34 −36 {bhyve → src}/pci_lpc.c
  103. +15 −18 {bhyve → src}/pci_uart.c
  104. +98 −77 {bhyve → src}/pci_virtio_block.c
  105. +92 −98 {bhyve → src}/pci_virtio_net.c
  106. +35 −34 {bhyve → src}/pci_virtio_rnd.c
  107. +811 −0 src/pci_virtio_vmnet.c
  108. +40 −50 {bhyve → src}/pm.c
  109. +7 −10 {bhyve → src}/post.c
  110. +23 −32 {bhyve → src}/rtc.c
  111. +110 −109 {bhyve → src}/smbiostbl.c
  112. +166 −167 {bhyve → src}/task_switch.c
  113. +23 −21 {bhyve → src}/uart_emul.c
  114. +45 −51 {bhyve → src}/virtio.c
  115. +245 −0 src/vmm/intel/vmcs.c
  116. +2,746 −0 src/vmm/intel/vmx.c
  117. +353 −0 src/vmm/intel/vmx_msr.c
  118. +58 −74 { → src}/vmm/io/vatpic.c
  119. +53 −60 { → src}/vmm/io/vatpit.c
  120. +68 −75 { → src}/vmm/io/vhpet.c
  121. +53 −69 { → src}/vmm/io/vioapic.c
  122. +172 −120 { → src}/vmm/io/vlapic.c
  123. +20 −21 { → src}/vmm/io/vpmtmr.c
  124. +207 −102 { → src}/vmm/io/vrtc.c
  125. +351 −870 { → src}/vmm/vmm.c
  126. +831 −0 src/vmm/vmm_api.c
  127. +379 −0 src/vmm/vmm_callout.c
  128. +60 −0 src/vmm/vmm_host.c
  129. +123 −142 { → src}/vmm/vmm_instruction_emul.c
  130. +22 −22 { → src}/vmm/vmm_ioport.c
  131. +21 −24 { → src}/vmm/vmm_lapic.c
  132. +69 −0 src/vmm/vmm_mem.c
  133. +8 −21 { → src}/vmm/vmm_stat.c
  134. +35 −47 { → src}/vmm/vmm_util.c
  135. +70 −125 { → src}/vmm/x86.c
  136. +923 −0 src/xhyve.c
  137. +104 −0 src/xmsr.c
  138. BIN test/initrd.gz
  139. +9 −0 test/tinycore.txt
  140. BIN test/vmlinuz
  141. +0 −648 vmm.h
  142. +0 −133 vmm/amd/amdv.c
  143. +0 −87 vmm/amd/npt.c
  144. +0 −36 vmm/amd/npt.h
  145. +0 −2,259 vmm/amd/svm.c
  146. +0 −54 vmm/amd/svm.h
  147. +0 −48 vmm/amd/svm_genassym.c
  148. +0 −165 vmm/amd/svm_msr.c
  149. +0 −44 vmm/amd/svm_msr.h
  150. +0 −114 vmm/amd/svm_softc.h
  151. +0 −121 vmm/amd/svm_support.S
  152. +0 −442 vmm/amd/vmcb.c
  153. +0 −334 vmm/amd/vmcb.h
  154. +0 −205 vmm/intel/ept.c
  155. +0 −503 vmm/intel/vmcs.c
  156. +0 −3,416 vmm/intel/vmx.c
  157. +0 −96 vmm/intel/vmx_controls.h
  158. +0 −218 vmm/intel/vmx_cpufunc.h
  159. +0 −88 vmm/intel/vmx_genassym.c
  160. +0 −483 vmm/intel/vmx_msr.c
  161. +0 −70 vmm/intel/vmx_msr.h
  162. +0 −262 vmm/intel/vmx_support.S
  163. +0 −688 vmm/intel/vtd.c
  164. +0 −285 vmm/io/iommu.c
  165. +0 −75 vmm/io/iommu.h
  166. +0 −651 vmm/io/ppt.c
  167. +0 −54 vmm/io/ppt.h
  168. +0 −689 vmm/vmm_dev.c
  169. +0 −161 vmm/vmm_host.c
  170. +0 −83 vmm/vmm_host.h
  171. +0 −154 vmm/vmm_mem.c
  172. +0 −43 vmm/vmm_mem.h
  173. +0 −365 vmm_dev.h
  174. BIN xhyve_logo.png
  175. +15 −0 xhyverun.sh
103 Makefile
@@ -0,0 +1,103 @@
ifeq ($V, 1)
VERBOSE =
else
VERBOSE = @
endif
include config.mk
VMM_SRC := \
src/vmm/x86.c \
src/vmm/vmm.c \
src/vmm/vmm_host.c \
src/vmm/vmm_mem.c \
src/vmm/vmm_lapic.c \
src/vmm/vmm_instruction_emul.c \
src/vmm/vmm_ioport.c \
src/vmm/vmm_callout.c \
src/vmm/vmm_stat.c \
src/vmm/vmm_util.c \
src/vmm/vmm_api.c \
src/vmm/intel/vmx.c \
src/vmm/intel/vmx_msr.c \
src/vmm/intel/vmcs.c \
src/vmm/io/vatpic.c \
src/vmm/io/vatpit.c \
src/vmm/io/vhpet.c \
src/vmm/io/vioapic.c \
src/vmm/io/vlapic.c \
src/vmm/io/vpmtmr.c \
src/vmm/io/vrtc.c
XHYVE_SRC := \
src/acpi.c \
src/atkbdc.c \
src/block_if.c \
src/consport.c \
src/dbgport.c \
src/inout.c \
src/ioapic.c \
src/md5c.c \
src/mem.c \
src/mevent.c \
src/mptbl.c \
src/pci_ahci.c \
src/pci_emul.c \
src/pci_hostbridge.c \
src/pci_irq.c \
src/pci_lpc.c \
src/pci_uart.c \
src/pci_virtio_block.c \
src/pci_virtio_vmnet.c \
src/pci_virtio_rnd.c \
src/pm.c \
src/post.c \
src/rtc.c \
src/smbiostbl.c \
src/task_switch.c \
src/uart_emul.c \
src/xhyve.c \
src/virtio.c \
src/xmsr.c
FIRMWARE_SRC := \
src/firmware/kexec.c
SRC := \
$(VMM_SRC) \
$(XHYVE_SRC) \
$(FIRMWARE_SRC)
OBJ := $(SRC:src/%.c=build/%.o)
DEP := $(OBJ:%.o=%.d)
INC := -Iinclude
TARGET = build/xhyve
all: $(TARGET) | build
.PHONY: clean all
.SUFFIXES:
-include $(DEP)
build:
@mkdir -p build
build/%.o: src/%.c
@echo cc $<
@mkdir -p $(dir $@)
$(VERBOSE) $(ENV) $(CC) $(CFLAGS) $(INC) $(DEF) -MMD -MT $@ -MF build/$*.d -o $@ -c $<
$(TARGET).sym: $(OBJ)
@echo ld $(notdir $@)
$(VERBOSE) $(ENV) $(LD) $(LDFLAGS) -Xlinker $(TARGET).lto.o -o $@ $(OBJ)
@echo dsym $(notdir $(TARGET).dSYM)
$(VERBOSE) $(ENV) $(DSYM) $@ -o $(TARGET).dSYM
$(TARGET): $(TARGET).sym
@echo strip $(notdir $@)
$(VERBOSE) $(ENV) $(STRIP) $(TARGET).sym -o $@
clean:
@rm -rf build
184 README.md
@@ -0,0 +1,184 @@
# xhyve
![](./xhyve_logo.png)
<!-- https://thenounproject.com/term/squirrel/57718/ -->
About
-----
The *xhyve hypervisor* is a port of [bhyve](http://www.bhyve.org) to OS X. It is built on top of Hypervisor.framework in OS X 10.10 Yosemite and higher, runs entirely in userspace, and has no other dependencies. It can run vanilla Linux distributions and may gain support for other guest operating systems in the future.
License: BSD
Introduction: [http://www.pagetable.com/?p=831](http://www.pagetable.com/?p=831)
Requirements
------------
* OS X 10.10 Yosemite or later
* A 2010 or later Mac
Building
--------
$ make
The resulting binary will be in build/xhyve
Usage
-----
$ xhyve -h
What is bhyve?
--------------
bhyve is the FreeBSD hypervisor, roughly analogous to KVM + QEMU on Linux. It has a focus on simplicity and being legacy free.
It exposes the following peripherals to virtual machines:
- Local x(2)APIC
- IO-APIC
- 8259A PIC
- 8253/8254 PIT
- HPET
- PM Timer
- RTC
- PCI
- host bridge
- passthrough
- UART
- AHCI (i.e. HDD and CD)
- VirtIO block device
- VirtIO networking
- VirtIO RNG
Notably absent are sound, USB, HID and any kind of graphics support. With a focus on server virtualization this is not strictly a requirement. bhyve may gain desktop virtualization capabilities in the future but this doesn't seem to be a priority.
Unlike QEMU, byhve also currently lacks any kind of guest-side firmware (QEMU uses the GPL3 [SeaBIOS](http://www.seabios.org)), but aims to provide a compatible [OVMF EFI](http://www.linux-kvm.org/page/OVMF) in the near future. It does however provide ACPI, SMBIOS and MP Tables.
bhyve architecture
------------------
Linux
I/O VM control FreeBSD NetBSD
OpenBSD
| A | A | |
V | V | V V
+-------------++-------------++-------------++-------------+
| || || || |
| bhyve || bhyvectl || bhyveload || grub2-bhyve |
| || || || |
| || || || |
+-------------++-------------++-------------++-------------+
+----------------------------------------------------------+
| libvmmapi |
+----------------------------------------------------------+
A
| user
------------------------------┼------------------------------
| ioctl FreeBSD kernel
V
+----------------------------+
| VMX/SVM host |
| VMX/SVM guest |
| VMX/SVM nested paging |
| Timers |
| Interrupts |
+----------------------------+
vmm.ko
**vmm.ko**
The bhyve FreeBSD kernel module. Manages VM and vCPU objects, the guest physical address space and handles guest interaction with PIC, PIT, HPET, PM Timer, x(2)APIC and I/O-APIC. Contains a minimal x86 emulator to decode guest MMIO. Executes the two innermost vCPU runloops (VMX/SVM and interrupts/timers/paging). Has backends for Intel VMX and AMD SVM. Provides an ioctl and mmap API to userspace.
**libvmmapi**
Thin abstraction layer between the vmm.ko ioctl interface and the userspace C API.
**bhyve**
The userspace bhyve component (kind of a very light-weight QEMU) that executes virtual machines. Runs the guest I/O vCPU runloops. Manages ACPI, PCI and all non in-kernel devices. Interacts with vmm.ko through libvmmapi.
**bhyvectl**
Somewhat superfluous utility to introspect and manage the life cycle of virtual machines. Virtual machines and vCPUs can exist as kernel objects independently of a bhyve host process. Typically used to delete VM objects after use. Odd architectural choice.
**bhyveload**
Userspace port of the FreeBSD bootloader. Since bhyve still lacks a firmware this is a cumbersome workaround to bootstrap a guest operating system. It creates a VM object, loads the FreeBSD kernel into guest memory, sets up the initial vCPU state and then exits. Only then a VM can be executed by bhyve.
**grub2-bhyve**
Performs the same function as bhyveload but is a userspace port of [GRUB2](http://github.com/grehan-freebsd/grub2-bhyve). It is used to bootstrap guest operating systems other than FreeBSD, i.e. Linux, OpenBSD and NetBSD.
Support for Windows guests is work in progress and dependent on the EFI port.
xhyve architecture
------------------
+----------------------------------------------------------+
| xhyve |
| |
| I/O |
| |
| |
| |
|+--------------------------------------------------------+|
|| vmm VMX guest ||
|| Timers ||
|| Interrupts ||
|+--------------------------------------------------------+|
+----------------------------------------------------------+
+----------------------------------------------------------+
| Hypervisor.framework |
+----------------------------------------------------------+
A
| user
------------------------------┼------------------------------
|syscall xnu kernel
V
VMX host
VMX nested paging
xhyve shares most of the code with bhyve but is architecturally very different. Hypervisor.framework provides an interface to the VMX VMCS guest state and a safe subset of the VMCS control fields, thus making userspace hypervisors without any additional kernel extensions possible. The VMX host state and all aspects of nested paging are handled by the OS X kernel, you can manage the guest physical address space simply through mapping of regions of your own address space.
*xhyve* is equivalent to the *bhyve* process but gains a subset of a userspace port of the vmm kernel module. SVM, PCI passthrough and the VMX host and EPT aspects are dropped. The vmm component provides a libvmmapi compatible interface to xhyve. Hypervisor.framework seems to enforce a strict 1:1 relationship between a host process/VM and host thread/vCPU, that means VMs and vCPUs can only be interacted with by the processes and threads that created them. Therefore, unlike bhyve, xhyve needs to adhere to a single process model. Multiple virtual machines can be created by launching multiple instances of xhyve. xhyve retains most of the bhyve command line interface.
*bhyvectl*, *bhyveload* and *grub2-bhyve* are incompatible with a single process model and are dropped. As a stop-gap solution until we have a proper firmware xhyve supports the Linux [kexec protocol](http://www.kernel.org/doc/Documentation/x86/boot.txt), a very simple and straightforward way to bootstrap a Linux kernel. It takes a bzImage and optionally initrd image and kernel parameter string as input.
TODO
----
- vmm:
- enable APIC access page to speed up APIC emulation
- enable x2APIC MSRs (even faster)
- vmm_callout:
- is a quick'n'dirty implementation of the FreeBSD kernel callout mechanism
- seems to be racy
- fix races or perhaps replace with something better
- use per vCPU timer event thread (performance)?
- some 32-bit guests are broken (support PAE paging in VMCS)
- PCID guest support (performance)
- block_if:
- OS X does not support preadv/pwritev, we need to serialize reads and writes for the time being until we find a better solution.
- support block devices other than plain files
- virtio_net:
- make it not require root
- unify TAP and vmnet backends
- performance: send/receive more than a single packet at a time
- ACPI tables don't work
- bhyve creates ASL on the fly and then calls out to an ASL compiler (iasl) on
every VM boot to create the DSDT:
- remove dependency on iasl by creating AML bytecode directly
- shouldn't be to hard since we we are only interested in a very small
subset of ASL
- virtio_rnd:
- is untested
- remove explicit state transitions:
- since only the owning task/thread can modify the VM/vCPUs a lot of the synchronization might be unnecessary
- performance, performance and performance
- remove vestigial code, cleanup

This file was deleted.

Oops, something went wrong.
Oops, something went wrong.

0 comments on commit 6322d86

Please sign in to comment.