Running Alpine Linux on xhyve

[philnalwalker]

Trying to run Alpine Linux 3.6.2 on xhyve but getting a "failed" error after installing OS and booting. Minikube/Docker clearly get past this issue. How?

Details follow:

MacOS Sierra + xhyve (tried both v0.2.0 and HEAD)

install.sh

KERNEL="mnt/boot/vmlinuz-virthardened"
INITRD="mnt/boot/initramfs-virthardened"
CMDLINE="alpine_dev=cdrom:iso9660 modules=af_socket,loop,squashfs,sd-mod,usb-storage,sr-mod,earlyprintk=serial console=ttyS0"
MEM="-m 8G"
SMP="-c 4"
NET="-s 2:0,virtio-net"
IMG_CD="-s 3,ahci-cd,alpine-virt-3.6.2-x86_64.iso"
IMG_HDD="-s 4,virtio-blk,hdd.img"
PCI_DEV="-s 0:0,hostbridge -s 31,lpc"
LPC_DEV="-l com1,stdio"
#ACPI="-A"
#UUID="-U FBD1B520-3130-488A-A4E4-AC490BDAC231"
./xhyve $ACPI $UUID $SMP $MEM $PCI_DEV $LPC_DEV $NET $IMG_CD $IMG_HDD -f kexec,$KERNEL,$INITRD,"$CMDLINE" -w

run.sh (tried adding root=/dev/vda1, no help)

KERNEL="vmlinuz-virthardened"
INITRD="initramfs-virthardened"
CMDLINE="earlyprintk=serial console=ttyS0 modules=af_socket,loop,squashfs,sd-mod,usb-storage,sr-mod,ext4"
MEM="-m 8G"
#SMP="-c 4"
NET="-s 2:0,virtio-net"
#IMG_CD="-s 3,ahci-cd,alpine-3.2.3-x86_64.iso"
IMG_HDD="-s 4,virtio-blk,hdd.img"
PCI_DEV="-s 0:0,hostbridge -s 31,lpc"
LPC_DEV="-l com1,stdio"
xhyve $ACPI $SMP $MEM $PCI_DEV $LPC_DEV $NET $IMG_HDD -f kexec,$KERNEL,$INITRD,"$CMDLINE" -w

Kernel Output

[    0.000000] Linux version 4.9.32-0-virthardened (buildozer@build-3-6-x86_64) (gcc version 6.3.0 (Alpine 6.3.0) ) #1-Alpine SMP Fri Jun 16 12:39:17 GMT 2017
[    0.000000] Command line: earlyprintk=serial console=ttyS0 modules=af_socket,loop,squashfs,sd-mod,usb-storage,sr-mod,ext4
[    0.000000] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[    0.000000] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[    0.000000] x86/fpu: xstate_offset[2]:  576, xstate_sizes[2]:  256
[    0.000000] x86/fpu: Enabled xstate features 0x7, context size is 832 bytes, using 'standard' format.
[    0.000000] x86/fpu: Using 'eager' FPU context switches.
[    0.000000] e820: BIOS-provided physical RAM map:
[    0.000000] BIOS-e820: [mem 0x0000000000000000-0x000000000009fbff] usable
[    0.000000] BIOS-e820: [mem 0x0000000000100000-0x00000000bfffffff] usable
[    0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000023fffffff] usable
[    0.000000] NX (Execute Disable) protection: active
[    0.000000] SMBIOS 2.6 present.
[    0.000000] e820: last_pfn = 0x240000 max_arch_pfn = 0x400000000
[    0.000000] MTRR: Disabled
[    0.000000] x86/PAT: MTRRs disabled, skipping PAT initialization too.
[    0.000000] x86/PAT: Configuration [0-7]: WB  WT  UC- UC  WB  WT  UC- UC
[    0.000000] CPU MTRRs all blank - virtualized system.
[    0.000000] e820: last_pfn = 0xc0000 max_arch_pfn = 0x400000000
[    0.000000] Using GB pages for direct mapping
[    0.000000] RAMDISK: [mem 0x01e46000-0x02367fff]
[    0.000000] ACPI: Early table checksum verification disabled
[    0.000000] ACPI BIOS Error (bug): A valid RSDP was not found (20160831/tbxfroot-244)
[    0.000000] Zone ranges:
[    0.000000]   DMA      [mem 0x0000000000001000-0x0000000000ffffff]
[    0.000000]   DMA32    [mem 0x0000000001000000-0x00000000ffffffff]
[    0.000000]   Normal   [mem 0x0000000100000000-0x000000023fffffff]
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000000001000-0x000000000009efff]
[    0.000000]   node   0: [mem 0x0000000000100000-0x00000000bfffffff]
[    0.000000]   node   0: [mem 0x0000000100000000-0x000000023fffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000000001000-0x000000023fffffff]
[    0.000000] smpboot: Boot CPU (id 0) not listed by BIOS
[    0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
[    0.000000] e820: [mem 0xc0000000-0xffffffff] available for PCI devices
[    0.000000] Booting paravirtualized kernel on bare hardware
[    0.000000] clocksource: refined-jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.000000] setup_percpu: NR_CPUS:32 nr_cpumask_bits:32 nr_cpu_ids:1 nr_node_ids:1
[    0.000000] percpu: Embedded 32 pages/cpu @ffff88023fc00000 s91352 r8192 d31528 u2097152
[    0.000000] Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 2064265
[    0.000000] Kernel command line: earlyprintk=serial console=ttyS0 modules=af_socket,loop,squashfs,sd-mod,usb-storage,sr-mod,ext4
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Dentry cache hash table entries: 1048576 (order: 11, 8388608 bytes)
[    0.000000] Inode-cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.000000] Memory: 8162356K/8388216K available (6018K kernel code, 730K rwdata, 1828K rodata, 1144K init, 680K bss, 225860K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000] 	Build-time adjustment of leaf fanout to 64.
[    0.000000] 	RCU restricting CPUs from NR_CPUS=32 to nr_cpu_ids=1.
[    0.000000] RCU: Adjusting geometry for rcu_fanout_leaf=64, nr_cpu_ids=1
[    0.000000] NR_IRQS:4352 nr_irqs:32 16
[    0.000000] Console: colour dummy device 80x25
[    0.000000] console [ttyS0] enabled
[    0.000000] tsc: Detected 2700.000 MHz processor
[    0.000047] Calibrating delay loop (skipped), value calculated using timer frequency.. 5424.00 BogoMIPS (lpj=27120000)
[    0.000776] pid_max: default: 32768 minimum: 501
[    0.001083] Mount-cache hash table entries: 16384 (order: 5, 131072 bytes)
[    0.001479] Mountpoint-cache hash table entries: 16384 (order: 5, 131072 bytes)
[    0.002515] CPU: Physical Processor ID: 0
[    0.002748] PAX: PCID detected
[    0.002917] Last level iTLB entries: 4KB 64, 2MB 8, 4MB 8
[    0.003196] Last level dTLB entries: 4KB 64, 2MB 0, 4MB 0, 1GB 4
[    0.027552] Freeing SMP alternatives memory: 20K (ffffffff81a10000 - ffffffff81a15000)
[    0.043651] smpboot: Max logical packages: 1
[    0.043913] smpboot: SMP motherboard not detected
[    0.044181] smpboot: SMP disabled
[    0.282258] Performance Events: unsupported p6 CPU model 94 no PMU driver, software events only.
[    0.283217] NMI watchdog: disabled (cpu0): hardware events not enabled
[    0.283592] NMI watchdog: Shutting down hard lockup detector on all cpus
[    0.283980] x86: Booted up 1 node, 1 CPUs
[    0.284206] ----------------
[    0.284377] | NMI testsuite:
[    0.284546] --------------------
[    0.284736]   remote IPI:  ok  |
[    0.284940]    local IPI:  ok  |
[    0.285165] --------------------
[    0.285356] Good, all   2 testcases passed! |
[    0.285602] ---------------------------------
[    0.285850] smpboot: Total of 1 processors activated (5424.00 BogoMIPS)
[    0.286554] devtmpfs: initialized
[    0.286835] x86/mm: Memory block size: 128MB
[    0.288891] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
[    0.289528] futex hash table entries: 256 (order: 2, 16384 bytes)
[    0.290011] NET: Registered protocol family 16
[    0.290426] cpuidle: using governor ladder
[    0.290651] cpuidle: using governor menu
[    0.290922] ACPI BIOS Error (bug): A valid RSDP was not found (20160831/tbxfroot-244)
[    0.291680] PCI: Using configuration type 1 for base access
[    0.298072] HugeTLB registered 1 GB page size, pre-allocated 0 pages
[    0.298477] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    0.299049] ACPI: Interpreter disabled.
[    0.299396] SCSI subsystem initialized
[    0.299834] PCI: Probing PCI hardware
[    0.300049] PCI host bridge to bus 0000:00
[    0.300266] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
[    0.300741] pci_bus 0000:00: root bus resource [mem 0x00000000-0x7fffffffff]
[    0.301171] pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]
[    0.303383] clocksource: Switched to clocksource refined-jiffies
[    0.303791] VFS: Disk quotas dquot_6.6.0
[    0.304071] VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.304520] pnp: PnP ACPI: disabled
[    0.308103] pci 0000:00:02.0: BAR 6: assigned [mem 0xc0004000-0xc00047ff pref]
[    0.308534] pci 0000:00:04.0: BAR 6: assigned [mem 0xc0004800-0xc0004fff pref]
[    0.309088] pci 0000:00:1f.0: BAR 6: assigned [mem 0xc0005000-0xc00057ff pref]
[    0.309548] NET: Registered protocol family 2
[    0.309934] TCP established hash table entries: 65536 (order: 7, 524288 bytes)
[    0.310813] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    0.312060] TCP: Hash tables configured (established 65536 bind 65536)
[    0.312497] UDP hash table entries: 4096 (order: 5, 131072 bytes)
[    0.312914] UDP-Lite hash table entries: 4096 (order: 5, 131072 bytes)
[    0.313391] NET: Registered protocol family 1
[    0.313642] pci 0000:00:1f.0: Activating ISA DMA hang workarounds
[    0.314417] Unpacking initramfs...
[    0.400522] Freeing initrd memory: 5256K (ffff880001e46000 - ffff880002368000)
[    0.401038] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    0.401446] software IO TLB [mem 0xbc000000-0xc0000000] (64MB) mapped at [ffff8800bc000000-ffff8800bfffffff]
rdmsr to register 0x64e on vcpu 0
                                 rdmsr to register 0x34 on vcpu 0
                                                                 [    0.402265] platform rtc_cmos: registered platform RTC device (no PNP device found)
[    0.403260] workingset: timestamp_bits=46 max_order=21 bucket_order=0
[    0.403885] Key type asymmetric registered
[    0.404171] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
[    0.404761] io scheduler noop registered
[    0.405034] io scheduler deadline registered (default)
[    0.405405] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[    0.405774] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
[    0.406349] ERST DBG: ERST support is disabled.
[    0.406607] xenfs: not registering filesystem on non-xen platform
[    0.407047] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.428186] serial8250: ttyS0 at I/O 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[    0.450200] serial8250: ttyS1 at I/O 0x2f8 (irq = 3, base_baud = 115200) is a 16550A
[    0.450976] VMware PVSCSI driver - version 1.0.7.0-k
[    0.451306] i8042: PNP: No PS/2 controller found. Probing ports directly.
[    0.974498] i8042: Can't read CTR while initializing i8042
[    0.974894] i8042: probe of i8042 failed with error -5
[    0.975712] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
[    0.976298] rtc_cmos rtc_cmos: alarms up to one day, 114 bytes nvram
[    0.976927] gre: GRE over IPv4 demultiplexor driver
[    0.977289] Key type dns_resolver registered
[    0.977666] registered taskstats version 1
[    0.978117] rtc_cmos rtc_cmos: setting system clock to 2017-09-22 22:59:04 UTC (1506121144)
[    0.979286] Freeing unused kernel memory: 1144K (ffffffff818f2000 - ffffffff81a10000)
[    0.979757] Write protecting the kernel read-only data: 8192k
[    0.980179] Freeing unused kernel memory: 76K (ffff8800015ed000 - ffff880001600000)
[    0.981132] Freeing unused kernel memory: 220K (ffff8800017c9000 - ffff880001800000)
Alpine Init 3.1.0-r3
 * Loading boot drivers: [    0.988843] loop: module loaded
[    0.991260] usbcore: registered new interface driver usbfs
[    0.991639] usbcore: registered new interface driver hub
[    0.991942] usbcore: registered new device driver usb
[    0.993032] usbcore: registered new interface driver usb-storage
ok.
 * Mounting boot media: [    1.011719] virtio-pci 0000:00:02.0: virtio_pci: leaving for legacy driver
[    1.012252] virtio-pci 0000:00:04.0: virtio_pci: leaving for legacy driver
[    1.020254]  vda: vda1 vda2 vda3
[    1.552578] EXT4-fs (vda1): mounted filesystem with ordered data mode. Opts: (null)
[    1.564974] EXT4-fs (vda3): mounted filesystem with ordered data mode. Opts: (null)
[    1.571517] random: fast init done
[    2.016920] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x2717868ea45, max_idle_ns: 440795316085 ns
[    2.018193] clocksource: Switched to clocksource tsc
failed.
initramfs emergency recovery shell launched. Type 'exit' to continue boot
sh: can't access tty; job control turned off
/ #

[philnalwalker]

Managed to get it working.

install.sh:

KERNEL="mnt/boot/vmlinuz-hardened"
INITRD="mnt/boot/initramfs-hardened"
CMDLINE="alpine_dev=cdrom:iso9660 modules=af_socket,loop,squashfs,sd-mod,usb-storage,sr-mod,earlyprintk=serial console=ttyS0"
MEM="-m 8G"
SMP="-c 4"
NET="-s 2:0,virtio-net"
IMG_CD="-s 3,ahci-cd,alpine-standard-3.6.2-x86_64.iso"
IMG_HDD="-s 4,virtio-blk,hdd.img"
PCI_DEV="-s 0:0,hostbridge -s 31,lpc"
LPC_DEV="-l com1,stdio"
ACPI="-A"
UUID="-U deadbeef-dead-beef-dead-beef00000000"
xhyve $ACPI $UUID $SMP $MEM $PCI_DEV $LPC_DEV $NET $IMG_CD $IMG_HDD -f kexec,$KERNEL,$INITRD,"$CMDLINE" -w

run.sh:

KERNEL="mnt/boot/vmlinuz-hardened"
INITRD="mnt/boot/initramfs-hardened"
CMDLINE="earlyprintk=serial console=ttyS0 modules=virtio_blk,af_socket,loop,squashfs,sd-mod,usb-storage,sr-mod,ext4 debug_init root=/dev/vda3"
MEM="-m 8G"
#SMP="-c 4"
NET="-s 2:0,virtio-net"
IMG_HDD="-s 4,virtio-blk,hdd.img"
PCI_DEV="-s 0:0,hostbridge -s 31,lpc"
LPC_DEV="-l com1,stdio"
ACPI="-A"
UUID="-U deadbeef-dead-beef-dead-beef00000000"
xhyve $UUID $ACPI $SMP $MEM $PCI_DEV $LPC_DEV $NET $IMG_HDD -f kexec,$KERNEL,$INITRD,"$CMDLINE" -w