Skip to content

KVM RISCV64 on Spike

Anup Patel edited this page Apr 13, 2024 · 10 revisions

Currently, we can boot RISC-V 64bit SMP Guest using KVM RISC-V on Spike.

To achieve this, we need following components:

  1. Spike Simulator
  2. OpenSBI Firmware
  3. Common Host & Guest Kernel
  4. KVM RISC-V module
  5. KVMTOOL
  6. Host RootFS with KVM RISC-V module, KVMTOOL and Guest Kernel

The below sub-sections provide detailed steps to build and run RISC-V KVM on QEMU. All components except QEMU require a RISC-V 64bit cross-compiler so for we use Linux multilib toolchain from RISC-V GNU Compiler Toolchain project (Refer, https://github.com/riscv/riscv-gnu-toolchain). If you have a different RISC-V 64bit cross-compiler then set CROSS_COMPILE environment variable appropriately in steps below.

1. Build Spike Simulator

git clone https://github.com/riscv/riscv-isa-sim.git
cd riscv-isa-sim
./configure
make
cd ..

The above commands will create ./riscv-isa-sim/spike which will be our Spike simulator.

2. Build OpenSBI Firmware

git clone https://github.com/riscv/opensbi.git
cd opensbi
export CROSS_COMPILE=riscv64-unknown-linux-gnu-
make PLATFORM=generic
cd ..

The above commands will create opensbi/build/platform/generic/firmware/fw_jump.elf which will be our M-mode runtime firmware.

3. Build Common Host & Guest Linux Kernel Image

We can use same RISC-V 64bit Linux kernel as Guest and Host kernel so no need to compile them separately.

git clone https://github.com/kvm-riscv/linux.git
export ARCH=riscv
export CROSS_COMPILE=riscv64-unknown-linux-gnu-
mkdir build-riscv64
make -C linux O=`pwd`/build-riscv64 defconfig
make -C linux O=`pwd`/build-riscv64

The above commands will create:

  1. build-riscv64/arch/riscv/boot/Image which will be our Guest and Host kernel
  2. build-riscv64/arch/riscv/kvm/kvm.ko which will be our KVM RISC-V module

4. Add libfdt library to CROSS_COMPILE SYSROOT directory

We need libfdt library in the cross-compile toolchain for compiling KVMTOOL RISC-V (described in next step). The libfdt library is generally not available in the cross-compile toolchain so we need to explicitly compile libfdt from DTC project and add it to CROSS_COMPILE SYSROOT directory.

git clone git://git.kernel.org/pub/scm/utils/dtc/dtc.git
cd dtc
export ARCH=riscv
export CROSS_COMPILE=riscv64-unknown-linux-gnu-
export CC="${CROSS_COMPILE}gcc -mabi=lp64d -march=rv64gc"
TRIPLET=$($CC -dumpmachine)
SYSROOT=$($CC -print-sysroot)
make libfdt
make NO_PYTHON=1 NO_YAML=1 DESTDIR=$SYSROOT PREFIX=/usr LIBDIR=/usr/lib64/lp64d install-lib install-includes
cd ..

The above commands will install cross-compiled libfdt library at $SYSROOT/usr/lib64/lp64d directory of cross-compile toolchain.

5. Build KVMTOOL

git clone https://git.kernel.org/pub/scm/linux/kernel/git/will/kvmtool.git
export ARCH=riscv
export CROSS_COMPILE=riscv64-unknown-linux-gnu-
cd kvmtool
make lkvm-static
${CROSS_COMPILE}strip lkvm-static
cd ..

The above commands will create kvmtool/lkvm-static which will be our user-space tool for KVM RISC-V.

6. Build Host RootFS containing KVM RISC-V module, KVMTOOL and Guest Linux

export ARCH=riscv
export CROSS_COMPILE=riscv64-unknown-linux-gnu-
git clone https://github.com/kvm-riscv/howto.git
wget https://busybox.net/downloads/busybox-1.33.1.tar.bz2
tar -C . -xvf ./busybox-1.33.1.tar.bz2
mv ./busybox-1.33.1 ./busybox-1.33.1-kvm-riscv64
cp -f ./howto/configs/busybox-1.33.1_defconfig busybox-1.33.1-kvm-riscv64/.config
make -C busybox-1.33.1-kvm-riscv64 oldconfig
make -C busybox-1.33.1-kvm-riscv64 install
mkdir -p busybox-1.33.1-kvm-riscv64/_install/etc/init.d
mkdir -p busybox-1.33.1-kvm-riscv64/_install/dev
mkdir -p busybox-1.33.1-kvm-riscv64/_install/proc
mkdir -p busybox-1.33.1-kvm-riscv64/_install/sys
mkdir -p busybox-1.33.1-kvm-riscv64/_install/apps
ln -sf /sbin/init busybox-1.33.1-kvm-riscv64/_install/init
cp -f ./howto/configs/busybox/fstab busybox-1.33.1-kvm-riscv64/_install/etc/fstab
cp -f ./howto/configs/busybox/rcS busybox-1.33.1-kvm-riscv64/_install/etc/init.d/rcS
cp -f ./howto/configs/busybox/motd busybox-1.33.1-kvm-riscv64/_install/etc/motd
cp -f ./kvmtool/lkvm-static busybox-1.33.1-kvm-riscv64/_install/apps
cp -f ./build-riscv64/arch/riscv/boot/Image busybox-1.33.1-kvm-riscv64/_install/apps
cp -f ./build-riscv64/arch/riscv/kvm/kvm.ko busybox-1.33.1-kvm-riscv64/_install/apps
cd busybox-1.33.1-kvm-riscv64/_install; find ./ | cpio -o -H newc > ../../rootfs_kvm_riscv64.img; cd -

The above commands will create rootfs_kvm_riscv64.img which will be our Host RootFS containing KVMTOOL and Guest Linux.

7. Run RISC-V KVM on Spike

Run Host Linux with Host RootFS on Spike

./riscv-isa-sim/spike -m512 --isa rv64gch_zicntr_zihpm --kernel ./build-riscv64/arch/riscv/boot/Image --initrd ./rootfs_kvm_riscv64.img opensbi/build/platform/generic/firmware/fw_jump.elf

Insert the KVM RISC-V module

insmod apps/kvm.ko

Run Guest Linux using KVMTOOL after the KVM RISC-V module is inserted

./apps/lkvm-static run -m 128 -c2 --console serial -p "console=ttyS0 earlycon" -k ./apps/Image --debug

8. Reference Bootlog

anup@anup-ubuntu64:~/Work/riscv-test/kvm-riscv$ ./riscv-isa-sim/spike -m512 --isa rv64gch --kernel ./build-riscv64/arch/riscv/boot/Image --initrd ./rootfs_kvm_riscv64.img opensbi/build/platform/generic/firmware/fw_jump.elf

OpenSBI v0.8-6-gec1abf6
   ____                    _____ ____ _____
  / __ \                  / ____|  _ \_   _|
 | |  | |_ __   ___ _ __ | (___ | |_) || |
 | |  | | '_ \ / _ \ '_ \ \___ \|  _ < | |
 | |__| | |_) |  __/ | | |____) | |_) || |_
  \____/| .__/ \___|_| |_|_____/|____/_____|
        | |
        |_|

Platform Name       : ucbbar,spike-bare
Platform Features   : timer,mfdeleg
Platform HART Count : 1
Boot HART ID        : 0
Boot HART ISA       : rv64imafdcsuh
BOOT HART Features  : pmp,scounteren,mcounteren
BOOT HART PMP Count : 16
Firmware Base       : 0x80000000
Firmware Size       : 92 KB
Runtime SBI Version : 0.2

MIDELEG : 0x0000000000001666
MEDELEG : 0x0000000000f0b509
PMP0    : 0x0000000080000000-0x000000008001ffff (A)
PMP1    : 0x0000000000000000-0x01ffffffffffffff (A,R,W,X)
[    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80200000
[    0.000000] Linux version 5.8.0-rc4-00022-g906eafa224e2 (anup@anup-ubuntu64-vm) (riscv64-unknown-linux-gnu-gcc (GCC) 9.2.0, GNU ld (GNU Binutils) 2.34) #1 SMP Fri Jul 24 19:10:43 IST 2020
[    0.000000] earlycon: sbi0 at I/O port 0x0 (options '')
[    0.000000] printk: bootconsole [sbi0] enabled
[    0.000000] Initial ramdisk at: 0x(____ptrval____) (29944320 bytes)
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000080200000-0x000000009fffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080200000-0x000000009fffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080200000-0x000000009fffffff]
[    0.000000] software IO TLB: mapped [mem 0x99c6e000-0x9dc6e000] (64MB)
[    0.000000] SBI specification v0.2 detected
[    0.000000] SBI implementation ID=0x1 Version=0x8
[    0.000000] SBI v0.2 TIME extension detected
[    0.000000] SBI v0.2 IPI extension detected
[    0.000000] SBI v0.2 RFENCE extension detected
[    0.000000] SBI v0.2 HSM extension detected
[    0.000000] riscv: ISA extensions acdfhim
[    0.000000] riscv: ELF capabilities acdfim
[    0.000000] percpu: Embedded 17 pages/cpu s31704 r8192 d29736 u69632
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 128775
[    0.000000] Kernel command line: root=/dev/ram console=hvc0 earlycon=sbi
[    0.000000] Dentry cache hash table entries: 65536 (order: 7, 524288 bytes, linear)
[    0.000000] Inode-cache hash table entries: 32768 (order: 6, 262144 bytes, linear)
[    0.000000] Sorting __ex_table...
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 401948K/522240K available (6456K kernel code, 4177K rwdata, 4096K rodata, 226K init, 323K bss, 120292K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=1.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=1
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] riscv-intc: 64 local interrupts mapped
[    0.000000] riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [0]
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x24e6a1710, max_idle_ns: 440795202120 ns
[    0.000005] sched_clock: 64 bits at 10MHz, resolution 100ns, wraps every 4398046511100ns
[    0.000540] Console: colour dummy device 80x25
[    0.000805] printk: console [hvc0] enabled
[    0.000805] printk: console [hvc0] enabled
[    0.001290] printk: bootconsole [sbi0] disabled
[    0.001290] printk: bootconsole [sbi0] disabled
[    0.001835] Calibrating delay loop (skipped), value calculated using timer frequency.. 20.00 BogoMIPS (lpj=40000)
[    0.002440] pid_max: default: 32768 minimum: 301
[    0.002785] Mount-cache hash table entries: 1024 (order: 1, 8192 bytes, linear)
[    0.003220] Mountpoint-cache hash table entries: 1024 (order: 1, 8192 bytes, linear)
[    0.004530] rcu: Hierarchical SRCU implementation.
[    0.005120] smp: Bringing up secondary CPUs ...
[    0.005390] smp: Brought up 1 node, 1 CPU
[    0.005785] devtmpfs: initialized
[    0.006270] random: get_random_u32 called from bucket_table_alloc.isra.0+0x4e/0x154 with crng_init=0
[    0.006440] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.007540] futex hash table entries: 256 (order: 2, 16384 bytes, linear)
[    0.008100] NET: Registered protocol family 16
[    0.014575] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.015595] vgaarb: loaded
[    0.015900] SCSI subsystem initialized
[    0.016310] usbcore: registered new interface driver usbfs
[    0.016685] usbcore: registered new interface driver hub
[    0.017015] usbcore: registered new device driver usb
[    0.017710] clocksource: Switched to clocksource riscv_clocksource
[    0.021500] NET: Registered protocol family 2
[    0.022100] tcp_listen_portaddr_hash hash table entries: 256 (order: 0, 4096 bytes, linear)
[    0.022595] TCP established hash table entries: 4096 (order: 3, 32768 bytes, linear)
[    0.023100] TCP bind hash table entries: 4096 (order: 4, 65536 bytes, linear)
[    0.023570] TCP: Hash tables configured (established 4096 bind 4096)
[    0.023970] UDP hash table entries: 256 (order: 1, 8192 bytes, linear)
[    0.024365] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes, linear)
[    0.024825] NET: Registered protocol family 1
[    0.025375] RPC: Registered named UNIX socket transport module.
[    0.025775] RPC: Registered udp transport module.
[    0.026055] RPC: Registered tcp transport module.
[    0.026335] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.026715] PCI: CLS 0 bytes, default 64
[    0.027030] Unpacking initramfs...
[    0.076440] Freeing initrd memory: 29240K
[    0.076735] kvm [1]: hypervisor extension available
[    0.077025] kvm [1]: host has 0 VMID bits
[    0.077605] workingset: timestamp_bits=62 max_order=17 bucket_order=0
[    0.082610] NFS: Registering the id_resolver key type
[    0.082915] Key type id_resolver registered
[    0.083165] Key type id_legacy registered
[    0.083410] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.083870] 9p: Installing v9fs 9p2000 file system support
[    0.084330] NET: Registered protocol family 38
[    0.084605] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
[    0.085040] io scheduler mq-deadline registered
[    0.085310] io scheduler kyber registered
[    0.105845] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    0.106725] [drm] radeon kernel modesetting enabled.
[    0.110175] loop: module loaded
[    0.110745] libphy: Fixed MDIO Bus: probed
[    0.111330] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    0.111675] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    0.112065] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.112450] ehci-pci: EHCI PCI platform driver
[    0.112730] ehci-platform: EHCI generic platform driver
[    0.113060] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.113425] ohci-pci: OHCI PCI platform driver
[    0.113750] ohci-platform: OHCI generic platform driver
[    0.114205] usbcore: registered new interface driver uas
[    0.114540] usbcore: registered new interface driver usb-storage
[    0.114960] mousedev: PS/2 mouse device common for all mice
[    0.115505] usbcore: registered new interface driver usbhid
[    0.115835] usbhid: USB HID core driver
[    0.116460] NET: Registered protocol family 10
[    0.117030] Segment Routing with IPv6
[    0.117280] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    0.117940] NET: Registered protocol family 17
[    0.118300] 9pnet: Installing 9P2000 support
[    0.118575] Key type dns_resolver registered
[    0.119220] Freeing unused kernel memory: 224K
[    0.129795] Run /init as init process
           _  _
          | ||_|
          | | _ ____  _   _  _  _ 
          | || |  _ \| | | |\ \/ /
          | || | | | | |_| |/    \
          |_||_|_| |_|\____|\_/\_/

               Busybox Rootfs

Please press Enter to activate this console. 

/ # ^[[47;5R

/ # ^[[47;5Rcat /proc/cpuinfo
cat /proc/cpuinfo
processor	: 0
hart		: 0
isa		: rv64imafdch
mmu		: sv48

/ # ^[[47;5Rcat /proc/interrupts
cat /proc/interrupts
           CPU0       
  5:         82  RISC-V INTC   5  riscv-timer
IPI0:         0  Rescheduling interrupts
IPI1:         0  Function call interrupts
IPI2:         0  CPU stop interrupts
/ # ^[[47;5R

/ # ^[[47;5R./apps/lkvm-static run -m 128 -c2 --console serial -p "console=ttyS0 earlycon=sbi" -k ./apps/Image --debug
./apps/lkvm-static run -m 128 -c2 --console serial -p "console=ttyS0 earlyco
n=sbi" -k ./apps/Image --debug
  # lkvm run -k ./apps/Image -m 128 -c 2 --name guest-46
  Info: (riscv/kvm.c) kvm__arch_load_kernel_image:115: Loaded kernel to 0x80200000 (17180396 bytes)
  Info: (riscv/kvm.c) kvm__arch_load_kernel_image:126: Placing fdt at 0x81800000 - 0x87ffffff
  # Warning: The maximum recommended amount of VCPUs is 1
  Info: (virtio/mmio.c) virtio_mmio_init:325: virtio-mmio.devices=0x200@0x10000000:5
  Info: (virtio/mmio.c) virtio_mmio_init:325: virtio-mmio.devices=0x200@0x10000200:6
  Info: (virtio/mmio.c) virtio_mmio_init:325: virtio-mmio.devices=0x200@0x10000400:7
[    0.000000] OF: fdt: Ignoring memory range 0x80000000 - 0x80200000
[    0.000000] Linux version 5.8.0-rc4-00022-g906eafa224e2 (anup@anup-ubuntu64-vm) (riscv64-unknown-linux-gnu-gcc (GCC) 9.2.0, GNU ld (GNU Binutils) 2.34) #1 SMP Fri Jul 24 19:10:43 IST 2020
[    0.000000] earlycon: sbi0 at I/O port 0x0 (options '')
[    0.000000] printk: bootconsole [sbi0] enabled
[    0.000000] initrd not found or empty - disabling initrd
[    0.000000] Zone ranges:
[    0.000000]   DMA32    [mem 0x0000000080200000-0x0000000087ffffff]
[    0.000000]   Normal   empty
[    0.000000] Movable zone start for each node
[    0.000000] Early memory node ranges
[    0.000000]   node   0: [mem 0x0000000080200000-0x0000000087ffffff]
[    0.000000] Initmem setup node 0 [mem 0x0000000080200000-0x0000000087ffffff]
[    0.000000] software IO TLB: mapped [mem 0x83e3b000-0x87e3b000] (64MB)
[    0.000000] SBI specification v0.1 detected
[    0.000000] riscv: ISA extensions acdfim
[    0.000000] riscv: ELF capabilities acdfim
[    0.000000] percpu: Embedded 17 pages/cpu s31704 r8192 d29736 u69632
[    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 31815
[    0.000000] Kernel command line:  console=ttyS0 rw rootflags=trans=virtio,version=9p2000.L,cache=loose rootfstype=9p init=/virt/init  ip=dhcp console=ttyS0 earlycon=sbi
[    0.000000] Dentry cache hash table entries: 16384 (order: 5, 131072 bytes, linear)
[    0.000000] Inode-cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
[    0.000000] Sorting __ex_table...
[    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
[    0.000000] Memory: 43852K/129024K available (6456K kernel code, 4177K rwdata, 4096K rodata, 226K init, 323K bss, 85172K reserved, 0K cma-reserved)
[    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=2, Nodes=1
[    0.000000] rcu: Hierarchical RCU implementation.
[    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=2.
[    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 25 jiffies.
[    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
[    0.000000] NR_IRQS: 64, nr_irqs: 64, preallocated irqs: 0
[    0.000000] riscv-intc: 64 local interrupts mapped
[    0.000000] plic: interrupt-controller@0c000000: mapped 1024 interrupts with 2 handlers for 4 contexts.
[    0.000000] riscv_timer_init_dt: Registering clocksource cpuid [0] hartid [1]
[    0.000000] clocksource: riscv_clocksource: mask: 0xffffffffffffffff max_cycles: 0x24e6a1710, max_idle_ns: 440795202120 ns
[    0.000015] sched_clock: 64 bits at 10MHz, resolution 100ns, wraps every 4398046511100ns
[    0.006745] Console: colour dummy device 80x25
[    0.008065] Calibrating delay loop (skipped), value calculated using timer frequency.. 20.00 BogoMIPS (lpj=40000)
[    0.015210] pid_max: default: 32768 minimum: 301
[    0.020780] Mount-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
[    0.022885] Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
[    0.031695] rcu: Hierarchical SRCU implementation.
[    0.037715] smp: Bringing up secondary CPUs ...
[    0.039850] smp: Brought up 1 node, 2 CPUs
[    0.056535] devtmpfs: initialized
[    0.065370] random: get_random_u32 called from bucket_table_alloc.isra.0+0x4e/0x154 with crng_init=0
[    0.080845] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 7645041785100000 ns
[    0.083665] futex hash table entries: 512 (order: 3, 32768 bytes, linear)
[    0.093070] NET: Registered protocol family 16
[    0.132545] HugeTLB registered 2.00 MiB page size, pre-allocated 0 pages
[    0.136970] vgaarb: loaded
[    0.138010] SCSI subsystem initialized
[    0.144945] usbcore: registered new interface driver usbfs
[    0.146575] usbcore: registered new interface driver hub
[    0.148175] usbcore: registered new device driver usb
[    0.168580] clocksource: Switched to clocksource riscv_clocksource
[    0.181160] NET: Registered protocol family 2
[    0.183125] tcp_listen_portaddr_hash hash table entries: 256 (order: 0, 4096 bytes, linear)
[    0.185790] TCP established hash table entries: 1024 (order: 1, 8192 bytes, linear)
[    0.188045] TCP bind hash table entries: 1024 (order: 2, 16384 bytes, linear)
[    0.190425] TCP: Hash tables configured (established 1024 bind 1024)
[    0.192785] UDP hash table entries: 256 (order: 1, 8192 bytes, linear)
[    0.194700] UDP-Lite hash table entries: 256 (order: 1, 8192 bytes, linear)
[    0.197155] NET: Registered protocol family 1
[    0.209480] RPC: Registered named UNIX socket transport module.
[    0.211195] RPC: Registered udp transport module.
[    0.212835] RPC: Registered tcp transport module.
[    0.214200] RPC: Registered tcp NFSv4.1 backchannel transport module.
[    0.216065] PCI: CLS 0 bytes, default 64
[    0.217890] kvm [1]: hypervisor extension not available
[    0.232510] workingset: timestamp_bits=62 max_order=14 bucket_order=0
[    0.245060] NFS: Registering the id_resolver key type
[    0.246540] Key type id_resolver registered
[    0.247755] Key type id_legacy registered
[    0.249270] nfs4filelayout_init: NFSv4 File Layout Driver Registering...
[    0.251505] 9p: Installing v9fs 9p2000 file system support
[    0.253445] NET: Registered protocol family 38
[    0.254745] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252)
[    0.257225] io scheduler mq-deadline registered
[    0.258540] io scheduler kyber registered
[    0.260420] pci-host-generic 30000000.pci: host bridge /smb/pci ranges:
[    0.262495] pci-host-generic 30000000.pci:       IO 0x0000000000..0x000000ffff -> 0x0000000000
[    0.265345] pci-host-generic 30000000.pci:      MEM 0x0040000000..0x007fffffff -> 0x0040000000
[    0.267900] pci-host-generic 30000000.pci: ECAM at [mem 0x30000000-0x3fffffff] for [bus 00-01]
[    0.270585] pci-host-generic 30000000.pci: PCI host bridge to bus 0000:00
[    0.272750] pci_bus 0000:00: root bus resource [bus 00-01]
[    0.274345] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
[    0.276140] pci_bus 0000:00: root bus resource [mem 0x40000000-0x7fffffff]
[    0.447025] Serial: 8250/16550 driver, 4 ports, IRQ sharing disabled
[    0.450325] printk: console [ttyS0] disabled
[    0.451585] 3f8.U6_16550A: ttyS0 at MMIO 0x3f8 (irq = 4, base_baud = 115200) is a 16550A
[    0.454300] printk: console [ttyS0] enabled
[    0.454300] printk: console [ttyS0] enabled
[    0.457820] printk: bootconsole [sbi0] disabled
[    0.457820] printk: bootconsole [sbi0] disabled
[    0.462045] 2f8.U6_16550A: ttyS1 at MMIO 0x2f8 (irq = 6, base_baud = 115200) is a 16550A
[    0.466605] 3e8.U6_16550A: ttyS2 at MMIO 0x3e8 (irq = 7, base_baud = 115200) is a 16550A
[    0.471185] 2e8.U6_16550A: ttyS3 at MMIO 0x2e8 (irq = 8, base_baud = 115200) is a 16550A
[    0.475845] [drm] radeon kernel modesetting enabled.
[    0.488970] loop: module loaded
[    0.497115] libphy: Fixed MDIO Bus: probed
[    0.503110] e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
[    0.506215] e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
[    0.509355] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.512500] ehci-pci: EHCI PCI platform driver
[    0.514835] ehci-platform: EHCI generic platform driver
[    0.517625] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.520780] ohci-pci: OHCI PCI platform driver
[    0.522920] ohci-platform: OHCI generic platform driver
[    0.536735] usbcore: registered new interface driver uas
[    0.539300] usbcore: registered new interface driver usb-storage
[    0.542665] mousedev: PS/2 mouse device common for all mice
[    0.552835] usbcore: registered new interface driver usbhid
[    0.555470] usbhid: USB HID core driver
[    0.558120] NET: Registered protocol family 10
[    0.565200] Segment Routing with IPv6
[    0.566995] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
[    0.570500] NET: Registered protocol family 17
[    0.573035] 9pnet: Installing 9P2000 support
[    0.576810] Key type dns_resolver registered
[    0.596695] Sending DHCP requests ., OK
[    0.599110] IP-Config: Got DHCP answer from 192.168.33.1, my address is 192.168.33.15
[    0.602935] IP-Config: Complete:
[    0.604695]      device=eth0, hwaddr=02:15:15:15:15:15, ipaddr=192.168.33.15, mask=255.255.255.0, gw=192.168.33.1
[    0.609795]      host=192.168.33.15, domain=, nis-domain=(none)
[    0.612785]      bootserver=192.168.33.1, rootserver=0.0.0.0, rootpath=
[    0.612790]      nameserver0=192.168.33.1
[    0.620275] VFS: Mounted root (9p filesystem) on device 0:15.
[    0.623735] devtmpfs: mounted
[    0.625975] Freeing unused kernel memory: 224K
[    0.640940] Run /virt/init as init process
Mounting...
/ # ^[[47;5R

/ # ^[[47;5Rcat /proc/cpuinfo
cat /proc/cpuinfo
[    1.267070] random: fast init done
processor	: 0
hart		: 1
isa		: rv64imafdc
mmu		: sv48

processor	: 1
hart		: 0
isa		: rv64imafdc
mmu		: sv48

/ # ^[[47;5Rcat /proc/interrupts
cat /proc/interrupts
           CPU0       CPU1       
  1:        117          0  SiFive PLIC   5  virtio0
  2:        145          0  SiFive PLIC   6  virtio1
  3:          5          0  SiFive PLIC   7  virtio2
  4:         14          0  SiFive PLIC   1  ttyS0
  5:        180        174  RISC-V INTC   5  riscv-timer
IPI0:         4         18  Rescheduling interrupts
IPI1:       465        800  Function call interrupts
IPI2:         0          0  CPU stop interrupts
/ # ^[[47;5R