OT (but not that much) : Invalid accelerator kvm / kvm run failed , Function not implemented ERRORS trying to virtualize a guest OS on the ARM Chromebook running with a relatively recent kernel version and KVM enabled

[Marietto2008]

Hello.

I see that you are experienced with qemu-kvm,I want to try to ask you what do you think about what I've got from the project that I've completed,but that does not work.

I've enabled KVM on a more recent kernel version,to be precise on the 5.4.244,on the old ARM Chromebook model xe303c12,following mainly this tutorial :

http://www.virtualopensystems.com/en/solutions/guides/kvm-on-chromebook/

This is how I have achieved the goal (or at least I think that I did) :

I have formatted my sd card following the procedure explained by the v.o.s website.

In the u boot source directory I did :

$ sudo ./scripts/sdcard.sh /dev/sdX

This erased all data and created 4 partitions in the sd-card, along with copying the u-boot binary to the first partition:

1. ChromeOS signed binary (vos chained u-boot)
   

2. Partition 2 not used
   

3. EXT2 partition for u-boot files (here I have copied your zImage (converted in uImage and your exynos5250-snow.dtb files). I've got the zip file that you created for me one month ago,here :
   

https://github.com/quarkscript/linux-armv7-xe303c12-only/files/11615040/5.4.244-kvm.zip

I have decompressed it and I've got the zImage file that was stored inside the deb file.

Since I want to use u-boot,I have converted zImage into uImage using the following command :

mkimage -A arm -O linux -T kernel -C none -a 0x80008000 -e 0x80008000 -n "Linux kernel" -d zImage uImage

4. EXT4 partition for userspace files. Here I have copied the devuan 4 userland files that I've got here :

https://drive.google.com/u/0/uc?id=1KSgE-3G2GEBzcYhk_JleVYWl6m061GdI&export=download

but I haven't used btrfs fs,but ext4. And I have extended the space on the partition from 1882M to 200 GB.

sudo dd if=nv_uboot-snow.kpart of=/dev/sdX1 (nv_uboot-snow.kpart taken from the vos website)

I wrote a little bash script that executes the command line :

dmesg > /root/Desktop/log

as soon as the desktop environment started. This is the content of the log file :

https://pastebin.ubuntu.com/p/GQ59sqt72v/

I read "kvm: Hyp mode initialized successfully". So I'm sure that kvm is working on the kernel version that I've used (5.4.244),infact :

OK,now let's talk about how to virtualize a GUEST os.

I've installed virt-manager and I tried to create a new virtual machine. With my big surprise,I saw the warning :

KVM is not available. This may mean the KVM package is not installed or the KVM kernel modules are not loaded.

What could be the cause ? KVM seems to be running great. It is detected by kvm-ok and even virt-manager can establish a connection to QEMU / KVM. The qemu version installed is 5.2.0.

I conducted some further experiments :

In this example I've used qemu 5.2.0 installed by default by the devuan 4 :

qemu-system-arm \
    -enable-kvm -serial stdio -kernel zImage \
    -m 512 -M vexpress-a15 -cpu cortex-a15 \
    -drive file=/mnt/fisso/bhyve/img/Linux/ubuntu2210.img,id=virtio-blk,if=none \
    -device virtio-blk,drive=virtio-blk,transport=virtio-mmio.0 \
    -device virtio-net,transport=virtio-mmio.1,netdev=net0,mac="52:54:00:12:34:55" \
    -netdev type=user,id=net0 \
    -append "earlyprintk=ttyAMA0 console=ttyAMA0 mem=512M \
             virtio_mmio.device=1M@0x4e000000:74:0 \
             virtio_mmio.device=1M@0x4e100000:75:1 \
             root=/dev/vda rw ip=dhcp --no-log"

qemu-system-arm: invalid accelerator kvm. 

In this example I've compiled qemu-system-arm on ubuntu 12.04 as described by the vos tutorial and then I've copied the libraries produced on ubuntu 12.04 to the devuan 4 + 5.4 kernel installation :

/12.04/usr/local/bin/./qemu-system-arm \
    -enable-kvm -serial stdio -kernel zImage \
    -m 512 -M vexpress-a15 -cpu cortex-a15 \
    -drive file=/mnt/fisso/bhyve/img/Linux/ubuntu2210.img,id=virtio-blk,if=none \
    -device virtio-blk,drive=virtio-blk,transport=virtio-mmio.0 \
    -device virtio-net,transport=virtio-mmio.1,netdev=net0,mac="52:54:00:12:34:55" \       
    -netdev type=user,id=net0 \
    -append "earlyprintk=ttyAMA0 console=ttyAMA0 mem=512M \
             virtio_mmio.device=1M@0x4e000000:74:0 \
             virtio_mmio.device=1M@0x4e100000:75:1 \
             root=/dev/vda rw ip=dhcp --no-log"


(process:2843): GLib-WARNING **: 14:17:40.742: ../../../glib/gmem.c:497: custom memory allocation vtable not supported

error: kvm run failed Function not implemented

At this point I don't know what to do to enable the kvm acceleration,but I have a vague idea about the reason why it does not work.

The virtual open systems developers says :

The guests that we will create later will be based on the Versatile Express QEMU machine model. For we will use a different kernel branch specific to that platform:

In my opinion :

In the first experiment,KVM does not work because qemu 5.2.0 is too new and / or it is not patched by v.s.o developers.

In the second experiment,KVM does not work because the qemu version used by v.o.s is very old and it is not compatible with the kernel that I'm using, 5.4.244.

My final goal is to virtualize FreeBSD on the ARM Chromebook,but I think that I will never be able to do this. Maybe,with your help I will be able to virtualize Ubuntu. Maybe.

I've also got an answer by Peter Maidell,a very known qemu-kvm developer,which says :

You can't use KVM with the vexpress-a15 board. The only Arm guest machine that will work with KVM is the "virt" board.
Further, it looks like your host CPU is 32-bit. QEMU 5.2 dropped support for running KVM on 32-bit hosts, because this
support was dropped from host kernels in kernel version 5.7. Basically, 32-bit hosts are just too small to do anything
sensible with virtual machines, which is why the kernel (and in turn QEMU) dropped that support. If you want to play
around with Arm virtualization, use a 64-bit host (64-bit hosts can still use KVM to virtualize 32-bit guests if you want
32 bit guests)

I don't know if he read well what I said or if I didn't understand something,BUT :

point 1)

On the virtual open systems website they used the vexpress-a15 board. You can read by yourself what they say :

The guests that we will create later will be based on the Versatile Express QEMU machine model. For we will use a different kernel branch specific to that platform:

$ git checkout origin/kvm-vexpress-3.9
$ curl http://www.virtualopensystems.com/downloads/guides/kvm_on_chromebook/guest-config > .config
$ ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make zImage dtbs -j4

Then we can append the dtb file to the kernel image with:

$ cat arch/arm/boot/dts/rtsm_ve-cortex_a15x1.dtb >> arch/arm/boot/zImage

That way we only need to handle one file. Make sure to copy arch/arm/boot/zImage for later usage.

point 2)

On the devuan 4 host os that I have installed I'm using kernel 5.4,that's lower than 5.7,so the qemu support for it should be there.

point 3)

I tried -M virt like this :

qemu-system-arm \
    -enable-kvm -serial stdio -kernel zImage \
    -m 512 -M virt -cpu cortex-a15 \

but I've got the same error : qemu-system-arm: invalid accelerator kvm.

[raspiduino]

From https://lists.gnu.org/archive/html/qemu-devel/2020-09/msg02074.html

We deprecated the support for KVM on 32-bit Arm hosts in time
for release 5.0, which means that our deprecation policy allows
us to drop it in release 5.2. Remove the code.

Can you try using an older version of QEMU? Like 4.x? Thanks!

[Marietto2008]

I've asked to the ML and the qemu developers said that I should use qemu 5.1. So,I have compiled and installed and I ran it using the following parameters :

/mnt/./monta-fisso
sleep 2
qemu-system-arm \
    -enable-kvm -serial stdio \
    -m 512 -M virt -cpu cortex-a15 \
    -drive if=pflash,format=raw,file=/usr/share/AAVMF/AAVMF32_CODE.fd \
    -drive file=/mnt/fisso/OS/FreeBSD/FreeBSD-13.2.qcow2,id=virtio-blk,if=none \
    -device virtio-blk,drive=virtio-blk \
    -device virtio-net,netdev=net0,mac="52:54:00:12:34:55" \
    -netdev type=user,id=net0 \

I think to be close,but it still does not work :

do you know what could be wrong ?

[raspiduino]

I've asked to the ML and the qemu developers said that I should use qemu 5.1. So,I have compiled and installed and I ran it using the following parameters :

/mnt/./monta-fisso
sleep 2
qemu-system-arm \
    -enable-kvm -serial stdio \
    -m 512 -M virt -cpu cortex-a15 \
    -drive if=pflash,format=raw,file=/usr/share/AAVMF/AAVMF32_CODE.fd \
    -drive file=/mnt/fisso/OS/FreeBSD/FreeBSD-13.2.qcow2,id=virtio-blk,if=none \
    -device virtio-blk,drive=virtio-blk \
    -device virtio-net,netdev=net0,mac="52:54:00:12:34:55" \
    -netdev type=user,id=net0 \

I think to be close,but it still does not work :

do you know what could be wrong ?

Now EFI shell booted. That means your KVM patched worked. Now the problem is with your boot disk. On QEMU start, you might try press ESC to enter EFI setting, then manually select the EFI file in your FreeBSD boot disk. I don't have much experience with booting FreeBSD, so you might have to research yourself.

[Marietto2008]

I tried a different set of parameters :

/mnt/./monta-fisso
sleep 2

UEFICODE=/usr/share/AAVMF/AAVMF32_CODE.fd
UEFIVARS=/usr/share/AAVMF/AAVMF32_VARS.fd
CD1=/mnt/fisso/OS/ISO/BSD/FreeBSD-13.2-RELEASE-arm-armv7-GENERICSD.img
CD2=/mnt/fisso/OS/ISO/Linux/bookworm_mini_armhf.iso
DISK=/mnt/fisso/bhyve/img/FreeBSD/FreeBSD.img

qemu-system-arm \
   -enable-kvm -serial stdio \
   -m 512 -M virt -cpu cortex-a15 \
   -drive if=pflash,format=raw,unit=0,file=$UEFICODE \
   -drive if=pflash,format=raw,unit=1,file=$UEFIVARS \
   -cdrom $CD2 \
   -drive file=$DISK,format=raw \
   -device virtio-net,netdev=net0,mac="52:54:00:12:34:55" \
   -netdev type=user,id=net0 \
   -device virtio-gpu-pci \
   -usb -device nec-usb-xhci \
   -device usb-kbd -device usb-mouse \
   -device vmware-svga,id=video0,vgamem_mb=16

but the error is always there.

[raspiduino]

IDK. But this is related to boot problem inside the VM (EFI failed to find boot partition), and not related to KVM or probably command line problem.

[Marietto2008]

I've found the solution.

Is here : utmapp/UTM#2333 (comment)

[raspiduino]

I've found the solution.

Is here : utmapp/UTM#2333 (comment)

Great to see it works for you!