Skip to content
This repository has been archived by the owner on May 30, 2024. It is now read-only.

avocado‐vt on aarch64

Dom edited this page Aug 21, 2023 · 16 revisions

Getting Started

Install avocado-vt:

dnf install xz tcpdump iproute iputils gcc glibc-headers nc git
dnf install genisoimage ksmtuned                  #They are what the cases requires
git clone https://github.com/avocado-framework/avocado-vt
cd avocado-vt
pip install .

pip install transitions

/root/avocado/data/avocado-vt/virttest/test-providers.d/downloads/io-github-autotest-qemu/generic/tests/cfg/unattended_install.cfg
/root/avocado/data/avocado-vt/backends/qemu/cfg/subtests.cfg
-REPLACE_THIS_WITH_TREE_URL
#https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/35/Server/aarch64/os/

-REPLACE_THIS_WITH_NFS_DIRECTORY
Create NFS server like:
nfs_server = "10.76.*.*"
nfs_dir = "/root/test/nfs"

Copy files to the following directory:

/usr/local/lib/python3.11/site-packages/virttest/shared/cfg/guest-os/Linux/Fedora/35.cfg
/usr/local/lib/python3.11/site-packages/virttest/shared/downloads/f35-aarch64.ini
/usr/local/lib/python3.11/site-packages/virttest/shared/unattended/Fedora-35.ks

Reload config:

avocado vt-bootstrap --vt-type qemu --vt-guest-os Fedora.35.aarch64 --vt-qemu-bin /usr/bin/qemu-system-aarch64

Run test:

avocado run io-github-autotest-qemu.unattended_install.url.http_ks.default_install.aio_native --vt-type qemu --vt-guest-os Fedora.35.aarch64 --vt-machine-type arm64-mmio
or
avocado --show all run io-github-autotest-qemu.unattended_install.url.http_ks.default_install.aio_native --vt-type qemu --vt-guest-os Fedora.35.aarch64 --vt-machine-type arm64-mmio

Check the log:

Test log:/root/avocado/job-results/latest/full.log
VM log:/root/avocado/job-results/latest/test-results/1-io-github-autotest-qemu.unattended_install.url.http_ks.default_install.aio_native/serial-serial0-avocado-vt-vm1.log

How to use monitor on qemu.

./qemu-system-aarch64 ... -monitor unix:/tmp/qmp-test,server,nowait

Start another shell:
nc -U /tmp/qmp-test 

New Fedora 35 configuration

/usr/local/lib/python3.11/site-packages/virttest/shared/cfg/guest-os/Linux/Fedora/35.cfg

- 35:
    variants:
        - aarch64:
            vm_arch_name = aarch64
       - x86_64:
            vm_arch_name = x86_64
    image_name = images/f35-${vm_arch_name}
    os_variant = fedora35
    shell_prompt = "^.*\[.*\][\#\$]\s*$"
    # default boot path is set in ../Fedora.cfg: boot_path = "images/pxeboot"
    no unattended_install..floppy_ks
    unattended_install, svirt_install:
        kernel = images/f35-${vm_arch_name}/vmlinuz
        initrd = images/f35-${vm_arch_name}/initrd.img
        # Unattended-file does not require any changes
        unattended_file = unattended/Fedora-35.ks
        unattended_install.url:
            unattended_file_kernel_param_name = "inst.ks"
            # Installation works fine with mem=1024 on methods such as cdrom
            # but fails ("No space left on device") with methods such as url.
            mem = 2048
            url = https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/35/Server/${vm_arch_name}/os/
        # ARCH dependent things
        aarch64:
            arm64-mmio:
                # Only latest updates contain virtio driver
                inactivity_watcher = none
                take_regular_screendumps = no
            kernel_params = "console=ttyAMA0 console=ttyS0"
            unattended_install.cdrom:
                md5sum_cd1 = 041e59c5b4bc2f05c9d8bdc7b697fef8
                md5sum_1m_cd1 = 3fe59272e7e3aa2d45db3d92f795f43d
            unattended_install.url:
                url = https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/35/Server/aarch64/os/
                sha1sum_vmlinuz = 3505f2751e2833c681de78cee8dda1e49cabd2e8
                sha1sum_initrd = df6acb53fa1ce6e15c2d65bda18a4ac8ea77701a
 #               url = http://dl.fedoraproject.org/pub/fedora/linux/releases/35/Server/${vm_arch_name}/os
 #               sha1sum_vmlinuz = 3505f2751e2833c681de78cee8dda1e49cabd2e8
 #               sha1sum_initrd = df6acb53fa1ce6e15c2d65bda18a4ac8ea77701a
        x86_64:
            kernel_params = "console=tty0 console=ttyS0"
            unattended_install.cdrom:
                md5sum_cd1 = af29363e2c8825948515aac36701fbfa
                md5sum_1m_cd1 = 8c09623eee12715472242f844cf69e4e
            unattended_install.url:
                url = http://dl.fedoraproject.org/pub/fedora/linux/releases/35/Server/${vm_arch_name}/os
                sha1sum_vmlinuz = 5b6f6876e1b5bda314f93893271da0d5777b1f3c
                sha1sum_initrd = 0d7b4bdccb3ca82e0a221104d9f20eda0cc53309
        # Shared specific setting
        unattended_install.url:
            # TODO: Remove the "inst.repo" in newer Fedora versions, requiring it was a bug/regression
            #kernel_params += " inst.repo=${url}"
        unattended_install.cdrom:
            cdrom_cd1 = iso/Fedora-Server-dvd-${vm_arch_name}-35-1.2.iso
        extra_cdrom_ks:
            kernel_params += " inst.ks=cdrom"
            cdrom_unattended = images/f35-${vm_arch_name}/ks.iso
        syslog_server_proto = tcp

/usr/local/lib/python3.11/site-packages/virttest/shared/downloads/f35-aarch64.ini

[f35-aarch64]
title = Fedora 35 aarch64
url = https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/35/Server/aarch64/images/Fedora-Server-35-1.2.aarch64.raw.xz
sha1_url = https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/35/Server/aarch64/images/Fedora-Server-35-1.2-aarch64-CHECKSUM
destination = images/f35-aarch64.raw.xz
destination_uncompressed = images/f35-aarch64.raw

/usr/local/lib/python3.11/site-packages/virttest/shared/unattended/Fedora-35.ks

# Generated by Anaconda 35.22.2
# Generated by pykickstart v3.34
#version=DEVEL
# Use text mode install
text

# Keyboard layouts
keyboard --vckeymap=us

# System language
lang en_US.UTF-8
network --bootproto dhcp --hostname avocado-vt
rootpw 123456
firewall --enabled --ssh
selinux --enforcing
timezone --utc America/New_York
firstboot --disable

# Use network installation
url --url="https://archives.fedoraproject.org/pub/archive/fedora/linux/releases/35/Server/aarch64/os/"

%packages --ignoremissing
@c-development
@development-tools
net-tools
sg3_utils
# include avocado: allows using this machine with remote runner
python3-avocado
@^server-product-environment

%end

# Run the Setup Agent on first boot
#firstboot --enable
# Do not configure the X Window System
skipx

# Generated using Blivet version 3.4.2
ignoredisk --only-use=vda
# System bootloader configuration
bootloader --location=mbr --boot-drive=vda
autopart --type=plain
# Partition clearing information
clearpart --all --initlabel --drives=vda

# System timezone
timezone America/New_York --utc
poweroff
# Root password
rootpw --iscrypted $y$j9T$mEWZGxZdNtI86rCNBK6BkkiT$buWPtZzwVtphGQjnitd1iQ4DIBHUBycXgrkoKKFxzRC

%post
# Output to all consoles defined in /proc/consoles, use "major:minor" as
# device names are unreliable on some platforms
# https://bugzilla.redhat.com/show_bug.cgi?id=1351968
function ECHO { for TTY in `cat /proc/consoles | awk '{print $NF}'`; do source "/sys/dev/char/$TTY/uevent" && echo "$*" > /dev/$DEVNAME; done }
ECHO "OS install is completed"
grubby --remove-args="rhgb quiet" --update-kernel=$(grubby --default-kernel)
dhclient
iptables -F
systemctl mask tmp.mount
selinux --enforcing
sed -i "/^HWADDR/d" /etc/sysconfig/network-scripts/ifcfg-eth0
# if packages were missing from main installation repo
# try again from installed system
ECHO "Install tools..."
dnf -y groupinstall c-development development-tools
dnf -y install net-tools sg3_utils python3-avocado lksctp-tools
sudo yum -y install lksctp-tools-devel
dnf -y install iperf iperf3 fio uperf

ECHO "Enable ssh connection..."
systemctl enable sshd
#dnf install -y --enablerepo=fedora-debuginfo --enablerepo=updates-debuginfo kexec-tools crash kernel-debuginfo
#grubby --args="crashkernel=512M" --update-kernel=ALL
echo "Enable kdump service for crash test..."
systemctl enable kdump.service
#From fedora31,root login is disabled by default,we need enable it for our test
echo "PermitRootLogin yes" >> /etc/ssh/sshd_config

ECHO 'Post set up finished'
%end


Update the config with avocado vt-bootstrap --vt-type qemu --vt-guest-os Fedora.35.aarch64

Valid QEMU config

/usr/bin/qemu-img create -f qcow2 /root/avocado/data/avocado-vt/images/f35-aarch64_test.qcow2 20G

MALLOC_PERTURB_=1  /usr/bin/qemu-system-aarch64 \
    -S  -monitor unix:/tmp/qmp-test,server,nowait -nographic \
    -name 'avocado-vt-vm1-ks' \
    -blockdev node-name=file_aavmf_code,driver=file,filename=/usr/share/edk2/aarch64/QEMU_EFI-silent-pflash.raw,auto-read-only=on,discard=unmap \
    -blockdev node-name=drive_aavmf_code,driver=raw,read-only=on,file=file_aavmf_code \
    -blockdev node-name=file_aavmf_vars,driver=file,filename=/root/avocado/data/avocado-vt/avocado-vt-vm1_f35-aarch64_qcow2_filesystem_VARS.raw,auto-read-only=on,discard=unmap \
    -blockdev node-name=drive_aavmf_vars,driver=raw,read-only=off,file=file_aavmf_vars \
    -machine virt,gic-version=host,pflash0=drive_aavmf_code,pflash1=drive_aavmf_vars,memory-backend=mem-machine_mem  \
    -m 2048 \
    -object memory-backend-ram,size=2048M,id=mem-machine_mem  \
    -smp 2,maxcpus=2,cores=2,threads=1,clusters=1,sockets=1  \
    -cpu 'host' \
    -device qemu-xhci,id=usb1 \
    -device usb-tablet,id=usb-tablet1,bus=usb1.0,port=1 \
    -blockdev node-name=file_image1,driver=file,aio=native,filename=/root/avocado/data/avocado-vt/images/f35-aarch64_test.qcow2,cache.direct=on,cache.no-flush=off \
    -blockdev node-name=drive_image1,driver=qcow2,cache.direct=on,cache.no-flush=off,file=file_image1 \
    -device virtio-blk-device,id=image1,drive=drive_image1,bootindex=1,write-cache=on \
    -kernel '/root/avocado/data/avocado-vt/images/f35-aarch64/vmlinuz'  \
    -append 'console=ttyAMA0 console=ttyS0 loglevel=7 inst.ks=http://10.76.*.*/ks.cfg'  \
    -initrd '/root/avocado/data/avocado-vt/images/f35-aarch64/initrd.img'  \
    -rtc base=utc,clock=host \
    -enable-kvm \
    -device virtio-mouse-device,id=input_mouse1 \
    -device virtio-keyboard-device,id=input_keyboard1

Q&A

Error:

sudo ./qemu-system-aarch64 -machine virt -cpu host -accel kvm -bios QEMU_EFI.fd -kernel Image -append 'root=/dev/vda1 nokaslr console=ttyAMA0' -drive if=none,file=debian-sid-nocloud-arm64-daily.qcow2,format=qcow2,id=hd -device virtio-blk-pci,drive=hd -nographic -net user,hostfwd=tcp::2222-:22 -net nic -smp cpus=8 -m 4G
./qemu-system-aarch64: error while loading shared libraries: libsasl2.so.2: cannot open shared object file: No such file or directory

Solution:

wget https://rpmfind.net/linux/openmandriva/cooker/repository/aarch64/main/release/lib64sasl2-2.1.28-2-omv2390.aarch64.rpm
rpm -i lib64sasl2-2.1.28-2-omv2390.aarch64.rpm

When use a external kernel to start JeOS.27,it crash::

-kernel Image -append 'root=/dev/vda1 nokaslr console=ttyAMA0'
-drive if=none,file=/root/avocado/data/avocado-vt/images/jeos-27-aarch64.qcow2,format=qcow2,id=hd.

[    0.557921] VFS: Mounted root (vfat filesystem) readonly on device 254:1.
[    0.558915] devtmpfs: error mounting -2
[    0.560725] Freeing unused kernel memory: 8192K
[    0.600837] Run /sbin/init as init process                                                                                                                                      [    0.602014] Run /etc/init as init process
[    0.603061] Run /bin/init as init process
[    0.604074] Run /bin/sh as init process
[    0.646743] Kernel panic - not syncing: No working init found.  Try passing init= option to kernel. See Linux Documentation/admin-guide/init.rst for guidance.
[    0.649363] CPU: 3 PID: 1 Comm: swapper/0 Not tainted 6.1.0-rc4-01340-g8bcf8bc8f52b #17
[    0.650859] Hardware name: QEMU KVM Virtual Machine, BIOS unknown unknown
[    0.652134] Call trace:
[    0.652741]  dump_backtrace+0x11c/0x128
[    0.653479]  show_stack+0x18/0x40
[    0.654112]  dump_stack_lvl+0x68/0x84
[    0.654810]  dump_stack+0x18/0x34
[    0.655438]  panic+0x184/0x344
[    0.656021]  kernel_init+0x128/0x130
[    0.656704]  ret_from_fork+0x10/0x20
[    0.657382] SMP: stopping secondary CPUs
[    0.658181] Kernel Offset: disabled
[    0.658847] CPU features: 0x00000,00070097,e6567ea7
[    0.659761] Memory Limit: none
[    1.800497] ---[ end Kernel panic - not syncing: No working init found.  Try passing init= option to kernel. See Linux Documentation/admin-guide/init.rst for guidance. ]---

Solution:

Delete -kernel Image -append "root=/dev/vda1 nokaslr console=ttyAMA0" in the command.
VM will use the kernel in the qcow2 to start.

Error of openvswitch on ubuntu:

2023-07-12 11:21:40,859 avocado.virttest.utils_net ERROR| 01-type_specific.io-github-autotest-qemu.migrate.default.tcp.default: Host does not support OpenVSwitch: Command         'systemctl restart openvswitch.service' failed.
stdout: b''
stderr: b'Failed to restart openvswitch.service: Unit openvswitch.service not found.\n'
additional_info: None
2023-07-12 11:21:40,859 avocado.virttest.utils_misc ERROR| 01-type_specific.io-github-autotest-qemu.migrate.default.tcp.default: Fail to create qemu command:
2023-07-12 11:21:40,859 avocado.virttest.utils_misc ERROR| 01-type_specific.io-github-autotest-qemu.migrate.default.tcp.default: Original Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/virttest/qemu_vm.py", line 3310, in create
    self.devices, self.spice_options = self.make_create_command()
  File "/usr/local/lib/python3.10/dist-packages/virttest/qemu_vm.py", line 1688, in make_create_command
    devs = devices.machine_by_params(params)
  File "/usr/local/lib/python3.10/dist-packages/virttest/qemu_devices/qcontainer.py", line 1506, in machine_by_params
    devices = machine_arm64_mmio(machine_params)
  File "/usr/local/lib/python3.10/dist-packages/virttest/qemu_devices/qcontainer.py", line 1347, in machine_arm64_mmio
    pflash_devices = pflash_handler("aavmf", machine_params)
  File "/usr/local/lib/python3.10/dist-packages/virttest/qemu_devices/qcontainer.py", line 1083, in pflash_handler
    raise exceptions.TestError("The firmware path is not exist."
avocado.core.exceptions.TestError: The firmware path is not exist. Maybe you need to install related packages.

Solution:

New version of openvswitch on ubuntu install /lib/systemd/system/openvswitch-switch.service instead of /lib/systemd/system/openvswitch.service.
But avocado use "systemctl restart openvswitch.service".
We can copy /lib/systemd/system/openvswitch-switch.service to /lib/systemd/system/openvswitch.service
service openvswitch start.
systemctl status openvswitch

Error:

service openvswitch-switch restart
A dependency job for openvswitch-switch.service failed. See 'journalctl -xe' for details.

Solution for db.sock:

rm -r /usr/var/run
ln -s /run /usr/var

Error:

/usr/bin/ld: cannot find -luuid: No such file or directory

Solution:

ln -s /usr/lib64/libuuid.so.1.3.0 /usr/lib64/libuuid.so

Error on Altra

Reproduced traceback from: /usr/local/lib/python3.10/dist-packages/avocado_vt/test.py:274
Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/virttest/error_context.py", line 135, in new_fn
    return fn(*args, **kwargs)
  File "/root/avocado/data/avocado-vt/virttest/test-providers.d/downloads/io-github-autotest-qemu/qemu/tests/migration.py", line 230, in run
    vm.migrate(mig_timeout, mig_protocol, mig_cancel_delay,
  File "/usr/local/lib/python3.10/dist-packages/virttest/error_context.py", line 135, in new_fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/virttest/qemu_vm.py", line 4631, in migrate
    clone.create(migration_mode=protocol, mac_source=self,
  File "/usr/local/lib/python3.10/dist-packages/virttest/error_context.py", line 135, in new_fn
    return fn(*args, **kwargs)
  File "/usr/local/lib/python3.10/dist-packages/virttest/qemu_vm.py", line 3450, in create
    raise e

Root cause:

[qemu output] qemu-system-aarch64: -vnc :0: Failed to find an available port: Address already in use

Solution:

Use other vnc number,eg 1.
qemu-syst 27125 27129 qemu-syst             root   16u     IPv4            5255373        0t0        TCP *:5900 (LISTEN)
qemu-syst 27324 27354 qemu-syst             root   17u     IPv6            5403990        0t0        TCP *:5901 (LISTEN)
vnc 0 use port 5900,vnc1 use port 5901

AmpereOne has the following error, but Ampere Altra uses the same command and the image works fine:

sudo ./qemu-system-aarch64 -machine virt -cpu max -accel kvm -bios QEMU_EFI.fd -drive if=none,file=/root/avocado/data/avocado-vt/images/jeos-27-aarch64.qcow2,format=qcow2,id=hd -device virtio-blk-pci,drive=hd -nographic -net user,hostfwd=tcp::2222-:22 -net nic -smp cpus=8 -m 4G

The VM blocks after print:
EFI stub: Booting Linux Kernel...
EFI stub: EFI_RNG_PROTOCOL unavailable, no randomness supplied
EFI stub: Using DTB from configuration table
EFI stub: Exiting boot services and installing virtual address map...

How to show more kernel log:

Press E in grub
Add earlycon=pl011,mmio,0x09000000 at the tail of "linux /boot/vmlinuz"
Press Ctrl+x

[    0.000000] Booting Linux on physical CPU 0x0                                                                                                                                   
[    0.000000] Linux version 4.13.9-300.fc27.aarch64 (mockbuild@buildvm-aarch64-02.arm.fedoraproject.org) (gcc version 7.2.1 20170915 (Red Hat 7.2.1-2) (GCC)) #1 SMP Mon Oct 23 13:33:18 UTC 2017                                                                                                                                                                    
[    0.000000] Boot CPU: AArch64 Processor [c00fac31]                                                                                                                              
[    0.000000] earlycon: pl11 at MMIO 0x0000000009000000 (options '')                                                                                                              
[    0.000000] bootconsole [pl11] enabled                                                                                                                                          
[    0.000000] efi: Getting EFI parameters from FDT:                                                                                                                               
[    0.000000] efi: EFI v2.70 by EDK II                                                                                                                                            
[    0.000000] efi:  SMBIOS 3.0=0x13bed0000  MEMATTR=0x13a892018  ACPI 2.0=0x138430018                                                                                             
[    0.000000] cma: Reserved 64 MiB at 0x00000000fc000000                                                                                                                          
[    0.000000] ------------[ cut here ]------------                                                                                                                                
[    0.000000] kernel BUG at arch/arm64/mm/mmu.c:154!                                                                                                                              
[    0.000000] Internal error: Oops - BUG: 0 [#1] SMP                                                                                                                              
[    0.000000] Modules linked in:                                                                                                                                                  
[    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.13.9-300.fc27.aarch64 #1                                                                                                  
[    0.000000] task: ffff0000092d3400 task.stack: ffff0000092c0000                                                                                                                 
[    0.000000] PC is at alloc_init_pud+0x530/0x598                                                                                                                                 
[    0.000000] LR is at alloc_init_pud+0x264/0x598

./run-kvm-unit-test.sh

RESULTS    : PASS 27 | ERROR 0 | FAIL 0 | SKIP 11 | WARN 0 | INTERRUPT 0 | CANCEL 0

Downalod https://nmap.org/dist/nmap-7.94.tar.bz2 and install it.

RESULTS    : PASS 33 | ERROR 0 | FAIL 0 | SKIP 5 | WARN 0 | INTERRUPT 0 | CANCEL 0

REPLACE_THIS_WITH_NFS_DIRECTORY Error:

avocado-vt/virttest/test-providers.d/downloads/io-github-autotest-qemu/generic/tests/cfg/unattended_install.cfg:241:            nfs_dir = REPLACE_THIS_WITH_NFS_DIRECTORY

Solution:

avocado-vt/backends/qemu/cfg/subtests.cfg
            - nfs:
                only Linux
                medium = nfs
                nfs_server = "10.76.*.*"       
                nfs_dir = "/root/Dom/nfs"
Setup nfs server and config nfs_server and nfs_dir.

Python.h Error:

    aarch64-linux-gnu-gcc -fno-strict-aliasing -Wdate-time -D_FORTIFY_SOURCE=2 -g -ffile-prefix-map=/build/python2.7-3iiHDl/python2.7-2.7.18=. -flto=auto -ffat-lto-objects -flto=auto -ffat-lto-objects -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/include/python2.7 -c src/wrapt/_wrappers.c -o build/temp.linux-aarch64-2.7/src/wrapt/_wrappers.o
    src/wrapt/_wrappers.c:3:10: fatal error: Python.h: No such file or directory
        3 | #include "Python.h"
          |          ^~~~~~~~~~
    compilation terminated.
    error: command 'aarch64-linux-gnu-gcc' failed with exit status 1

Solution:

apt-get install python-dev

redir Error

qemu-system-aarch64: -redir: invalid option
qemu-system-aarch64: -net user: network backend 'user' is not compiled into this binary

Solution:

Enable slirp when compile qemu8.0
./configure --target-list=aarch64-softmmu --disable-lzo --disable-snappy --disable-docs --enable-slirp --prefix=/usr

qemu4.2 doesn't need '--enable-slirp',but the user backend works well.

imaging library Error

No python imaging library installed
pip install Pillow

io-github-autotest-qemu.save_restore Error:

avocado run io-github-autotest-qemu.save_restore --vt-type qemu --vt-guest-os Fedora.35.aarch64 --vt-machine-type arm64-mmio

QMP command 'migrate_set_speed' failed    (arguments: {'value': 1152921504606846976},    error message: {'class': 'CommandNotFound', 'desc': 'The command migrate_set_speed has not been found'})

Root cause:

virttest/qemu_devices/qcontainer.py
            _ = process.run("echo -e 'help\nquit' | %s -monitor "
                            "stdio -vnc none -S" % qemu_binary,
                            timeout=10, ignore_status=True,
                            shell=True, verbose=False).stdout_text

It returns:

QEMU 8.0.3 monitor - type 'help' for more information^M (qemu) -^[[K^[[D-e^[[K^[[D^[[D-e ^[[K^[[D^[[D^[[D-e h^[[K^[[D^[[D^[[D^[[D-e he^[[K^[[D^[[D^[[D^[[D^[[D-e hel^[[K^[[D^[[D^[[D^[[D^[[D^[[D-e help^[[K^M unknown command: '-e'^M (qemu) q^[[K^[[Dqu^[[K^[[D^[[Dqui^[[K^[[D^[[D^[[Dquit^[[K^M

Solution:

            _ = process.run("echo 'help\nquit' | %s -monitor "
                            "stdio -vnc none -S" % qemu_binary,
                            timeout=10, ignore_status=True,
                            shell=True, verbose=False).stdout_text
It returns:
QEMU 8.0.3 monitor - type 'help' for more information^M
(qemu) h^[[K^[[Dhe^[[K^[[D^[[Dhel^[[K^[[D^[[D^[[Dhelp^[[K^M
announce_self [interfaces] [id] -- Trigger GARP/RARP announcements^M
balloon target -- request VM to change its memory allocation (in MB)^M
block_job_cancel [-f] device -- stop an active background block operation (use -f^M
             if you want to abort the operation immediately^M
             instead of keep running until data is in sync)^M
...
xp /fmt addr -- physical memory dump starting at 'addr'^M
(qemu) q^[[K^[[Dqu^[[K^[[D^[[Dqui^[[K^[[D^[[D^[[Dquit^[[K^M

Nonetype error:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/avocado/core/main.py", line 50, in handle_exception
    tmp, name = tempfile.mkstemp(".log", prefix, get_crash_dir())
  File "/usr/local/lib/python3.10/dist-packages/avocado/core/main.py", line 37, in get_crash_dir
    crash_dir_path = os.path.join(config.get("datadir.paths.data_dir"), "crashes")
  File "/usr/lib/python3.10/posixpath.py", line 76, in join
    a = os.fspath(a)
TypeError: expected str, bytes or os.PathLike object, not NoneType

Solution,reinstall avocado:

pip uninstall avocado-framework
pip uninstall avocado-framework-plugin-vt
pip install git+https://github.com/avocado-framework/avocado-vt

Error about kernel config:

avocado.virttest.tests.unattended_install: 1-io-github-autotest-qemu.unattended_install.nfs.extra_cdrom_ks.default_install.aio_native: Monitoring serial console log for completion
 message: /root/avocado/job-results/job-2023-08-03T23.19-46b7f59/test-results/1-io-github-autotest-qemu.unattended_install.nfs.extra_cdrom_ks.default_install.aio_native/serial-serial0-avocado-vt-vm1.log
avocado.virttest.tests.unattended_install: 1-io-github-autotest-qemu.unattended_install.nfs.extra_cdrom_ks.default_install.aio_native: Cannot read from serial log file after 11 tries

./qemu-system-aarch64 -nographic -machine virt,gic-version=max -enable-kvm -bios /usr/share/edk2/aarch64/QEMU_EFI.fd -smp cpus=8 -m 4G -net nic -net user,hostfwd=tcp::2222-:22 -drive if=none,file=debian-12-nocloud-arm64-20230711-1438.qcow2,format=qcow2,id=hd1 -device virtio-blk-pci,drive=hd1,bootindex=0 -drive if=none,file=cloudinit,format=raw,id=hd10 -device virtio-blk-pci,drive=hd10,bootindex=10
qemu-system-aarch64: KVM is not supported for this guest CPU type
qemu-system-aarch64: kvm_init_vcpu failed: Invalid argument

Solution:

It has something to do with the kernel configuration, but I don't know which one.

make check-avocado Error:

 (09/37) tests/avocado/machine_aarch64_virt.py:Aarch64VirtMachine.test_alpine_virt_tcg_gic_max: INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout reached\nOriginal status: ERROR\n{'name': '09-tests/avocado/machine_aarch64_virt.py:Aarch64VirtMachine.test_alpine_virt_tcg_gic_max', 'logdir': '/home/dom/Qemu/code/qemu/build/tests/results/job-2023... (360.53 s)

2023-08-11 15:04:32,625 stacktrace       L0045 ERROR|   Command: ./qemu-system-aarch64 -display none -vga none -chardev socket,id=mon,fd=15 -mon chardev=mon,mode=control -incoming tcp:localhost:31130 -nodefaults
2023-08-11 15:04:32,625 stacktrace       L0045 ERROR|   Output: qemu-system-aarch64: No machine specified, and there is no default
(12/37) tests/avocado/migration.py:Migration.test_migration_with_tcp_localhost: ERROR: ConnectError: Failed to establish session: EOFError\n   Exit code: 1\n  Command: ./qemu-system-aarch64 -display none -vga none -chardev socket,id=mon,fd=15 -mon chardev=mon,mode=control -incoming tcp:localhost:31130 -nodefaults\n      Output: qemu-system-aarch64: No... (0.10 s)

Solution:

TBD

type_specific.io-github-autotest-qemu.passthrough_with_multipath_device Error:

avocado run type_specific.io-github-autotest-qemu.passthrough_with_multipath_device --vt-type qemu --vt-guest-os Fedora.35.aarch64 --vt-machine-type arm64-mmio
  ERROR: VM 'avocado-vt-vm1' failed to start: Error occurred while executing make_create_command()
avocado run io-github-autotest-qemu.timerclock.rtc --vt-type qemu --vt-guest-os Fedora.35.aarch64 --vt-machine-type arm64-mmio
  FAIL: Test fail on RTC device
avocado run io-github-autotest-qemu.timerclock.hwclock --vt-type qemu --vt-guest-os Fedora.35.aarch64 --vt-machine-type arm64-mmio
  ERROR: Shell command failed: '/sbin/hwclock --set --date "2/2/80 03:04:00"'
avocado run io-github-autotest-qemu.kdump.one_vm.with_stress.netperf_stress.default --vt-type qemu --vt-guest-os Fedora.35.aarch64 --vt-machine-type arm64-mmio
  ERROR: Stress isn't running
avocado run io-github-autotest-qemu.trans_hugepage.migration --vt-type qemu --vt-guest-os Fedora.35.aarch64 --vt-machine-type arm64-mmio
  ERROR: VM creation command failed

Solution:

TBD

type_specific.io-github-autotest-qemu.balloon_stress error:

This case will repeat a test 1000 times, and each time after sending BALLOON_CHANGE, QMP does not respond and avocado has to wait for 2 minutes before continuing to run.It looks like it's stuck.

Solution:

TBD

libvhost-user Error:

/usr/libexec/virtiofsd --socket-path=/var/tmp/avocado_5ngyr17b/avocado-vt-vm1-fs-virtiofsd.sock -o source=/tmp/virtio_fs_test -o cache=auto
    -chardev socket,id=char_virtiofs_fs,path=/var/tmp/avocado_5ngyr17b/avocado-vt-vm1-fs-virtiofsd.sock \
    -device vhost-user-fs-device,id=vufs_virtiofs_fs,chardev=char_virtiofs_fs,tag=myfs,queue-size=1024

mkdir -p /mnt/myfs
mount -t virtiofs myfs /mnt/myfs

virtio_session_mount: Waiting for vhost-user socket connection...
virtio_session_mount: Received vhost-user socket connection
virtio_loop: Entry
Killed
fv_panic: libvhost-user: virtio legacy devices aren't supported by libvhost-user

Solution:

TBD

virtiofsd Error:

/usr/libexec/virtiofsd --socket-path=/var/tmp/avocado-vt-vm1-fs-virtiofsd.sock -o source=/mnt/virtio_fs_test/ --thread-pool-size=32 --allow-direct-io -o cache=auto
fuse: unknown option(s): `--allow-direct-io'

Solution:

/usr/libexec/virtiofsd --socket-path=/var/tmp/avocado-vt-vm1-fs-virtiofsd.sock -o source=/mnt/virtio_fs_test/ --thread-pool-size=32 -o allow_direct_io -o cache=auto

EDK2 error:

UEFI firmware (version edk2-20230301gitf80f052277c8-26.fc38 built at 00:00:00 on May  5 2023)
Synchronous Exception at 0x000000007FDB4B84                                                                                                                                        PC 0x00007FDB4B84 (0x00007FDB0000+0x00004B84) [ 0] FaultTolerantWriteDxe.dll
PC 0x00007FDB4860 (0x00007FDB0000+0x00004860) [ 0] FaultTolerantWriteDxe.dll
ASSERT [ArmCpuDxe] /builddir/build/BUILD/edk2-f80f052277c8/ArmPkg/Library/DefaultExceptionHandlerLib/AArch64/DefaultExceptionHandler.c

Solution:

rm /root/avocado/data/avocado-vt/avocado-vt-vm1_f35-aarch64_qcow2_filesystem_VARS.raw

Chronyd error:

type_specific.io-github-autotest-qemu.timedrift_with_multi_vms.same_cpu: 
Context: Sync host time with ntp server
Running 'systemctl stop chronyd.service'
Running '(chronyd -q 'server clock.redhat.com iburst' || ntpdate clock.redhat.com)'

It's no response to ping clock.redhat.com iburst

Solution:

TBD