This repository has been archived by the owner on May 30, 2024. It is now read-only.
-
Notifications
You must be signed in to change notification settings - Fork 16
Features and functionality of cloud hypervisor
Dom edited this page Jun 30, 2023
·
9 revisions
TBD: https://github.com/cloud-hypervisor/cloud-hypervisor/blob/main/docs/snapshot_restore.md
Not ready for ARM64. Refer to: https://github.com/cloud-hypervisor/linux/pull/13
All pass
--cpus boot=4,topology=1:1:1:1
Error parsing config: Error validating configuration: Product of CPU topology parts does not match maximum vCPUs
--cpus features=amx
Error parsing config: Invalid feature in --cpus features list: amx
All pass.
sudo ./ch-remote --api-socket /tmp/ch-socket3 resize --cpus 8
Error running command: Server responded with an error: InternalServerError: ApiError(VmResize(CpuManager(VcpuCreate(Failed to create Vcpu: Device or resource busy (os error 16)
sudo ./ch-remote --api-socket /tmp/ch-socket3 add-fs tag=myfs,socket=/tmp/virtiofs.sock
Error running command: Server responded with an error: InternalServerError: ApiError(VmAddFs(ConfigValidation(VhostUserRequiresSharedMemory)))
sudo ./ch-remote --api-socket /tmp/ch-socket3 add-net tap=chtap0
{"id":"_net2","bdf":"0000:00:05.0"}
但VM没有新增虚拟网卡
sudo ./ch-remote --api-socket /tmp/ch-socket3 add-pmem file=/tmp/ubuntu-cloudinit_2.img
{"id":"_pmem4","bdf":"0000:00:06.0"}
sudo ./ch-remote --api-socket /tmp/ch-socket3 add-vsock cid=3,socket=/tmp/vsock.sock
命令运行成功,但不知道在VM中如何验证
--memory size=8G,hugepages=on 不识别hugepages=on,看help是支持的
Error booting VM: VmBoot(MemoryManager(SharedFileSetLen(Os { code: 22, kind: InvalidInput, message: "Invalid argument" })))
The following code returns SharedFileSetLen:
vmm/src/memory_manager.rs
f.set_len(size as u64).map_err(Error::SharedFileSetLen)?;
The size is 3154116608.
After modifying to the following configuration, the VM can be started normally.
--memory size=2G,hugepages=on
--cmdline "console=ttyS0 console=hvc0 root=/dev/vda1 rw hugepagesz=2M hugepages=512"
The size changes to 2147483648 for set_len.
No test case.
--net ip=192.168.101.1
[ OK ] Started Network Time Synchronization.
[ OK ] Reached target System Time Set.
[ OK ] Reached target System Time Synchronized.
It will be blocked for a few minutes to wait for the network.
The document require to run cloud-hypervisor in VM,but it return error:
./cloud-hypervisor: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.33' not found (required by ./cloud-hypervisor)
./cloud-hypervisor: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.32' not found (required by ./cloud-hypervisor)
./cloud-hypervisor: /lib/aarch64-linux-gnu/libc.so.6: version `GLIBC_2.34' not found (required by ./cloud-hypervisor)
TBD
Please reference Macvtap-bridge-for-cloud-hypvervisor
这两条命令在VM中的内存总量是1.5G,其他的都是1G
--memory size=1G,hotplug_method=virtio-mem,hotplug_size=1G,hotplugged_size=512M
--memory size=0,hotplug_method=virtio-mem --memory-zone id=mem0,size=1G,hotplug_size=1G,hotplugged_size=512M
--memory size=0 --memory-zone id=mem0,size=1G,hotplug_size=1G
cloud-hypervisor: 4.763259ms: <vmm> ERROR:vmm/src/memory_manager.rs:741 -- Invalid to set ACPI hotplug method for memory zones
Error booting VM: VmBoot(MemoryManager(InvalidHotplugMethodWithMemoryZones))
正确配置:
--memory size=0,hotplug_method=virtio-mem --memory-zone id=mem0,size=1G,hotplug_size=1G
--numa guest_numa_id=0
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', vmm/src/config.rs:1897:67
--numa guest_numa_id=0,cpus=[1-3,7] guest_numa_id=1,cpus=[0,4-6]
Unrecognized argument: guest_numa_id=1,cpus=[0,4-6]
--memory size=0 --memory-zone id=mem0,size=1G --memory-zone id=mem1,size=1G --memory-zone id=mem2,size=1G --numa guest_numa_id=0,memory_zones=[mem0,mem2] --numa guest_numa_id=1, memory_zones=mem1
thread 'vmm' panicked at 'called `Option::unwrap()` on a `None` value', arch/src/aarch64/fdt.rs:729:22
--sgx-epc id=epc0,size=32M id=epc1,size=64M id=epc2,size=32M
--numa guest_numa_id=0,sgx_epc_sections=epc1 --numa guest_numa_id=1,sgx_epc_sections=[epc0,epc2]
Arm不支持sgx
All pass
Remember the VM is restored in a state, which was the VM's state when it was snapshot. For this reason, one must explicitly the VM before to start using it.pausedresume
./ch-remote --api-socket /tmp/cloud-hypervisor.sock resume
Please reference edk2-firmware
CH issue:VM starts failed when use parameter of vfio on Ampere Altra
确认/sys/bus/pci/devices/0003:04:00.0/下有多个文件
用 --device path=/sys/bus/pci/devices/0003:04:00.0/启动vm
Error booting VM: VmBoot(DeviceManager(VfioCreate(OpenGroup(Os { code: 2, kind: NotFound, message: "No such file or directory" }, "44"))))
--device 'path=/sys/bus/pci/devices/0003\:04\:00.0/
Error booting VM: VmBoot(DeviceManager(VfioCreate(InvalidPath)))
Solution:
echo 0003:04:00.0 > /sys/bus/pci/devices/0003\:04\:00.0/driver/unbind
echo 144d a808 > /sys/bus/pci/drivers/vfio-pci/new_id
ls /dev/vfio/
46 vfio
New error:
cloud-hypervisor: 154.505510ms: <vmm> ERROR:/usr/local/rust/git/checkouts/vfio-3ea6ca0c73c7450b/ea8f710/crates/vfio-ioctls/src/vfio_device.rs:786 -- Could not get region #8 info
cloud-hypervisor: 294.621808ms: <vmm> ERROR:pci/src/vfio.rs:1527 -- Could not remove the userspace memory region: Failed to remove user memory: Invalid argument (os error 22)
Error booting VM: VmBoot(DeviceManager(VfioMapRegion(CreateUserMemoryRegion(CreateUserMemory(Invalid argument (os error 22))))))
Solution:
Modify config for host:
CONFIG_ARM64_4K_PAGES=y
CONFIG_ARM64_VA_BITS_48=y
CONFIG_ARM64_VA_BITS=48
CONFIG_ARM64_PA_BITS_48=y
CONFIG_ARM64_PA_BITS=48
You can find pci device with the command lspci on guest.
BTW,qemu vfio works fine with 64KB pages
CH discussion:Failed to start VM when using feature of vfio-user on aarch64 platform.
More info: Cloud-hypervisor-vfio-user
VM可以启动,但linux无法启动,一直卡住
https://github.com/cloud-hypervisor/cloud-hypervisor/blob/main/docs/virtiofs-root.md#to-create-the-vm-rootfs
for循环中所有提到的文件在host上都找不到.
TBD