[Bug] Firecracker panicked at 'Error creating the Kvm object: Error(13)'

[tmx-dxiao]

Describe the bug

firecracker 10| 4:04 [~] ./firecracker --api-sock /tmp/firecracker.socket
2020-11-10T04:08:43.403438585 [anonymous-instance:ERROR:src/firecracker/src/main.rs:52] Firecracker panicked at 'Error creating the Kvm object: Error(13)', src/vmm/src/vstate/system.rs:53:19
[1]    1750 abort (core dumped)  ./firecracker --api-sock /tmp/firecracker.socket

To Reproduce

I was able to reproduce it by following get-started guideline

Expected behaviour

Environment

[Author TODO: Please supply the following information):]
[ - Firecracker version.] v0.23.0-x86_64
[ - Host and guest kernel versions.] ubuntu 20.04 with latest updates applied
[ - Rootfs used.] from the get-started guide
[ - Architecture.] from the get-started guide
[ - Any other relevant software versions.]

Additional context

CPU: AMD Zen 2 3700x

Checks

• Have you searched the Firecracker Issues database for similar problems?
• Have you read the existing relevant Firecracker documentation?
• Are you certain the bug being reported is a Firecracker issue?

[KarthikNedunchezhiyan]

@tmx-dxiao It seems to be an insufficient permission issue for accessing /dev/kvm. can you try running firecracker in sudo mode or after providing sufficient permission to /dev/kvm and update behavior here?

[dianpopa]

@KarthikNedunchezhiyan is right, this error appears when Firecracker does not have adequate permissions for opening /dev/kvm. Try to run Firecracker with the suggestion from our getting-started guide. if this does not solve your issue, feel free to re-open it for more details. Be aware that setfacl needs to be run before every firecracker launch.

[bryanmacfarlane]

We're having this issue on one VM and oddly not another. In the start up guide it lists it as a pre-requisite and not per launch. Doing it per launch would be problematic since /dev/kvm is a shared resource and we will have many firecrackers run and launched concurrently (and overlapping in time). If it helps, one is ubuntu 18.04 and one is ubuntu 20.04 (the one with the issue). We're trying to dig in and find the difference.

[ican2002]

I met this problem on aarch64(RaspBerry4B)., follwing the guide from https://dev.to/l1x/getting-started-with-firecracker-on-raspberry-pi-1pbc
also as the reference: https://github.com/firecracker-microvm/firecracker/blob/main/docs/getting-started.md
the same problem.

I execute all the commands with sudo. and grant access with "sudo setfacl -m u:${USER}:rw /dev/kvm"

Can anyone give some help? thanks

[ican2002]

I met this problem on aarch64(RaspBerry4B)., follwing the guide from https://dev.to/l1x/getting-started-with-firecracker-on-raspberry-pi-1pbc
also as the reference: https://github.com/firecracker-microvm/firecracker/blob/main/docs/getting-started.md
the same problem.

I execute all the commands with sudo. and grant access with "sudo setfacl -m u:${USER}:rw /dev/kvm"

Can anyone give some help? thanks
the error message:
{"fault_message":"System configuration error: SetupFDT(ReadCacheInfo(FailedToReadCacheInfo(Os { code: 2, kind: NotFound, message: "No such file or directory" })))"}

[ican2002]

I met this problem on aarch64(RaspBerry4B)., follwing the guide from https://dev.to/l1x/getting-started-with-firecracker-on-raspberry-pi-1pbc
also as the reference: https://github.com/firecracker-microvm/firecracker/blob/main/docs/getting-started.md
the same problem.
I execute all the commands with sudo. and grant access with "sudo setfacl -m u:${USER}:rw /dev/kvm"
Can anyone give some help? thanks
the error message:
{"fault_message":"System configuration error: SetupFDT(ReadCacheInfo(FailedToReadCacheInfo(Os { code: 2, kind: NotFound, message: "No such file or directory" })))"}

sudo ./firecracker --api-sock /tmp/firecracker.socket
2021-09-19T12:00:47.812895420 [anonymous-instance:main:ERROR:src/firecracker/src/main.rs:55] Could not enable SSBD mitigation through prctl, error 22
2021-09-19T12:00:47.813045382 [anonymous-instance:main:ERROR:src/firecracker/src/main.rs:60] The host does not support SSBD mitigation through prctl.
2021-09-19T12:01:17.198673983 [anonymous-instance:fc_api:ERROR:src/api_server/src/parsed_request.rs:122] Received Error. Status code: 400 Bad Request. Message: System configuration error: SetupFDT(ReadCacheInfo(FailedToReadCacheInfo(Os { code: 2, kind: NotFound, message: "No such file or directory" })))

[bryanmacfarlane]

In addition to setting permissions on /dev/kvm per the users guide, on ubuntu 20.04 we also had to do this:

sudo usermod -aG kvm ${USER}

you have to log out and back in as that user for it to take effect.

changing permissions on each execution (the second failed on ubuntu 20) isn't really an option since these are all concurrent on the same host VM. The group solution is a do once and forget solution.

[alindima]

I met this problem on aarch64(RaspBerry4B)., follwing the guide from https://dev.to/l1x/getting-started-with-firecracker-on-raspberry-pi-1pbc
also as the reference: https://github.com/firecracker-microvm/firecracker/blob/main/docs/getting-started.md
the same problem.
I execute all the commands with sudo. and grant access with "sudo setfacl -m u:${USER}:rw /dev/kvm"
Can anyone give some help? thanks
the error message:
{"fault_message":"System configuration error: SetupFDT(ReadCacheInfo(FailedToReadCacheInfo(Os { code: 2, kind: NotFound, message: "No such file or directory" })))"}

sudo ./firecracker --api-sock /tmp/firecracker.socket
2021-09-19T12:00:47.812895420 [anonymous-instance:main:ERROR:src/firecracker/src/main.rs:55] Could not enable SSBD mitigation through prctl, error 22
2021-09-19T12:00:47.813045382 [anonymous-instance:main:ERROR:src/firecracker/src/main.rs:60] The host does not support SSBD mitigation through prctl.
2021-09-19T12:01:17.198673983 [anonymous-instance:fc_api:ERROR:src/api_server/src/parsed_request.rs:122] Received Error. Status code: 400 Bad Request. Message: System configuration error: SetupFDT(ReadCacheInfo(FailedToReadCacheInfo(Os { code: 2, kind: NotFound, message: "No such file or directory" })))

@ican2002 Your errors don't seem to be related to permissions of /dev/kvm. Please open a separate issue if you are still seeing these errors, with a detailed description of your setup and commands you are running.
Off the top of my head, it seems that the user either does not have permissions to read the cache topology of the host from the sysfs located at: /sys/devices/system/cpu/cpuX/cache, or that said files do not exist or have the expected format.