firecracker coredump #2393
Comments
magnate3
added
the
Type: Bug
|
Hi @magnate3 It seems that the built-in seccomp filters are stopping the Firecracker process when the kernel is loaded. The seccomp filters can be disabled by passing the Could you give us more details on what happens on your host? I did not manage to reproduce this on one of our machines. You can start the firecracker process using strace: Run the curl request and check paste the last 100-200 lines from the |
|
@gc-plp the curl request is : curl --unix-socket /tmp/firecracker.socket -i |
|
@gc-plp thank you very much, I follow you advice and try like following and the firecracker start successfully |
|
Hi @magnate3. You don't need to run Firecracker with strace. You can just do I took a quick look at the first strace log and it looks similar to what I can see on my machine. |
|
ok，I understand what you say. thank you. execve("/usr/local/bin/firecracker", ["firecracker", "--api-sock", "/tmp/firecracker.socket", "--seccomp-level", "0"], 0xffffe84883d0 /* 22 vars */) = 0 |
|
Hi @magnate3, I took another look at the output and don't find an obvious reason for the failure just from the logs. I also tried reproducing it with different kernels on an aarch64 machine but it doesn't trigger on our side, which could indicate that the kernel that you are using is somehow different from the one that I used, which is 5.4.0, based on an Ubuntu 20.04. Could you give us a few details on the environment you are running on? Information on the processor model and overall environment would be useful. I also noticed that you're running Ubuntu with a 5.0.0 kernel. Is this the vanilla version of the kernel? What Ubuntu version are you using? You might also see valuable information by enabling the logging feature. Just add Thanks! |
|
@gc-plp thanks for you detail response. root@ubuntu: root@ubuntu:~# cat logs.fifo |
|
The Regarding the firecracker binary, I wanted to confirm with you that you're running Could you also try running the prebuilt binary on our release page? |
|
@gc-plp hello, if I run the prebuilt binary,there is no problem. the following test is exected by my own firecracker |
|
@magnate3 in the tests that you did before, are you using the binary built using devtool? If not, are you by any chance running |
|
I build like this : |
|
@magnate3 what commit ID are you basing your build on? What's the output of |
|
@gc-plp |
|
@magnate3 I'd like to go back a bit to running strace if possible, I seem to have missed a parameter. Could you add the
|
|
@gc-plp 11428 execve("/usr/local/bin/firecracker", ["firecracker", "--api-sock", "/tmp/firecracker.socket", "--seccomp-level", "0"], 0xffffea55b7a8 /* 23 vars */) = 0 |
|
@magnate3, what's the output of Also, could you update to the latest commit ID and rebuild? |
|
@gc-plp root@ubuntu:~/myfirecracker/firecracker# git status nothing to commit, working tree clean firecracker# git fetch buf,after I execute fit fetch and tools/devtool build, the firecracker still has the same bug. cp ./build/cargo_target/aarch64-unknown-linux-musl/debug/firecracker /usr/local/bin/ |
|
Hi @magnate3. Thanks for all the info! It's been very useful so far. I do have a few more requests: Can you run Before building the binary on your machine, please make sure that you're checking out the corresponding 0.24 tag by running |
|
@gc-plp hi， my branck is v0.24.0 after run tools/devtool build root@ubuntu: |
|
@magnate3 could also run the same |
|
@gc-plp ok, cat firecracker.output |
|
Thanks @magnate3 for reporting this issue and giving us all the requested information. The Firecracker process is terminated by the This issue is not reproduced on all platforms and environments because of vdso. Vdso is a linux shared library that implements a couple of frequently used syscalls in user space, to improve application performance. For example, for aarch64, these are: Calling a system call through vdso, in userspace, essentially bypasses seccomp filters. This is a long-known problem of seccomp: https://lwn.net/Articles/795128/. Musl, the default libc implementation that Firecracker uses, first tries to find the according symbol in the vdso library. If it doesn't exist there, it falls back to calling the actual It would be interesting to see what platform are you testing on (CPU model, etc.) and whether you have any special configuration that disables vdso. We are going to shortly publish a fix for your issue. Thank you again for reporting this and all your cooperation. |
|
The PR addressing this issue is merged, please confirm that it fixes your problem. |
|
@alindima thanks,but there is still the same problem root@ubuntu:~/myfirecracker/firecracker# git branch --all
|
|
Can you please first run a |
|
@alindima thanks, but the problem still exist root@ubuntu:~/myfirecracker/firecracker# git fetch --all root@ubuntu: root@ubuntu: root@ubuntu: |
|
Looking into the log you pasted, you are running firecracker v0.24, not the latest Firecracker: You copied the latest binary into |
|
@alindima nice,thank you .This is caused by my carelessness.now, the problem is solved. Ubuntu 18.04.2 LTS fadfdd4af58a ttyS0 fadfdd4af58a login: |
|
great to hear, thanks |
Describe the bug
[Author TODO: A clear and concise description of what the bug is.]root@ubuntu:~/myfirecracker/firecracker# firecracker --api-sock /tmp/firecracker.socket
Bad system call (core dumped)
To Reproduce
1、 start firecrakcer: firecracker --api-sock /tmp/firecracker.socket
2、 send curl request
curl --unix-socket /tmp/firecracker.socket -i
-X PUT 'http://localhost/boot-source'
-H 'Accept: application/json'
-H 'Content-Type: application/json'
-d "{
"kernel_image_path": "${kernel_path}",
"boot_args": "keep_bootcon console=ttyS0 reboot=k panic=1 pci=off"
}"
curl: (52) Empty reply from server
curl: (7) Couldn't connect to server
curl: (7) Couldn't connect to server
Environment
uname -a
Linux ubuntu 5.0.0-23-generic #24~18.04.1-Ubuntu SMP Mon Jul 29 16:10:24 UTC 2019 aarch64 aarch64 aarch64 GNU/Linux
the firecrakcer is builded as following:
root@ubuntu:
/myfirecracker# git clone https://github.com/firecracker-microvm/firecracker.git/myfirecracker# cd firecrackerCloning into 'firecracker'...
remote: Enumerating objects: 82, done.
remote: Counting objects: 100% (82/82), done.
remote: Compressing objects: 100% (69/69), done.
remote: Total 25405 (delta 31), reused 25 (delta 12), pack-reused 25323
Receiving objects: 100% (25405/25405), 17.58 MiB | 5.38 MiB/s, done.
Resolving deltas: 100% (15370/15370), done.
root@ubuntu:
root@ubuntu:~/myfirecracker/firecracker# tools/devtool build
The text was updated successfully, but these errors were encountered: