Docker fails to create container after upgrading docker on Jetpack 4.9

[LukaTri]

I upgraded docker using sudo apt-get update/upgrade, and now when I try to run the nvcr.io/nvidia/l4t-ml:r32.6.1-py3 container, I get this error message:

`docker: Error response from daemon: failed to create shim: OCI runtime create failed: container_linux.go:380: starting container process caused: error adding seccomp filter rule for syscall

clone3: permission denied: unknown.`

If I have to downgrade docker to the previous version, how do I do that? And if not, what can I do to fix this error?

[Robotmad]

JetPack 4.6 on Jetson Nano - I'm getting the same error during building of containers (setup for runtime = nvidia) which were fine previously. Have issues with all nvidia containers which includes: nvcr.io/nvidia/l4t-pytorch:r32.6.1-pth1.9-py3, nvcr.io/nvidia/l4t-base:r32.6.1 & dustynv/ros:noetic-ros-base-l4t-r32.6.1

Is this related perhaps? NVIDIA/nvidia-container-runtime#157

[LukaTri]

Yes, that seems to be the same issue. I will look through that thread and update any solitons/updates here. Thanks.

[jgardynik]

I ran into this today as well (what a bad time to update), and spent a couple of hours fiddling with things to attempt to fix it.

Downgrading is probably the easiest approach at the moment. The newest version of the nvidia-docker-toolkit that fixes this problem is currently in experimental, and packages haven't even been built for arm64 yet (I let them know on that linked issue). You can download a .deb file of the last version of Docker 19 at https://launchpad.net/ubuntu/bionic/arm64/docker.io/19.03.6-0ubuntu1~18.04.3 .

You'll probably also have to downgrade containerd to 1.5.2 by doing

apt install containerd=1.5.2-0ubuntu1~18.04.3

You may also want to pin docker.io to version 19 and containerd to 1.5.2 for now so it doesn't get updated again until they sort things out (or until Jetpack 5.0 releases next year sometime and we get a slightly less crusty version of Ubuntu). You can do that by editing /etc/apt/preferences and adding:

Package: docker.io
Pin: version 19.03*
Pin-Priority: 1001

Package: containerd
Pin: version 1.5.2*
Pin-Priority: 1001

[LukaTri]

Thank you, that worked perfectly. I did just as you said, I downloaded the docker.io v19... in the link you provided, then I ran the downgrade for containerd. I also did create a file in /etc/apt/preferences to ignore docker and containerd updates for now.

[yamachaso]

I had an error when I tried to downgrade to Docker 19 before I downgraded containerd, so you should downgrade containerd first,

$ sudo apt install containerd=1.5.2-0ubuntu1~18.04.3

and then Docker 19.

$ sudo apt install ./docker.io_19.03.6-0ubuntu1_18.04.3_arm64.deb

Also, you can fix the version of docker.io and containerd in the following way, without editting /etc/apt/preferences

$ sudo apt-mark hold docker.io containerd

If the problem is solved and you want to upgrade them,

$ sudo apt-mark unhold docker.io containerd

[JeremieBourque1]

Do we know if an update has been published to solve this problem? If not, do we know when we can expect it to be published?

[LukaTri]

@JeremieBourque1 I am unaware of any updates that have fixed this issue. I still have used this thread's solution as:

1. Downgrading containerd to containerd=1.5.2-0ubuntu1~18.04.3
2. Downloading the docker.io package, then running this command: ./docker.io_19.03.6-0ubuntu1_18.04.3_arm64.deb
3. Marking the docker.io package and containerd package as hold by doing: sudo apt-mark hold docker.io containerd

If you do find that an update has been released that addresses this issue, please post it here!

[dusty-nv]

We have released a fix for this, here are the steps to run it:

distribution=$(. /etc/os-release;echo $ID$VERSION_ID) \
   && curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | sudo apt-key add - \
   && curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | sudo tee /etc/apt/sources.list.d/nvidia-docker.list

sudo apt-get update
sudo apt-get install nvidia-docker2=2.8.0-1

Let me know if that works for you guys.

[JeremieBourque1]

@dusty-nv It works for me, thank you!