Podman container not starting on Debian Bookworm

[Joulinar]

/kind bug

Description

I was playing with my NanoPi R6S Debian Bookworm. However, it seems I'm not able to get a container running due to issues with netavark. As I'm using a minimal Debian image, I'm not sure if something else is missing. At least on my NanoPi R5S Bullseye, it's working without issues. Probably it is related to Debian Bookworm.

Steps to reproduce the issue:

1. apt install podman
2. podman pull docker.io/nginx
3. podman run --name docker-nginx -p 8080:80 docker.io/nginx

Describe the results you received:

root@DietPiR6S:~# podman  run --name docker-nginx -p 8080:80 docker.io/nginx
Error: netavark: No such file or directory (os error 2)

Describe the results you expected:

Container is able to start

Additional information you deem important (e.g. issue happens only occasionally):

Even netavark seems to be missing, it has been installed along with podman

root@DietPiR6S:~# dpkg -l netavark
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version      Architecture Description
+++-==============-============-============-=======================================
ii  netavark       1.0.3-4      arm64        Rust based network stack for containers

Output of podman version:

root@DietPiR6S:~# podman version
Client:       Podman Engine
Version:      4.3.1
API Version:  4.3.1
Go Version:   go1.19.3
Built:        Thu Jan  1 01:00:00 1970
OS/Arch:      linux/arm64

Output of podman info:

root@DietPiR6S:~# podman info
host:
  arch: arm64
  buildahVersion: 1.28.0
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  cgroupManager: systemd
  cgroupVersion: v2
  conmon:
    package: conmon_2.1.3+ds1-1_arm64
    path: /usr/bin/conmon
    version: 'conmon version 2.1.3, commit: unknown'
  cpuUtilization:
    idlePercent: 99.4
    systemPercent: 0.22
    userPercent: 0.37
  cpus: 8
  distribution:
    codename: bookworm
    distribution: debian
    version: unknown
  eventLogger: journald
  hostname: DietPiR6S
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.10.110
  linkmode: dynamic
  logDriver: journald
  memFree: 7563628544
  memTotal: 8045228032
  networkBackend: netavark
  ociRuntime:
    name: crun
    package: crun_1.5+dfsg-1+b1_arm64
    path: /usr/bin/crun
    version: |-
      crun version 1.5
      commit: 54ebb8ca8bf7e6ddae2eb919f5b82d1d96863dea
      spec: 1.0.0
      +SYSTEMD +SELINUX +APPARMOR +CAP +SECCOMP +EBPF +YAJL
  os: linux
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 0
  swapTotal: 0
  uptime: 0h 10m 8.00s
plugins:
  authorization: null
  log:
  - k8s-file
  - none
  - passthrough
  - journald
  network:
  - bridge
  - macvlan
  volume:
  - local
registries: {}
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 0
    stopped: 1
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/lib/containers/storage
  graphRootAllocated: 30875299840
  graphRootUsed: 3058674688
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 1
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 4.3.1
  Built: 0
  BuiltTime: Thu Jan  1 01:00:00 1970
  GitCommit: ""
  GoVersion: go1.19.3
  Os: linux
  OsArch: linux/arm64
  Version: 4.3.1

Package info (e.g. output of rpm -q podman or apt list podman or brew info podman):

root@DietPiR6S:~# apt list podman
Listing... Done
podman/testing,now 4.3.1+ds1-5 arm64 [installed]

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

Yes/

Additional environment details (AWS, VirtualBox, physical, etc.):

Minimized Debian Bookworm on a NanoPi R6S

[vrothberg]

Did you install podman from the main Debian repository? It seems more like a packaging issue to me.

[Joulinar]

yes, installed directly from global Debian package repository.

Ok, issue is missing iptables package. It is marked as optional but should be a dependency.

[vrothberg]

Can you file an issue against Debian for that? I suspect that a netavark package isn't available on bookworm for this version of the Pi.

As a workaround you can change the network backend to CNI. To do that, set network_backened="cni" in the [network] table in /etc/containers/containers.conf.

I am going to close the issue, as it's a downstream packaging issue but we can for sure continue the conversation here.

[Joulinar]

issue is the missing iptables package. It is marked as optional but should be a dependency. Installing iptables is fixing it. Btw. on Bullseye, iptables package is a dependency.

https://packages.debian.org/bullseye/podman
https://packages.debian.org/bookworm/podman

[vrothberg]

Thanks! Did you open against Debian for it?

[vrothberg]

@flouthoc, could you improve the error message in netavark to include which binary it was looking for?

Error: netavark: No such file or directory (os error 2) is missing a bit of context.

[Joulinar]

Thanks! Did you open against Debian for it?

No didn't do that actually. I was not sure about the issue at the beginning. The missing iptables just turned out later on.

[vrothberg]

Thanks! Did you open against Debian for it?

No didn't do that actually. I was not sure about the issue at the beginning. The missing iptables just turned out later on.

If you find the time, please open a bug. There's not much we can do upstream for downstream packaging.