Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Restoring a checkpoint fails: requested static ip not in any subnet on network podman #12762

Closed
TheloTheGreat opened this issue Jan 6, 2022 · 11 comments · Fixed by containers/common#881
Closed

Restoring a checkpoint fails: requested static ip not in any subnet on network podman #12762

TheloTheGreat opened this issue Jan 6, 2022 · 11 comments · Fixed by containers/common#881
Assignees
@Luap99
Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.

Comments

@TheloTheGreat
Copy link

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

Following the sequence from this checkpoint/restore test, restoring a checkpoint fails when podman tries to reassign the static ip to the container:
https://src.fedoraproject.org/rpms/criu/blob/093f8b4513016427781231995f20e1213b722370/f/tests/run-podman-checkpoint-restore.sh

Steps to reproduce the issue:

Short repro steps:

root@Experience:/home/thelo/proj/podman-scratch# podman --log-level debug run --tmpfs /tmp --tmpfs /usr/local/tomcat/logs -d docker://docker.io/yovfiatbeb/podman-criu-test
<...snip...>
root@Experience:/home/thelo/proj/podman-scratch# podman inspect -l | jq -r '.[0].NetworkSettings.IPAddress'
172.16.16.8
root@Experience:/home/thelo/proj/podman-scratch# curl http://172.16.16.8:8080/examples/servlets/servlet/HelloWorldExample
1
root@Experience:/home/thelo/proj/podman-scratch# curl http://172.16.16.8:8080/examples/servlets/servlet/HelloWorldExample
2
root@Experience:/home/thelo/proj/podman-scratch# podman container checkpoint -l
fde04aefe184ad4ad235c9a3abcc8af7e0603bed56f2efb6911be550820a9feb
root@Experience:/home/thelo/proj/podman-scratch# podman container restore -l
Error: requested static ip 172.16.16.8 not in any subnet on network podman

Notably, every time I remove that container then re-create it with the same command, it gets assigned a new ip (172.16.16.8, 172.16.16.9, 172.16.16.10 and so on). But the old IP seems to become unusable afterwards.

If I try to re-create a container with an option like --ip=172.16.16.8 when a previous, removed container used that ip, then that will fail with a similar error:
Error: requested static ip 172.16.16.8 not in any subnet on network podman

But that IP really was the one that was auto-assigned to the previous container, and that was tested to work correctly. So I guess that somehow, the original container never releases its IP address when it gets checkpointed and stopped. Then it can't get restored because that IP is still blocked? (I don't really understand how the IP addresses get assigned here.)

This is the full output from my repro sequence, including the output from podman run:
https://thelo.ca/podman-cant-restore-ip.txt

Output of podman version:

root@Experience:/home/thelo/proj/podman-scratch# podman version
Client:       Podman Engine
Version:      4.0.0-dev
API Version:  4.0.0-dev
Go Version:   go1.17
Git Commit:   d67f178563a8948ddc625f99f27be84a27b2cb50
Built:        Wed Jan  5 16:50:00 2022
OS/Arch:      linux/amd64

Output of podman info --debug:

root@Experience:/home/thelo/proj/podman-scratch# podman info --debug
host:
  arch: amd64
  buildahVersion: 1.23.1
  cgroupControllers:
  - cpuset
  - cpu
  - io
  - memory
  - hugetlb
  - pids
  - rdma
  cgroupManager: cgroupfs
  cgroupVersion: v2
  conmon:
    package: 'conmon: /usr/bin/conmon'
    path: /usr/bin/conmon
    version: 'conmon version 2.0.25, commit: unknown'
  cpus: 12
  distribution:
    codename: impish
    distribution: ubuntu
    version: "21.10"
  eventLogger: file
  hostname: Experience
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 5.11.0-20-generic
  linkmode: dynamic
  logDriver: k8s-file
  memFree: 10140643328
  memTotal: 16111820800
  networkBackend: cni
  ociRuntime:
    name: runc
    package: 'runc: /usr/sbin/runc'
    path: /usr/sbin/runc
    version: |-
      runc version 1.0.1-0ubuntu2
      spec: 1.0.2-dev
      go: go1.16.5
      libseccomp: 2.5.1
  os: linux
  remoteSocket:
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: true
    capabilities: CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    seccompProfilePath: /usr/share/containers/seccomp.json
    selinuxEnabled: false
  serviceIsRemote: false
  slirp4netns:
    executable: /usr/bin/slirp4netns
    package: 'slirp4netns: /usr/bin/slirp4netns'
    version: |-
      slirp4netns version 1.0.1
      commit: 6a7b16babc95b6a3056b33fb45b74a6f62262dd4
      libslirp: 4.4.0
  swapFree: 6442446848
  swapTotal: 6442446848
  uptime: 1h 57m 43.45s (Approximately 0.04 days)
plugins:
  log:
  - k8s-file
  - none
  - passthrough
  network:
  - bridge
  - macvlan
  - ipvlan
  volume:
  - local
registries: {}
store:
  configFile: /usr/share/containers/storage.conf
  containerStore:
    number: 8
    paused: 0
    running: 1
    stopped: 7
  graphDriverName: overlay
  graphOptions: {}
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "true"
    Supports d_type: "true"
    Using metacopy: "false"
  imageCopyTmpDir: /var/tmp
  imageStore:
    number: 3
  runRoot: /run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 4.0.0-dev
  Built: 1641419400
  BuiltTime: Wed Jan  5 16:50:00 2022
  GitCommit: d67f178563a8948ddc625f99f27be84a27b2cb50
  GoVersion: go1.17
  OsArch: linux/amd64
  Version: 4.0.0-dev

Package info (e.g. output of rpm -q podman or apt list podman):

Probably not useful because I have rebuilt podman from source since then, but:

root@Experience:/home/thelo/proj/podman-scratch# apt list podman
Listing... Done
podman/impish,now 3.2.1+ds1-2ubuntu3 amd64 [residual-config]

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide? (https://github.com/containers/podman/blob/master/troubleshooting.md)

Yes.

Additional environment details (AWS, VirtualBox, physical, etc.):
Physical machine running Ubuntu 21.10, but with the older kernel version 5.11.0-20-generic (to work around a kernel issue that can otherwise cause criu checkpoints to fail).
@openshift-ci openshift-ci bot added the kind/bug Categorizes issue or PR as related to a bug. label Jan 6, 2022
@TheloTheGreat
Copy link
Author

An extra note: if I restore using the --ignore-static-ip flag, then that does correctly restore the container, but on a different IP address.

root@Experience:/home/thelo/proj/podman-scratch# podman --log-level debug run --tmpfs /tmp --tmpfs /usr/local/tomcat/logs -d docker://docker.io/yovfiatbeb/podman-criu-test
<...snip...>
root@Experience:/home/thelo/proj/podman-scratch# podman inspect -l | jq -r '.[0].NetworkSettings.IPAddress'
172.16.16.10
root@Experience:/home/thelo/proj/podman-scratch# curl http://172.16.16.10:8080/examples/servlets/servlet/HelloWorldExample
1
root@Experience:/home/thelo/proj/podman-scratch# curl http://172.16.16.10:8080/examples/servlets/servlet/HelloWorldExample
2
root@Experience:/home/thelo/proj/podman-scratch# curl http://172.16.16.10:8080/examples/servlets/servlet/HelloWorldExample
3
root@Experience:/home/thelo/proj/podman-scratch# podman container checkpoint -l
c8aa514b09f71efd85d033b2ff42d576e069ebb4850cac9b0dd5534da532dcb5
root@Experience:/home/thelo/proj/podman-scratch# podman container restore --ignore-static-ip -l
c8aa514b09f71efd85d033b2ff42d576e069ebb4850cac9b0dd5534da532dcb5
root@Experience:/home/thelo/proj/podman-scratch# curl http://172.16.16.11:8080/examples/servlets/servlet/HelloWorldExample
4

@Luap99
Copy link
Member

Luap99 commented Jan 7, 2022

Where does the 172.16.16.X subnet come from? Did you change the default network subnet? Could you provide the full output of podman inspect and also podman network inspect podman.

@TheloTheGreat
Copy link
Author

I did not change any of the default settings, this is essentially straight after building podman. I don't know where that subnet comes from, I didn't type that in at any point.

root@Experience:/home/thelo/proj/podman-scratch# podman inspect -l
[
    {
        "Id": "c8aa514b09f71efd85d033b2ff42d576e069ebb4850cac9b0dd5534da532dcb5",
        "Created": "2022-01-06T16:29:05.351545832-05:00",
        "Path": "catalina.sh",
        "Args": [
            "run"
        ],
        "State": {
            "OciVersion": "1.0.2-dev",
            "Status": "running",
            "Running": true,
            "Paused": false,
            "Restarting": false,
            "OOMKilled": false,
            "Dead": false,
            "Pid": 13773,
            "ConmonPid": 13762,
            "ExitCode": 0,
            "Error": "",
            "StartedAt": "2022-01-06T16:29:05.558119729-05:00",
            "FinishedAt": "2022-01-06T16:30:00.787094859-05:00",
            "Health": {
                "Status": "",
                "FailingStreak": 0,
                "Log": null
            },
            "CgroupPath": "/libpod_parent/libpod-c8aa514b09f71efd85d033b2ff42d576e069ebb4850cac9b0dd5534da532dcb5",
            "CheckpointedAt": "0001-01-01T00:00:00Z",
            "RestoredAt": "2022-01-06T16:30:29.816895079-05:00",
            "Restored": true
        },
        "Image": "d2d8da4009e962d6507767b423fc22c9b0858e603a5a16e07d14e67c1ae612e6",
        "ImageName": "docker.io/yovfiatbeb/podman-criu-test:latest",
        "Rootfs": "",
        "Pod": "",
        "ResolvConfPath": "/run/containers/storage/overlay-containers/c8aa514b09f71efd85d033b2ff42d576e069ebb4850cac9b0dd5534da532dcb5/userdata/resolv.conf",
        "HostnamePath": "/run/containers/storage/overlay-containers/c8aa514b09f71efd85d033b2ff42d576e069ebb4850cac9b0dd5534da532dcb5/userdata/hostname",
        "HostsPath": "/run/containers/storage/overlay-containers/c8aa514b09f71efd85d033b2ff42d576e069ebb4850cac9b0dd5534da532dcb5/userdata/hosts",
        "StaticDir": "/var/lib/containers/storage/overlay-containers/c8aa514b09f71efd85d033b2ff42d576e069ebb4850cac9b0dd5534da532dcb5/userdata",
        "OCIConfigPath": "/var/lib/containers/storage/overlay-containers/c8aa514b09f71efd85d033b2ff42d576e069ebb4850cac9b0dd5534da532dcb5/userdata/config.json",
        "OCIRuntime": "runc",
        "ConmonPidFile": "/run/containers/storage/overlay-containers/c8aa514b09f71efd85d033b2ff42d576e069ebb4850cac9b0dd5534da532dcb5/userdata/conmon.pid",
        "PidFile": "/run/containers/storage/overlay-containers/c8aa514b09f71efd85d033b2ff42d576e069ebb4850cac9b0dd5534da532dcb5/userdata/pidfile",
        "Name": "upbeat_wing",
        "RestartCount": 0,
        "Driver": "overlay",
        "MountLabel": "",
        "ProcessLabel": "",
        "AppArmorProfile": "containers-default-0.46.1-dev",
        "EffectiveCaps": [
            "CAP_CHOWN",
            "CAP_DAC_OVERRIDE",
            "CAP_FOWNER",
            "CAP_FSETID",
            "CAP_KILL",
            "CAP_NET_BIND_SERVICE",
            "CAP_SETFCAP",
            "CAP_SETGID",
            "CAP_SETPCAP",
            "CAP_SETUID",
            "CAP_SYS_CHROOT"
        ],
        "BoundingCaps": [
            "CAP_CHOWN",
            "CAP_DAC_OVERRIDE",
            "CAP_FOWNER",
            "CAP_FSETID",
            "CAP_KILL",
            "CAP_NET_BIND_SERVICE",
            "CAP_SETFCAP",
            "CAP_SETGID",
            "CAP_SETPCAP",
            "CAP_SETUID",
            "CAP_SYS_CHROOT"
        ],
        "ExecIDs": [],
        "GraphDriver": {
            "Name": "overlay",
            "Data": {
                "LowerDir": "/var/lib/containers/storage/overlay/cb8186d1c41eb42ad1312f8ecd8ae627ce31b92048cfd4161119c0539098b7dd/diff:/var/lib/containers/storage/overlay/6e2ad7ce4afd38f77be88f4d7068eb64a68fcd3865c15d79e1534506af0a78e1/diff:/var/lib/containers/storage/overlay/70327e4209d7fa0826a8139e3d96486401b59ff9434f1409f3ac23d326099b3c/diff:/var/lib/containers/storage/overlay/e830d5464fbd3ac4e3148376c0336cdcaa5e53b7c139529ba502a411754b3c71/diff:/var/lib/containers/storage/overlay/ceca1844ed0eef892782eaaa083d96ee3a640401b0471f3076f6aed1756c89db/diff:/var/lib/containers/storage/overlay/958936231ad7350763c5530a2790b6545f172f58deb0b5d902c21470fa8c313f/diff:/var/lib/containers/storage/overlay/d476866719152ee2261125965c6c6f19476afcf51427ed2d8cfdb684bfe787f7/diff:/var/lib/containers/storage/overlay/71d7f6f303cc2ae20f2c4d37fdf2d5d4ca257d77d866a788ed512a2181dbf96c/diff:/var/lib/containers/storage/overlay/d4635607b48f709bed15be61b845664d550a1b7543ebe4ca9d0363672320c5e0/diff:/var/lib/containers/storage/overlay/3c122b418f7d6caae677790c0c8c14a33754eb48af02f15284406d031413e8bd/diff:/var/lib/containers/storage/overlay/cbb4d1e4b8e8a2eec8d59867ca01430117cf87fd53281bb119f1adccb44e38bb/diff:/var/lib/containers/storage/overlay/2896fbd2d1bcb261f6dec4de4847f1227ea270bfe12aaaaac336ace04a30f1dc/diff:/var/lib/containers/storage/overlay/8cdca145313fd5261d3e650868b7146de8da2fda06203d426d295b5fbaffa56e/diff:/var/lib/containers/storage/overlay/b28ef0b6fef80faa25436bec0a1375214d9a23a91e9b75975bba3b2889f8504f/diff",
                "MergedDir": "/var/lib/containers/storage/overlay/d1e517b3aa74046d559340c3e7e4153138fcc9016a3976d268f3a2518d71de09/merged",
                "UpperDir": "/var/lib/containers/storage/overlay/d1e517b3aa74046d559340c3e7e4153138fcc9016a3976d268f3a2518d71de09/diff",
                "WorkDir": "/var/lib/containers/storage/overlay/d1e517b3aa74046d559340c3e7e4153138fcc9016a3976d268f3a2518d71de09/work"
            }
        },
        "Mounts": [],
        "Dependencies": [],
        "NetworkSettings": {
            "EndpointID": "",
            "Gateway": "172.16.16.1",
            "IPAddress": "172.16.16.11",
            "IPPrefixLen": 24,
            "IPv6Gateway": "",
            "GlobalIPv6Address": "",
            "GlobalIPv6PrefixLen": 0,
            "MacAddress": "8a:72:5c:47:a0:93",
            "Bridge": "",
            "SandboxID": "",
            "HairpinMode": false,
            "LinkLocalIPv6Address": "",
            "LinkLocalIPv6PrefixLen": 0,
            "Ports": {
                "8080/tcp": null
            },
            "SandboxKey": "/run/netns/cni-6a4e0609-8f6e-5bfc-f984-b046f67cab35",
            "Networks": {
                "podman": {
                    "EndpointID": "",
                    "Gateway": "172.16.16.1",
                    "IPAddress": "172.16.16.11",
                    "IPPrefixLen": 24,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "8a:72:5c:47:a0:93",
                    "NetworkID": "podman",
                    "DriverOpts": null,
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": [
                        "c8aa514b09f7"
                    ]
                }
            }
        },
        "Namespace": "",
        "IsInfra": false,
        "Config": {
            "Hostname": "c8aa514b09f7",
            "Domainname": "",
            "User": "",
            "AttachStdin": false,
            "AttachStdout": false,
            "AttachStderr": false,
            "Tty": false,
            "OpenStdin": false,
            "StdinOnce": false,
            "Env": [
                "OPENSSL_VERSION=1.1.0f-3+deb9u2",
                "CATALINA_HOME=/usr/local/tomcat",
                "TERM=xterm",
                "JAVA_VERSION=8u181",
                "JAVA_DEBIAN_VERSION=8u181-b13-1~deb9u1",
                "TOMCAT_MAJOR=8",
                "TOMCAT_SHA512=131dfe23918f33fb24cefa7a03286c786304151f95f7bc0b6e34dfb6b0d1e65fe606e48b85c60c8a522938d1a01a36b540e69c94f36973321858e229731cda82",
                "TOMCAT_ASC_URLS=https://www.apache.org/dyn/closer.cgi?action=download\u0026filename=tomcat/tomcat-8/v8.5.34/bin/apache-tomcat-8.5.34.tar.gz.asc \thttps://www-us.apache.org/dist/tomcat/tomcat-8/v8.5.34/bin/apache-tomcat-8.5.34.tar.gz.asc \thttps://www.apache.org/dist/tomcat/tomcat-8/v8.5.34/bin/apache-tomcat-8.5.34.tar.gz.asc \thttps://archive.apache.org/dist/tomcat/tomcat-8/v8.5.34/bin/apache-tomcat-8.5.34.tar.gz.asc",
                "TOMCAT_VERSION=8.5.34",
                "PATH=/usr/local/tomcat/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
                "container=podman",
                "TOMCAT_NATIVE_LIBDIR=/usr/local/tomcat/native-jni-lib",
                "LD_LIBRARY_PATH=/usr/local/tomcat/native-jni-lib",
                "TOMCAT_TGZ_URLS=https://www.apache.org/dyn/closer.cgi?action=download\u0026filename=tomcat/tomcat-8/v8.5.34/bin/apache-tomcat-8.5.34.tar.gz \thttps://www-us.apache.org/dist/tomcat/tomcat-8/v8.5.34/bin/apache-tomcat-8.5.34.tar.gz \thttps://www.apache.org/dist/tomcat/tomcat-8/v8.5.34/bin/apache-tomcat-8.5.34.tar.gz \thttps://archive.apache.org/dist/tomcat/tomcat-8/v8.5.34/bin/apache-tomcat-8.5.34.tar.gz",
                "JAVA_HOME=/docker-java-home/jre",
                "LANG=C.UTF-8",
                "GPG_KEYS=05AB33110949707C93A279E3D3EFE6B686867BA6 07E48665A34DCAFAE522E5E6266191C37C037D42 47309207D818FFD8DCD3F83F1931D684307A10A5 541FBE7D8F78B25E055DDEE13C370389288584E7 61B832AC2F1C5A90F0F9B00A1C506407564C17A3 713DA88BE50911535FE716F5208B0AB1D63011C7 79F7026C690BAA50B92CD8B66A3AD3F4F22C4FED 9BA44C2621385CB966EBA586F72C284D731FABEE A27677289986DB50844682F8ACB77FC2E86E29AC A9C5DF4D22E99998D9875A5110C01C5A2F6059E7 DCFD35E0BF8CA7344752DE8B6FB21E8933C60243 F3A04C595DB5B6A5F1ECA43E3B7BBB100D811BBE F7DA48BB64BCB84ECBA7EE6935CD23C10D498E23",
                "CA_CERTIFICATES_JAVA_VERSION=20170531+nmu1",
                "HOME=/root",
                "HOSTNAME=c8aa514b09f7"
            ],
            "Cmd": [
                "catalina.sh",
                "run"
            ],
            "Image": "docker.io/yovfiatbeb/podman-criu-test:latest",
            "Volumes": null,
            "WorkingDir": "/usr/local/tomcat",
            "Entrypoint": "",
            "OnBuild": null,
            "Labels": null,
            "Annotations": {
                "io.container.manager": "libpod",
                "io.kubernetes.cri-o.Created": "2022-01-06T16:29:05.351545832-05:00",
                "io.kubernetes.cri-o.TTY": "false",
                "io.podman.annotations.autoremove": "FALSE",
                "io.podman.annotations.init": "FALSE",
                "io.podman.annotations.privileged": "FALSE",
                "io.podman.annotations.publish-all": "FALSE",
                "org.opencontainers.image.stopSignal": "15"
            },
            "StopSignal": 15,
            "CreateCommand": [
                "podman",
                "--log-level",
                "debug",
                "run",
                "--tmpfs",
                "/tmp",
                "--tmpfs",
                "/usr/local/tomcat/logs",
                "-d",
                "docker://docker.io/yovfiatbeb/podman-criu-test"
            ],
            "Umask": "0022",
            "Timeout": 0,
            "StopTimeout": 10,
            "Passwd": true
        },
        "HostConfig": {
            "Binds": [],
            "CgroupManager": "cgroupfs",
            "CgroupMode": "private",
            "ContainerIDFile": "",
            "LogConfig": {
                "Type": "k8s-file",
                "Config": null,
                "Path": "/var/lib/containers/storage/overlay-containers/c8aa514b09f71efd85d033b2ff42d576e069ebb4850cac9b0dd5534da532dcb5/userdata/ctr.log",
                "Tag": "",
                "Size": "0B"
            },
            "NetworkMode": "bridge",
            "PortBindings": {
                "8080/tcp": null
            },
            "RestartPolicy": {
                "Name": "",
                "MaximumRetryCount": 0
            },
            "AutoRemove": false,
            "VolumeDriver": "",
            "VolumesFrom": null,
            "CapAdd": [],
            "CapDrop": [
                "CAP_AUDIT_WRITE",
                "CAP_MKNOD",
                "CAP_NET_RAW"
            ],
            "Dns": [],
            "DnsOptions": [],
            "DnsSearch": [],
            "ExtraHosts": [],
            "GroupAdd": [],
            "IpcMode": "private",
            "Cgroup": "",
            "Cgroups": "default",
            "Links": null,
            "OomScoreAdj": 0,
            "PidMode": "private",
            "Privileged": false,
            "PublishAllPorts": false,
            "ReadonlyRootfs": false,
            "SecurityOpt": [],
            "Tmpfs": {
                "/tmp": "rw,rprivate,nosuid,nodev,tmpcopyup",
                "/usr/local/tomcat/logs": "rw,rprivate,nosuid,nodev,tmpcopyup"
            },
            "UTSMode": "private",
            "UsernsMode": "",
            "ShmSize": 65536000,
            "Runtime": "oci",
            "ConsoleSize": [
                0,
                0
            ],
            "Isolation": "",
            "CpuShares": 0,
            "Memory": 0,
            "NanoCpus": 0,
            "CgroupParent": "",
            "BlkioWeight": 0,
            "BlkioWeightDevice": null,
            "BlkioDeviceReadBps": null,
            "BlkioDeviceWriteBps": null,
            "BlkioDeviceReadIOps": null,
            "BlkioDeviceWriteIOps": null,
            "CpuPeriod": 0,
            "CpuQuota": 0,
            "CpuRealtimePeriod": 0,
            "CpuRealtimeRuntime": 0,
            "CpusetCpus": "",
            "CpusetMems": "",
            "Devices": [],
            "DiskQuota": 0,
            "KernelMemory": 0,
            "MemoryReservation": 0,
            "MemorySwap": 0,
            "MemorySwappiness": 0,
            "OomKillDisable": false,
            "PidsLimit": 2048,
            "Ulimits": [
                {
                    "Name": "RLIMIT_NOFILE",
                    "Soft": 1048576,
                    "Hard": 1048576
                },
                {
                    "Name": "RLIMIT_NPROC",
                    "Soft": 4194304,
                    "Hard": 4194304
                }
            ],
            "CpuCount": 0,
            "CpuPercent": 0,
            "IOMaximumIOps": 0,
            "IOMaximumBandwidth": 0,
            "CgroupConf": null
        }
    }
]

root@Experience:/home/thelo/proj/podman-scratch# podman network inspect podman
[
    {
        "name": "podman",
        "id": "2f259bab93aaaaa2542ba43ef33eb990d0999ee1b9924b557b7be53c0b7a1bb9",
        "driver": "ptp",
        "created": "2021-12-16T20:01:27.217177016-05:00",
        "ipv6_enabled": false,
        "internal": false,
        "dns_enabled": false
    }
]

@Luap99
Copy link
Member

Luap99 commented Jan 7, 2022

It looks like you changed your default CNI config file to a driver we do not officially support. Is there any specific reason you do not use the default bridge?

Anyway I see the problem now. The network does not contain a subnet (I assume you cni config contains the dhcp ipam plugin) and we try to check if the ip is in a given subnet but since there are subnets this check fails.

I think we must only do this check if the local ipam driver is used.

@Luap99 Luap99 self-assigned this Jan 7, 2022
@TheloTheGreat
Copy link
Author

I did not intentionally change any CNI config file. In fact I don't even know what a CNI config file is or where it is. I definitely do want to use "the default bridge", whatever it is. How can I do that?

The closest thing might be that I previously got an older version of podman from the Ubuntu repositories, before uninstalling it and building podman from source. Would that have left over any CNI junk?

@Luap99
Copy link
Member

Luap99 commented Jan 7, 2022

Check the files in /etc/cni/net.d I recommend to use the default config from here: https://github.com/containers/podman/blob/main/cni/87-podman-bridge.conflist

If you never changed that then I have no idea were it came from. It might be worth to check if this is/was shipped by the debian package.

@TheloTheGreat
Copy link
Author

Thanks, after replacing my existing /etc/cni/net.d/87-podman-ptp.conflist with that one you linked above, my repro steps above do allow checkpointing to work as expected.

FYI, my previous 87-podman-ptp.conflist looked like this:

{
  "cniVersion": "0.4.0",
  "name": "podman",
  "plugins": [
    {
        "type": "ptp",
            "Documentation": "/usr/share/doc/containernetworking-plugins/main_ptp.md",
        "ipMasq": true,
        "ipam": {
            "type": "host-local",
                "Documentation": "/usr/share/doc/containernetworking-plugins/ipam_host-local.md",
            "subnet": "172.16.16.0/24",
            "routes": [
                { "dst": "0.0.0.0/0" }
            ]
        }
    },

    {
        "type": "portmap",
            "Documentation": "/usr/share/doc/containernetworking-plugins/meta_portmap.md",
        "capabilities": { "portMappings": true }
    },

    {
        "type": "firewall",
            "Documentation": "/usr/share/doc/containernetworking-plugins/meta_firewall.md",
        "backend": "iptables"
    }
  ]
}

@TheloTheGreat
Copy link
Author

Looking it up, yeah that file I pasted above came directly from the current Ubuntu podman package, podman_3.2.1+ds1-2ubuntu3_amd64.deb , the one we can see here: https://packages.ubuntu.com/impish/amd64/podman

@rhatdan
Copy link
Member

rhatdan commented Jan 7, 2022

Could yo uopen an issue with them?

@rhatdan rhatdan closed this as completed Jan 7, 2022
@Luap99
Copy link
Member

Luap99 commented Jan 9, 2022

This is still an issue with the new network code, we should not error in this case.

@Luap99 Luap99 reopened this Jan 9, 2022
@Luap99
Copy link
Member

Luap99 commented Jan 9, 2022

It also looks like this was already changed in the debian repo: https://salsa.debian.org/debian/libpod/-/commit/39723afc6b6d8067f63df6a2741cf71a0ce71dee

Luap99 added a commit to Luap99/common that referenced this issue Jan 10, 2022
@Luap99
libnetwork: only validate static ip when ipam is host-local
abdcf76 
If the hdcp ipam driver is used podman does not know any subnets so we
cannot verify if the guven static ip is in the subnet.

Fixes containers/podman#12762

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Luap99 added a commit to Luap99/common that referenced this issue Jan 10, 2022
@Luap99
libnetwork: only validate static ip when ipam is host-local
0a0bc1c 
If the dhcp ipam driver is used podman does not know any subnets so we
cannot verify if the given static ip is in the subnet.

Fixes containers/podman#12762

Signed-off-by: Paul Holzinger <pholzing@redhat.com>
@Luap99 Luap99 mentioned this issue Jan 10, 2022
Merged
@github-actions github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 21, 2023
@github-actions github-actions bot locked as resolved and limited conversation to collaborators Sep 21, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@Luap99 Luap99

Labels
kind/bug Categorizes issue or PR as related to a bug. locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments.
Projects
None yet
Milestone
No milestone
3 participants
@rhatdan @TheloTheGreat @Luap99