[Regression on main] Segfaults at runtime/restart/monitor/monitor.go:226

[AkihiroSuda]

Description

The restart monitor segfaults during restarting the daemon.
Seems a regression in #6744

# journalctl -u containerd -o cat -f
containerd.service: Scheduled restart job, restart counter is at 19.
Stopped containerd container runtime.
Starting containerd container runtime...
time="2022-04-20T15:30:40+09:00" level=warning msg="deprecated version : `1`, please switch to version `2`"
time="2022-04-20T15:30:40.197183901+09:00" level=info msg="starting containerd" revision=fe7f0aeef42b9cd8c5b0c45f889bbbf9ad766ab2 version=v1.6.0-276-gfe7f0aeef
time="2022-04-20T15:30:40.213830418+09:00" level=info msg="loading plugin \"io.containerd.content.v1.content\"..." type=io.containerd.content.v1
time="2022-04-20T15:30:40.213929276+09:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.aufs\"..." type=io.containerd.snapshotter.v1
time="2022-04-20T15:30:40.215907767+09:00" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.aufs\"..." error="aufs is not supported (modprobe aufs failed: exit status 1 \"modprobe: FATAL: Module aufs not found in directory /lib/modules/5.13.0-39-generic\\n\"): skip plugin" type=io.containerd.snapshotter.v1
time="2022-04-20T15:30:40.215941253+09:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.btrfs\"..." type=io.containerd.snapshotter.v1
time="2022-04-20T15:30:40.216112999+09:00" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.btrfs\"..." error="path /var/lib/containerd/io.containerd.snapshotter.v1.btrfs (ext4) must be a btrfs filesystem to be used with the btrfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
time="2022-04-20T15:30:40.216126889+09:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.devmapper\"..." type=io.containerd.snapshotter.v1
time="2022-04-20T15:30:40.216136520+09:00" level=warning msg="failed to load plugin io.containerd.snapshotter.v1.devmapper" error="devmapper not configured"
time="2022-04-20T15:30:40.216145048+09:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.native\"..." type=io.containerd.snapshotter.v1
time="2022-04-20T15:30:40.216165051+09:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.overlayfs\"..." type=io.containerd.snapshotter.v1
time="2022-04-20T15:30:40.216286150+09:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.zfs\"..." type=io.containerd.snapshotter.v1
time="2022-04-20T15:30:40.216400450+09:00" level=info msg="skip loading plugin \"io.containerd.snapshotter.v1.zfs\"..." error="path /var/lib/containerd/io.containerd.snapshotter.v1.zfs must be a zfs filesystem to be used with the zfs snapshotter: skip plugin" type=io.containerd.snapshotter.v1
time="2022-04-20T15:30:40.216424586+09:00" level=info msg="loading plugin \"io.containerd.snapshotter.v1.stargz\"..." type=io.containerd.snapshotter.v1
time="2022-04-20T15:30:40.216523275+09:00" level=info msg="loading plugin \"io.containerd.metadata.v1.bolt\"..." type=io.containerd.metadata.v1
time="2022-04-20T15:30:40.216544861+09:00" level=warning msg="could not use snapshotter devmapper in metadata plugin" error="devmapper not configured"
time="2022-04-20T15:30:40.216554152+09:00" level=info msg="metadata content store policy set" policy=shared
time="2022-04-20T15:30:40.216683336+09:00" level=info msg="loading plugin \"io.containerd.differ.v1.walking\"..." type=io.containerd.differ.v1
time="2022-04-20T15:30:40.216705589+09:00" level=info msg="loading plugin \"io.containerd.event.v1.exchange\"..." type=io.containerd.event.v1
time="2022-04-20T15:30:40.216714500+09:00" level=info msg="loading plugin \"io.containerd.gc.v1.scheduler\"..." type=io.containerd.gc.v1
time="2022-04-20T15:30:40.216734656+09:00" level=info msg="loading plugin \"io.containerd.lease.v1.manager\"..." type=io.containerd.lease.v1
time="2022-04-20T15:30:40.216744862+09:00" level=info msg="loading plugin \"io.containerd.service.v1.introspection-service\"..." type=io.containerd.service.v1
time="2022-04-20T15:30:40.216757520+09:00" level=info msg="loading plugin \"io.containerd.service.v1.containers-service\"..." type=io.containerd.service.v1
time="2022-04-20T15:30:40.216767736+09:00" level=info msg="loading plugin \"io.containerd.service.v1.content-service\"..." type=io.containerd.service.v1
time="2022-04-20T15:30:40.216777474+09:00" level=info msg="loading plugin \"io.containerd.service.v1.diff-service\"..." type=io.containerd.service.v1
time="2022-04-20T15:30:40.216787070+09:00" level=info msg="loading plugin \"io.containerd.service.v1.images-service\"..." type=io.containerd.service.v1
time="2022-04-20T15:30:40.216798446+09:00" level=info msg="loading plugin \"io.containerd.service.v1.namespaces-service\"..." type=io.containerd.service.v1
time="2022-04-20T15:30:40.216808224+09:00" level=info msg="loading plugin \"io.containerd.runtime.v2.task\"..." type=io.containerd.runtime.v2
time="2022-04-20T15:30:40.216867167+09:00" level=info msg="loading plugin \"io.containerd.runtime.v2.shim\"..." type=io.containerd.runtime.v2
time="2022-04-20T15:30:40.216877084+09:00" level=info msg="loading plugin \"io.containerd.service.v1.sandbox-controller-service\"..." type=io.containerd.service.v1
time="2022-04-20T15:30:40.216885808+09:00" level=info msg="loading plugin \"io.containerd.service.v1.sandbox-store-service\"..." type=io.containerd.service.v1
time="2022-04-20T15:30:40.216893933+09:00" level=info msg="loading plugin \"io.containerd.service.v1.snapshots-service\"..." type=io.containerd.service.v1
time="2022-04-20T15:30:40.216903377+09:00" level=info msg="loading plugin \"io.containerd.runtime.v1.linux\"..." type=io.containerd.runtime.v1
time="2022-04-20T15:30:40.216929797+09:00" level=info msg="loading plugin \"io.containerd.monitor.v1.cgroups\"..." type=io.containerd.monitor.v1
time="2022-04-20T15:30:40.217188514+09:00" level=info msg="loading plugin \"io.containerd.service.v1.tasks-service\"..." type=io.containerd.service.v1
time="2022-04-20T15:30:40.217213216+09:00" level=info msg="loading plugin \"io.containerd.grpc.v1.introspection\"..." type=io.containerd.grpc.v1
time="2022-04-20T15:30:40.217222681+09:00" level=info msg="loading plugin \"io.containerd.internal.v1.restart\"..." type=io.containerd.internal.v1
time="2022-04-20T15:30:40.217271210+09:00" level=info msg="loading plugin \"io.containerd.grpc.v1.containers\"..." type=io.containerd.grpc.v1
time="2022-04-20T15:30:40.217280999+09:00" level=info msg="loading plugin \"io.containerd.grpc.v1.content\"..." type=io.containerd.grpc.v1
time="2022-04-20T15:30:40.217290514+09:00" level=info msg="loading plugin \"io.containerd.grpc.v1.diff\"..." type=io.containerd.grpc.v1
time="2022-04-20T15:30:40.217297809+09:00" level=info msg="loading plugin \"io.containerd.grpc.v1.events\"..." type=io.containerd.grpc.v1
time="2022-04-20T15:30:40.217305978+09:00" level=info msg="loading plugin \"io.containerd.grpc.v1.healthcheck\"..." type=io.containerd.grpc.v1
time="2022-04-20T15:30:40.217314295+09:00" level=info msg="loading plugin \"io.containerd.grpc.v1.images\"..." type=io.containerd.grpc.v1
time="2022-04-20T15:30:40.217322018+09:00" level=info msg="loading plugin \"io.containerd.grpc.v1.leases\"..." type=io.containerd.grpc.v1
time="2022-04-20T15:30:40.217329491+09:00" level=info msg="loading plugin \"io.containerd.grpc.v1.namespaces\"..." type=io.containerd.grpc.v1
time="2022-04-20T15:30:40.217341320+09:00" level=info msg="loading plugin \"io.containerd.internal.v1.opt\"..." type=io.containerd.internal.v1
time="2022-04-20T15:30:40.217378328+09:00" level=info msg="loading plugin \"io.containerd.grpc.v1.sandbox-controllers\"..." type=io.containerd.grpc.v1
time="2022-04-20T15:30:40.217391110+09:00" level=info msg="loading plugin \"io.containerd.grpc.v1.sandboxes\"..." type=io.containerd.grpc.v1
time="2022-04-20T15:30:40.217399800+09:00" level=info msg="loading plugin \"io.containerd.grpc.v1.snapshots\"..." type=io.containerd.grpc.v1
time="2022-04-20T15:30:40.217408926+09:00" level=info msg="loading plugin \"io.containerd.grpc.v1.tasks\"..." type=io.containerd.grpc.v1
time="2022-04-20T15:30:40.217417179+09:00" level=info msg="loading plugin \"io.containerd.grpc.v1.version\"..." type=io.containerd.grpc.v1
time="2022-04-20T15:30:40.217425356+09:00" level=info msg="loading plugin \"io.containerd.grpc.v1.cri\"..." type=io.containerd.grpc.v1
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x98 pc=0x56380058ee32]
goroutine 27 [running]:
github.com/containerd/containerd/runtime/restart/monitor.(*monitor).monitor(0xc000010228, {0x56380169c2e8, 0xc0007ae0c0})
        /home/suda/gopath/src/github.com/containerd/containerd/runtime/restart/monitor/monitor.go:226 +0x272
github.com/containerd/containerd/runtime/restart/monitor.(*monitor).reconcile.func1()
        /home/suda/gopath/src/github.com/containerd/containerd/runtime/restart/monitor/monitor.go:187 +0xab
created by github.com/containerd/containerd/runtime/restart/monitor.(*monitor).reconcile
        /home/suda/gopath/src/github.com/containerd/containerd/runtime/restart/monitor/monitor.go:184 +0x18c
containerd.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
containerd.service: Failed with result 'exit-code'.
Failed to start containerd container runtime.

Steps to reproduce the issue

# nerdctl version
Client:
 Version:       v0.18.0
 Git commit:    77276ff0fffad3f855ab9f2f5a4ad5527ef76485

Server:
 containerd:
  Version:      v1.6.0-276-gfe7f0aeef
  GitCommit:    fe7f0aeef42b9cd8c5b0c45f889bbbf9ad766ab2

# nerdctl run -d --restart=always --name nginx -p 80:80 nginx:alpine
cd753d109812f9bb06f609db858e32cea1c01bc4ed72382709db4d5f9a9195f2

# systemctl kill -s KILL containerd

# journalctl -u containerd -o cat -f
...
panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x98 pc=0x56380058ee32]
goroutine 27 [running]:
github.com/containerd/containerd/runtime/restart/monitor.(*monitor).monitor(0xc000010228, {0x56380169c2e8, 0xc0007ae0c0})
        /home/suda/gopath/src/github.com/containerd/containerd/runtime/restart/monitor/monitor.go:226 +0x272
github.com/containerd/containerd/runtime/restart/monitor.(*monitor).reconcile.func1()
        /home/suda/gopath/src/github.com/containerd/containerd/runtime/restart/monitor/monitor.go:187 +0xab
created by github.com/containerd/containerd/runtime/restart/monitor.(*monitor).reconcile
        /home/suda/gopath/src/github.com/containerd/containerd/runtime/restart/monitor/monitor.go:184 +0x18c
containerd.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
containerd.service: Failed with result 'exit-code'.
Failed to start containerd container runtime.

Describe the results you received and expected

Received: segfault
Expected: shout not segfault

What version of containerd are you using?

containerd github.com/containerd/containerd v1.6.0-276-gfe7f0aeef fe7f0ae

Any other relevant information

No response

Show configuration if it is related to CRI plugin.

No response