service restart doesn't connect to memlogd system

[stijnopheide]

Description

When enabling memlogd through the logging example service restarts don't connect the output of the restarted service to the memlogd system. The issue is that the pipes in /var/run are not being removed when the service is stopped.

Context: we are implementing a health check container that can restart services in case of failure. This all works with standard ctr until we enabled memlogd. After looking at the code for the memlogd package and how it integrates with linuxkit services, we came to the conclusion that probably using /usr/bin/service is the way to go. If this is not true, than maybe this issue might not be a valid one and there is another way of managing service / task restarts.

Steps to reproduce the issue:

Change the logging.yml example to include a bind that adds /usr/bin/service in the getty container:

# Simple example of using an external logging service
kernel:
  image: linuxkit/kernel:5.10.104
  cmdline: "console=tty0 console=ttyS0 console=ttyAMA0"
init:
  - linuxkit/init:14df799bb3b9e0eb0491da9fda7f32a108a2e2a5
  - linuxkit/runc:436357ce16dd663e24f595bcec26d5ae476c998e
  - linuxkit/containerd:eeb3aaf497c0b3f6c67f3a245d61ea5a568ca718
  - linuxkit/ca-certificates:4de36e93dc87f7ccebd20db616ed10d381911d32
  - linuxkit/memlogd:67e3bc9d0f271336567f4bffbb80a8d57d5eddba
onboot:
  - name: sysctl
    image: linuxkit/sysctl:e5959517fab7b44692ad63941eecf37486e73799
  - name: dhcpcd
    image: linuxkit/dhcpcd:2a8ed08fea442909ba10f950d458191ed3647115
    command: ["/sbin/dhcpcd", "--nobackground", "-f", "/dhcpcd.conf", "-1"]
services:
# Inside the getty type `/proc/1/root/usr/bin/logread -F` to follow the log
  - name: getty
    image: linuxkit/getty:06f34bce0facea79161566d67345c3ea49965437
    binds.add:
      - /usr/bin/service:/usr/bin/service
    env:
     - INSECURE=true
# A service which generates log messages for testing
  - name: write-to-the-logs
    image: alpine:3.13
    command: ["/bin/sh", "-c", "while /bin/true; do echo hello $(date); sleep 1; done" ]
  - name: write-and-rotate-logs
    image: linuxkit/logwrite:f3ba46a1785036931bd156996659e967b594bb83
  - name: kmsg
    image: linuxkit/kmsg:d86e8b0b44a01195f5ca644adb25994a24703f0f

linuxkit build logging.yml
linuxkit run logging

in the getty container

service stop write-to-the-logs
service start write-to-the-logs
tail -f /var/log/write-to-the-logs.out.log

Describe the results you received:

The tail command does not display new log entries created by the write-to-the-logs service.

Describe the results you expected:

The tail command shows log entries being added to the file.

Additional information you deem important (e.g. issue happens only occasionally):

You can work around the issue by deleting the pipes for the service in /var/run. The following set of commands work:

service stop write-to-the-logs
service start write-to-the-logs
rm /var/run/write-to-the-logs.*
tail -f /var/log/write-to-the-logs.out.log