Error starting a container with journald driver

[linggao]

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description
Got the "Error: write child: broken pipe" when starting a container with journald log driver

Steps to reproduce the issue:

1. podman run --name test --network foo-a --log-driver journald -d alpine sleep 1000

Describe the results you received:
podman run --name test --network foo-a --log-driver journald -d alpine sleep 1000
[conmon:e] Include journald in compilation path to log to systemd journal
Error: write child: broken pipe

Describe the results you expected:

Additional information you deem important (e.g. issue happens only occasionally):

Output of podman version:

Version:      3.0.1-dev
API Version:  3.0.0
Go Version:   go1.15.7
Built:        Tue Feb 16 06:47:41 2021
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.19.2
  cgroupManager: systemd
  cgroupVersion: v1
  conmon:
    package: Unknown
    path: /usr/local/libexec/podman/conmon
    version: 'conmon version 2.0.27-dev, commit: 7310bf13319ee8ed50799b202509bedc27b36cf8'
  cpus: 2
  distribution:
    distribution: '"rhel"'
    version: "8.3"
  eventLogger: file
  hostname: lingvs1.dev.edge-fabric.com
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 4.18.0-240.15.1.el8_3.x86_64
  linkmode: dynamic
  memFree: 5325225984
  memTotal: 8342462464
  ociRuntime:
    name: runc
    package: runc-1.0.0-70.rc92.module+el8.4.0+9980+44630550.x86_64
    path: /usr/bin/runc
    version: 'runc version spec: 1.0.2-dev'
  os: linux
  remoteSocket:
    exists: true
    path: /run/podman/podman.sock
  security:
    apparmorEnabled: false
    capabilities: CAP_NET_RAW,CAP_CHOWN,CAP_DAC_OVERRIDE,CAP_FOWNER,CAP_FSETID,CAP_KILL,CAP_NET_BIND_SERVICE,CAP_SETFCAP,CAP_SETGID,CAP_SETPCAP,CAP_SETUID,CAP_SYS_CHROOT
    rootless: false
    seccompEnabled: true
    selinuxEnabled: true
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 2146758656
  swapTotal: 2146758656
  uptime: 109h 5m 35.06s (Approximately 4.54 days)
registries:
  search:
  - registry.access.redhat.com
  - registry.redhat.io
  - docker.io
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 1
    paused: 0
    running: 0
    stopped: 1
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageStore:
    number: 7
  runRoot: /var/run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 3.0.0
  Built: 1613479661
  BuiltTime: Tue Feb 16 06:47:41 2021
  GitCommit: ""
  GoVersion: go1.15.7
  OsArch: linux/amd64
  Version: 3.0.1-dev

Package info (e.g. output of rpm -q podman or apt list podman):

podman-3.0.0-2.module+el8.4.0+9980+44630550.x86_64

Have you tested with the latest version of Podman and have you checked the Podman Troubleshooting Guide?

Yes

Additional environment details (AWS, VirtualBox, physical, etc.):

[mheon]

Can I ask what distro you're seeing this on? It looks like Conmon was built with incorrect tags that left journald support out.

[vrothberg]

It looks like a custom build of Podman and Conmon on RHEL 8.3.

As @mheon mentioned, Conmon was built without journald support. Building Conmon with that will resolve the issue.

Note that you can use CentOS Stream if you want to have a look at the next version of Podman in RHEL. Podman 3.0 will be released with RHEL 8.4 but can already be tested in CentOS 8 Stream.

[linggao]

@vrothberg @mheon I installed systemd-devel and rebult Conmon and podman, now the journald log driver is working. My question is: Is conmon that you ship usually have the jorunald support? We are going to use journald as the default log driver for contianers, just wonder what is the installation requirement.

[mheon]

Yes, any Conmon we package will include support by default.