podman exec commands use log-driver from podman run #6555

goochjj · 2020-06-10T16:04:15Z

Is this a BUG REPORT or FEATURE REQUEST? (leave only one on its own line)

/kind bug

Description

When starting a container with a --log-driver journald, it logs all stdout/err to journald as expect.

What is NOT expected is that when doing a podman exec -it $container bash, all stdin/out is ALSO logged.

Steps to reproduce the issue:

podman run -d --log-driver journald alpine sleep infinity
journalctl -f &
podman exec -it -l sh
type ls and hit enter.

Describe the results you received:
Jun 10 15:56:12 photon-machine conmon[12093]: [8B blob data]
Jun 10 15:56:12 photon-machine conmon[12093]: [8B blob data]
Jun 10 15:56:13 photon-machine conmon[12093]: l
Jun 10 15:56:13 photon-machine conmon[12093]: s
Jun 10 15:56:14 photon-machine conmon[12093]: [2B blob data]
Jun 10 15:56:14 photon-machine conmon[12093]: [117B blob data]
Jun 10 15:56:14 photon-machine conmon[12093]: [8B blob data]

Describe the results you expected:

No data in journald. It's an interactive session.

Additional information you deem important (e.g. issue happens only occasionally):

In docker, log drivers only affect the logs from the container's parent process, i.e. /proc/1/fd/1 and 2, as passed up into the docker daemon. In this case conmon fills that role, and that all works fine.

Docker exec has no concept of log-driver because it's not passing output through dockerd. podman exec DOES spawn a new copy of conmon, and preserves the log-driver from the container initialization, which is not desired nor expected.

Since podman does allow exec -d (which docker doesn't, because it wouldn't make sense w/ their model), log-driver should probably continue to work on podman exec but should have to be explicitly supplied. i.e.

podman run --log-driver determines the log driver for the container's init output. NOT ALL container output.

podman exec --log-driver determines the log driver for THIS command's output. In most cases, if using -i or -it, you'd want that to be "none", not whatever was used to run the container.

log-level debug shows it:
running conmon: /usr/local/libexec/podman/conmon args="[--api-version 1 -c e6fc4ef4ccab59064a4de35ff68818612041cb90c892cd30a2c8d0b0e41673db -u f5452b69ee19b32e675476be605d8856db03f8dfaa6f1ab1235e70516068572b -r /usr/bin/runc -b /var/lib/containers/storage/overlay-containers/e6fc4ef4ccab59064a4de35ff68818612041cb90c892cd30a2c8d0b0e41673db/userdata/f5452b69ee19b32e675476be605d8856db03f8dfaa6f1ab1235e70516068572b -p /var/lib/containers/storage/overlay-containers/e6fc4ef4ccab59064a4de35ff68818612041cb90c892cd30a2c8d0b0e41673db/userdata/f5452b69ee19b32e675476be605d8856db03f8dfaa6f1ab1235e70516068572b/exec_pid -n friendly_mirzakhani --exit-dir /var/lib/containers/storage/overlay-containers/e6fc4ef4ccab59064a4de35ff68818612041cb90c892cd30a2c8d0b0e41673db/userdata/f5452b69ee19b32e675476be605d8856db03f8dfaa6f1ab1235e70516068572b/exit --socket-dir-path /var/run/libpod/socket -s -l journald --log-level debug --syslog -t -i -e --exec-attach --exec-process-spec /var/lib/containers/storage/overlay-containers/e6fc4ef4ccab59064a4de35ff68818612041cb90c892cd30a2c8d0b0e41673db/userdata/f5452b69ee19b32e675476be605d8856db03f8dfaa6f1ab1235e70516068572b/exec-process-811522033]"

even though my command line was:
podman --log-level debug exec -itl sh

It must have inherited the -l journald from somewhere, i.e. the container run line. I've verified if I do NOT set journald on the container run line, the command line is different.

DEBU[0000] running conmon: /usr/local/libexec/podman/conmon args="[--api-version 1 -c 5146c02a71efe886446522509d0ab616f3cb5aafd523b591a44d64c0507da4f1 -u d6cfc85f7b9f582dcf11b478dbb3d72954fe7bec4266f604ac83d983c0faf891 -r /usr/bin/runc -b /var/lib/containers/storage/overlay-containers/5146c02a71efe886446522509d0ab616f3cb5aafd523b591a44d64c0507da4f1/userdata/d6cfc85f7b9f582dcf11b478dbb3d72954fe7bec4266f604ac83d983c0faf891 -p /var/lib/containers/storage/overlay-containers/5146c02a71efe886446522509d0ab616f3cb5aafd523b591a44d64c0507da4f1/userdata/d6cfc85f7b9f582dcf11b478dbb3d72954fe7bec4266f604ac83d983c0faf891/exec_pid -n nice_galileo --exit-dir /var/lib/containers/storage/overlay-containers/5146c02a71efe886446522509d0ab616f3cb5aafd523b591a44d64c0507da4f1/userdata/d6cfc85f7b9f582dcf11b478dbb3d72954fe7bec4266f604ac83d983c0faf891/exit --socket-dir-path /var/run/libpod/socket -s -l k8s-file:/var/lib/containers/storage/overlay-containers/5146c02a71efe886446522509d0ab616f3cb5aafd523b591a44d64c0507da4f1/userdata/d6cfc85f7b9f582dcf11b478dbb3d72954fe7bec4266f604ac83d983c0faf891/exec_log --log-level debug --syslog -t -i -e --exec-attach --exec-process-spec /var/lib/containers/storage/overlay-containers/5146c02a71efe886446522509d0ab616f3cb5aafd523b591a44d64c0507da4f1/userdata/d6cfc85f7b9f582dcf11b478dbb3d72954fe7bec4266f604ac83d983c0faf891/exec-process-709543616]"

Specifically the -l journald option is not passed through.

Output of podman version:

Version:      2.0.0-dev
API Version:  1
Go Version:   go1.13.3
Git Commit:   e8818ced806910f8ce44939fef08f89139be4119
Built:        Thu May 28 23:38:50 2020
OS/Arch:      linux/amd64

Output of podman info --debug:

host:
  arch: amd64
  buildahVersion: 1.15.0-dev
  cgroupVersion: v1
  conmon:
    package: Unknown
    path: /usr/local/libexec/podman/conmon
    version: 'conmon version 2.0.18-dev, commit: d0f367daacf6cb19c605acdd4880ecc243bf26d9'
  cpus: 1
  distribution:
    distribution: photon
    version: "3.0"
  eventLogger: file
  hostname: photon-machine
  idMappings:
    gidmap: null
    uidmap: null
  kernel: 4.19.115-6.ph3-esx
  linkmode: dynamic
  memFree: 146153472
  memTotal: 2095681536
  ociRuntime:
    name: runc
    package: runc-1.0.0.rc9-2.ph3.x86_64
    path: /usr/bin/runc
    version: |-
      runc version 1.0.0-rc9
      commit: d736ef14f0288d6993a1845745d6756cfc9ddd5a
      spec: 1.0.1-dev
  os: linux
  rootless: false
  slirp4netns:
    executable: ""
    package: ""
    version: ""
  swapFree: 0
  swapTotal: 0
  uptime: 141h 1m 54.92s (Approximately 5.88 days)
registries:
  search:
  - docker.io
  - registry.fedoraproject.org
  - registry.access.redhat.com
store:
  configFile: /etc/containers/storage.conf
  containerStore:
    number: 13
    paused: 0
    running: 1
    stopped: 12
  graphDriverName: overlay
  graphOptions:
    overlay.mountopt: nodev,metacopy=on
  graphRoot: /var/lib/containers/storage
  graphStatus:
    Backing Filesystem: extfs
    Native Overlay Diff: "false"
    Supports d_type: "true"
    Using metacopy: "true"
  imageStore:
    number: 4
  runRoot: /var/run/containers/storage
  volumePath: /var/lib/containers/storage/volumes
version:
  APIVersion: 1
  Built: 1590709130
  BuiltTime: Thu May 28 23:38:50 2020
  GitCommit: e8818ced806910f8ce44939fef08f89139be4119
  GoVersion: go1.13.3
  OsArch: linux/amd64
  Version: 2.0.0-dev

Package info (e.g. output of rpm -q podman or apt list podman):

Compiled from source.

The text was updated successfully, but these errors were encountered:

goochjj · 2020-06-10T16:08:58Z

Ideally podman exec would support --log-driver and allow it to be specified. Even if I could do --log-driver none, it would be better than what it does now.

Preserving from the "run" command makes podman exec pretty useless when using a log driver.

mheon · 2020-06-10T17:25:57Z

I'll take this one

mheon · 2020-06-10T17:45:23Z

For now, I'm going to force log-driver off for podman exec; we can add the flag at a later date

When the container uses journald logging, we don't want to automatically use the same driver for its exec sessions. If we do we will pollute the journal (particularly in the case of healthchecks) with large amounts of undesired logs. Instead, force exec sessions logs to file for now; we can add a log-driver flag later (we'll probably want to add a `podman logs` command that reads exec session logs at the same time). Fixes containers#6555 Signed-off-by: Matthew Heon <matthew.heon@pm.me>

When the container uses journald logging, we don't want to automatically use the same driver for its exec sessions. If we do we will pollute the journal (particularly in the case of healthchecks) with large amounts of undesired logs. Instead, force exec sessions logs to file for now; we can add a log-driver flag later (we'll probably want to add a `podman logs` command that reads exec session logs at the same time). Great thanks to Joe Gooch (mrwizard@dok.org) for adding support to Conmon for a null log driver, and wiring it in here. Fixes containers#6555 Signed-off-by: Matthew Heon <matthew.heon@pm.me>

When the container uses journald logging, we don't want to automatically use the same driver for its exec sessions. If we do we will pollute the journal (particularly in the case of healthchecks) with large amounts of undesired logs. Instead, force exec sessions logs to file for now; we can add a log-driver flag later (we'll probably want to add a `podman logs` command that reads exec session logs at the same time). As part of this, add support for the new 'none' logs driver in Conmon. It will be the default log driver for exec sessions, and can be optionally selected for containers. Great thanks to Joe Gooch (mrwizard@dok.org) for adding support to Conmon for a null log driver, and wiring it in here. Fixes containers#6555 Signed-off-by: Matthew Heon <matthew.heon@pm.me>

openshift-ci-robot added the kind/bug Categorizes issue or PR as related to a bug. label Jun 10, 2020

mheon self-assigned this Jun 10, 2020

mheon mentioned this issue Jun 10, 2020

Do not share container log driver for exec #6560

Merged

openshift-merge-robot closed this as completed in #6560 Jun 17, 2020

allisonkarlitskaya mentioned this issue Jun 15, 2022

please add --log-driver for podman exec #14594

Open

github-actions bot added the locked - please file new issue/PR Assist humans wanting to comment on an old issue or PR with locked comments. label Sep 23, 2023

github-actions bot locked as resolved and limited conversation to collaborators Sep 23, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

podman exec commands use log-driver from podman run #6555

podman exec commands use log-driver from podman run #6555

goochjj commented Jun 10, 2020

goochjj commented Jun 10, 2020

mheon commented Jun 10, 2020

mheon commented Jun 10, 2020

podman exec commands use log-driver from podman run #6555

podman exec commands use log-driver from podman run #6555

Comments

goochjj commented Jun 10, 2020

goochjj commented Jun 10, 2020

mheon commented Jun 10, 2020

mheon commented Jun 10, 2020