Logging:driver flag is not respected

[WCByrne]

Description

It seems like docker compose is not respecting logging: driver settings from the compose file. I looked around a bit but didn't see anything hinting that this shouldn't work anymore.

service:
   logging:
      driver: none

Describe the results you received:

With no other changes
docker-compose up does not include logs from service
docker compose up includes logs from service

Describe the results you expected:

Same as docker-compose. Respect the logging option

Output of docker version:

Client: Docker Engine - Community
 Cloud integration: 1.0.12
 Version:           20.10.5
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        55c4c88
 Built:             Tue Mar  2 20:13:00 2021
 OS/Arch:           darwin/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.5
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       363e9a8
  Built:            Tue Mar  2 20:15:47 2021
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.4.4
  GitCommit:        05f951a3781f4f2c1911b05e61c160e9c30eaa8e
 runc:
  Version:          1.0.0-rc93
  GitCommit:        12644e614e25b05da6fd08a38ffa0cfe1903fdec
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Output of docker context show:
You can also run docker context inspect context-name to give us more details but don't forget to remove sensitive content.

default

Output of docker info:

Client:
 Context:    default
 Debug Mode: false
 Plugins:
  app: Docker App (Docker Inc., v0.9.1-beta3)
  buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)
  scan: Docker Scan (Docker Inc., v0.6.0)

Server:
 Containers: 12
  Running: 0
  Paused: 0
  Stopped: 12
 Images: 324
 Server Version: 20.10.5
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 1
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 05f951a3781f4f2c1911b05e61c160e9c30eaa8e
 runc version: 12644e614e25b05da6fd08a38ffa0cfe1903fdec
 init version: de40ad0
 Security Options:
  seccomp
   Profile: default
 Kernel Version: 5.10.25-linuxkit
 Operating System: Docker Desktop
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 5.806GiB
 Name: docker-desktop
 ID: 4TKN:DWKM:IUGU:4TP2:EP3H:MOTW:5SXT:3ARS:6S6W:BOD2:DQRG:CQFQ
 Docker Root Dir: /var/lib/docker
 Debug Mode: true
  File Descriptors: 44
  Goroutines: 49
  System Time: 2021-04-28T22:25:10.3047409Z
  EventsListeners: 4
 HTTP Proxy: http.docker.internal:3128
 HTTPS Proxy: http.docker.internal:3128
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Additional environment details (AWS ECS, Azure ACI, local, etc.):

[ndeloof]

Hi.

The reason you see this distinct behaviour is because docker compose up attach to the containers being ran, it does not rely on the logging API.

Same applies when you use --log-driver=none with the docker cli, you still get container output:

➜  docker run --log-driver=none alpine ping localhost
PING localhost (127.0.0.1): 56 data bytes
64 bytes from 127.0.0.1: seq=0 ttl=64 time=0.045 ms
...

using log-driver: none will actually prevent the daemon to capture and store container logs, which won't then be available if you later use docker (compose) logs command.

[WCByrne]

I'm not sure I follow that last bit. Are you saying that is a workaround or that docker compose up just doesn't work with this right now? Is this a bug/oustanding todo or do I need to update my compose file or some other option?

I'd love to start using docker compose but one of my containers is bit noisy and we've always relied on just disabling logging for the whole thing.

[ndeloof]

Sorry for the technical details :P

[WCByrne]

Interesting to know what is going on but the end user "what do I do" summary would also be super helpful if there is one 😄

[ndeloof]

while docker-compose ignore logs on up when logging driver is set to none, this is not consistent with docker run and we just filled the gap. Can be considered a backward compatibility bug, but actually also considered a compatibility issue with docker cli.

So, "works as expected" with existing options. If you want to mute a service, you can use compose up -d then compose logs SERVICES

[WCByrne]

So there wouldn't be a way to specify this in the compose file for a container anymore?

[ndeloof]

We probably could introduce a new --attach option to list service to attach to, that would align with the existing --attach-dependencies

(sorry for the multi-comments on this issue, bad connexion here)

[ndeloof]

So there wouldn't be a way to specify this in the compose file for a container anymore?

indeed. Even this "used to work" this is not consistent with docker run and as such doesn't make much sense, would have to be fixed one way or the other.

[WCByrne]

hm... well. I'm not savvy on the implementation details but +1 from me for someway to tell a container to not log anything via a setting in a compose file. We share our compose file across repos and extend it and that logging:driver setting was applied on the base container so it was an easy way to do this across the board and never have to think about it when running a command.

[ndeloof]

I created https://github.com/docker/compose-cli/issues/1615 to track this as a feature request.
To hard-code this in the compose file, we might introduce attach_stdout attribute to service definition.

[WCByrne]

An option like that makes a lot of sense to me. Thanks!

[majoyal]

+1 on the feature request (#1615). Some containers in development are very verbose (pgAdmin is one that we use) and they flood the terminal, making it very difficult to keep track of the useful development logs of our own containers. We were able to found a workaround for pgAdmin by sending its access logs to /dev/null, but that would be very useful to have a similar option like Compose v1 to silence some containers.

[davidjb99]

+1 on the feature request (#1615). Some containers in development are very verbose (pgAdmin is one that we use) and they flood the terminal, making it very difficult to keep track of the useful development logs of our own containers. We were able to found a workaround for pgAdmin by sending its access logs to /dev/null, but that would be very useful to have a similar option like Compose v1 to silence some containers.

@majoyal could you outline how you accomplished this in docker compose? Perhaps by posting the compose file? I need to do exactly this for pgAdmin.

[majoyal]

@davidjb99 This is what we have for pgAdmin:

version: '3.4'
services:
  frontend:
    ...
  backend:
    ...
  postgres:
    ...
  pgadmin:
    image: dpage/pgadmin4:4
    environment:
      PGADMIN_DEFAULT_EMAIL: 'pg@dev.local'
      PGADMIN_DEFAULT_PASSWORD: 'abc123'
      GUNICORN_ACCESS_LOGFILE: '/dev/null'
      PGADMIN_CONFIG_UPGRADE_CHECK_ENABLED: 'False'
    depends_on:
      - postgres
    ports:
      - '11000:80'
    restart: unless-stopped
    volumes:
      - pgadmin-data:/var/lib/pgadmin
      - ./apps/backend/scripts/pgadmin/servers.json:/pgadmin4/servers.json:ro
    logging:
      driver: none

volumes:
  pgadmin-data:

The important part is the GUNICORN_ACCESS_LOGFILE environment variable that let's us specify something else than stdout for the Gunicorn access logs.

Reference: https://www.pgadmin.org/docs/pgadmin4/latest/container_deployment.html

Hope this helps!

[melkishengue]

+1 for this feature request. I would love to be able to do this directly from the compose file.

[octopusOnJellyfish]

+1 for this feature request, getting a ton of logs from an image that doesn't have a flag to disable logs, the only way to disable logs was using
logging:
driver: none

[ndeloof]

@octopusOnJellyfish can't you rely on the recently introduced up --attach SERVICE flag?

[octopusOnJellyfish]

I have to attach to multiple containers, how can I pass more than one to the --attach flag?

[ndeloof]

flag can be repeated, i.e --attach FOO --attach BAR

[octopusOnJellyfish]

flag can be repeated, i.e --attach FOO --attach BAR

Thank you, this is what I was looking for.

[DesignByOnyx]

Conversely, there is now a --no-attach flag that allows you to omit container logs: docker compose up --no-attach mongo --no-attach redis

[jac1013]

Both --attach and no-attach worked for me.

flag can be repeated, i.e --attach FOO --attach BAR

Thanks for this @ndeloof, really appreciate it.