New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: Ability to clear log history #1083

Closed
djessup opened this Issue Mar 9, 2015 · 138 comments

Comments

Projects
None yet
@djessup

djessup commented Mar 9, 2015

A feature I've thought would be useful since originally using Fig, and now Compose would be the ability to clear the log history for Composed-managed containers. Long-running or "chatty" containers can end up with a lot of log noise that may not be wanted.

I'd expect a command like the following would solve the problem:
$ docker-compose logs --clear [service]

@dnephin

This comment has been minimized.

Show comment
Hide comment
@dnephin

dnephin Mar 13, 2015

Contributor

I don't think that is a feature supported by the docker daemon. Looking at the api docs, the only option would be to truncate the returned logs to a limited number of lines per container:

https://docs.docker.com/reference/api/docker_remote_api_v1.17/#get-container-logs

Contributor

dnephin commented Mar 13, 2015

I don't think that is a feature supported by the docker daemon. Looking at the api docs, the only option would be to truncate the returned logs to a limited number of lines per container:

https://docs.docker.com/reference/api/docker_remote_api_v1.17/#get-container-logs

@metal3d

This comment has been minimized.

Show comment
Hide comment
@metal3d

metal3d Apr 9, 2015

Contributor

👍 for this issue. I actually use compose to develop a website in Golang, Mongodb and nginx... 5 days after begining I have a long logs that becomes to be worrying. Each time I restart containers append many lines to the log.
@dnephin I don't understand if you give a solution (that I don't understand :)) or if you propose to check if it's possible with api. Sorry for my bad english.

Contributor

metal3d commented Apr 9, 2015

👍 for this issue. I actually use compose to develop a website in Golang, Mongodb and nginx... 5 days after begining I have a long logs that becomes to be worrying. Each time I restart containers append many lines to the log.
@dnephin I don't understand if you give a solution (that I don't understand :)) or if you propose to check if it's possible with api. Sorry for my bad english.

@thaJeztah

This comment has been minimized.

Show comment
Hide comment
@thaJeztah

thaJeztah Apr 9, 2015

Member

Docker 1.6 will add support for logging drivers, see moby/moby#10568 (currently; JSON, syslog and "none") work is in progress for basic log-rotation; moby/moby#11485

Member

thaJeztah commented Apr 9, 2015

Docker 1.6 will add support for logging drivers, see moby/moby#10568 (currently; JSON, syslog and "none") work is in progress for basic log-rotation; moby/moby#11485

@metal3d

This comment has been minimized.

Show comment
Hide comment
@metal3d

metal3d Apr 13, 2015

Contributor

Good to hear that, thanks a lot :)

Contributor

metal3d commented Apr 13, 2015

Good to hear that, thanks a lot :)

@djessup

This comment has been minimized.

Show comment
Hide comment
@djessup

djessup Apr 17, 2015

Thanks for the background and update on Docker 1.6 - looking forward to it!

djessup commented Apr 17, 2015

Thanks for the background and update on Docker 1.6 - looking forward to it!

chernjie added a commit to chernjie/elk that referenced this issue Apr 20, 2015

chernjie added a commit to chernjie/elk that referenced this issue Apr 20, 2015

@sgarbesi

This comment has been minimized.

Show comment
Hide comment
@sgarbesi

sgarbesi Jul 16, 2015

docker logs -c (clear) <container> would be great.

+1

sgarbesi commented Jul 16, 2015

docker logs -c (clear) <container> would be great.

+1

@baptistedonaux

This comment has been minimized.

Show comment
Hide comment
@baptistedonaux

baptistedonaux Sep 11, 2015

+1 Really important

baptistedonaux commented Sep 11, 2015

+1 Really important

@egyptianbman

This comment has been minimized.

Show comment
Hide comment
@egyptianbman

egyptianbman Sep 14, 2015

Holy cow, I just sat through several minutes of logs to get to the end. I'd really appreciate this as well for the sake of not having to rebuild containers constantly.

egyptianbman commented Sep 14, 2015

Holy cow, I just sat through several minutes of logs to get to the end. I'd really appreciate this as well for the sake of not having to rebuild containers constantly.

@jghibiki

This comment has been minimized.

Show comment
Hide comment
@jghibiki

jghibiki commented Sep 15, 2015

+1

@vronin

This comment has been minimized.

Show comment
Hide comment
@vronin

vronin commented Sep 15, 2015

+1

@junajan

This comment has been minimized.

Show comment
Hide comment
@junajan

junajan commented Sep 17, 2015

+1

@knutole

This comment has been minimized.

Show comment
Hide comment
@knutole

knutole commented Sep 18, 2015

👍

@Rodeoclash

This comment has been minimized.

Show comment
Hide comment
@Rodeoclash

Rodeoclash commented Sep 20, 2015

+1

@nafg

This comment has been minimized.

Show comment
Hide comment
@nafg

nafg commented Sep 21, 2015

+2

@FnTm

This comment has been minimized.

Show comment
Hide comment
@FnTm

FnTm commented Sep 21, 2015

+1

@dmage

This comment has been minimized.

Show comment
Hide comment
@dmage

dmage Sep 21, 2015

Just for the record, with Docker 1.8 and docker-compose 1.4 there is already exists a method to limit log size using https://docs.docker.com/compose/yml/#log-driver and log-opt max-size:

  log_driver: "json-file"
  log_opt:
    max-size: "100k"
    max-file: "20"

dmage commented Sep 21, 2015

Just for the record, with Docker 1.8 and docker-compose 1.4 there is already exists a method to limit log size using https://docs.docker.com/compose/yml/#log-driver and log-opt max-size:

  log_driver: "json-file"
  log_opt:
    max-size: "100k"
    max-file: "20"
@knutole

This comment has been minimized.

Show comment
Hide comment
@knutole

knutole Sep 21, 2015

@dmage Thanks, exactly what I need.

knutole commented Sep 21, 2015

@dmage Thanks, exactly what I need.

@OlivierCuyp

This comment has been minimized.

Show comment
Hide comment
@OlivierCuyp

OlivierCuyp commented Oct 6, 2015

+1000

@sheyman

This comment has been minimized.

Show comment
Hide comment
@sheyman

sheyman Oct 11, 2015

+1 would LOVE this

sheyman commented Oct 11, 2015

+1 would LOVE this

@joegoggins

This comment has been minimized.

Show comment
Hide comment
@joegoggins

joegoggins commented Oct 12, 2015

+1

@Rodeoclash

This comment has been minimized.

Show comment
Hide comment
@Rodeoclash

Rodeoclash Oct 12, 2015

@dmage solution works great for us. Those others who are +1ing this, what is wrong with the solution that he's given?

Rodeoclash commented Oct 12, 2015

@dmage solution works great for us. Those others who are +1ing this, what is wrong with the solution that he's given?

@rosskevin

This comment has been minimized.

Show comment
Hide comment
@rosskevin

rosskevin Oct 12, 2015

@Rodeoclash - I think one use case for this is reuse of a given set of containers for a new run i.e. CI test run. The old logs are irrelevant to the new run, so a clear prior to the next command would clear up confusion.

rosskevin commented Oct 12, 2015

@Rodeoclash - I think one use case for this is reuse of a given set of containers for a new run i.e. CI test run. The old logs are irrelevant to the new run, so a clear prior to the next command would clear up confusion.

@SpeCT

This comment has been minimized.

Show comment
Hide comment
@SpeCT

SpeCT commented Oct 14, 2015

+1

@ballinette

This comment has been minimized.

Show comment
Hide comment
@ballinette

ballinette commented Oct 19, 2015

+1, same need as @rosskevin

@dnephin

This comment has been minimized.

Show comment
Hide comment
@dnephin

dnephin Oct 19, 2015

Contributor

Ok, to recap:

  • you can do this from a compose file (see #1083 (comment))
  • #265 covers being able to limit the output from the logs command
  • #1756 covers the re-used container case

Going to close this since it's already support or tracked in other issues.

Contributor

dnephin commented Oct 19, 2015

Ok, to recap:

  • you can do this from a compose file (see #1083 (comment))
  • #265 covers being able to limit the output from the logs command
  • #1756 covers the re-used container case

Going to close this since it's already support or tracked in other issues.

@chris-martin

This comment has been minimized.

Show comment
Hide comment
@chris-martin

chris-martin Jan 13, 2016

I don't quite understand why this was closed. How do you clear the log history?

chris-martin commented Jan 13, 2016

I don't quite understand why this was closed. How do you clear the log history?

@egyptianbman

This comment has been minimized.

Show comment
Hide comment
@egyptianbman

egyptianbman Jan 13, 2016

I add:

  log_opt:
    max-size: 50k

To limit the length of the logs.

egyptianbman commented Jan 13, 2016

I add:

  log_opt:
    max-size: 50k

To limit the length of the logs.

@sgarbesi

This comment has been minimized.

Show comment
Hide comment
@sgarbesi

sgarbesi Jan 13, 2016

They really need to add this, it's essential. Limiting logs is good and all but there should be a simple command to clear logs.

docker logs -c <container>

What gives?

sgarbesi commented Jan 13, 2016

They really need to add this, it's essential. Limiting logs is good and all but there should be a simple command to clear logs.

docker logs -c <container>

What gives?

@chris-martin

This comment has been minimized.

Show comment
Hide comment
@chris-martin

chris-martin Jan 14, 2016

I understand how to limit the log size, but how do you clear the log?

chris-martin commented Jan 14, 2016

I understand how to limit the log size, but how do you clear the log?

@dnephin

This comment has been minimized.

Show comment
Hide comment
@dnephin

dnephin Jan 14, 2016

Contributor

I don't think clearing a log is supported by the docker engine, which is what managed the logs.

Maybe with a custom log driver you could do that, but that would be external to compose.

Contributor

dnephin commented Jan 14, 2016

I don't think clearing a log is supported by the docker engine, which is what managed the logs.

Maybe with a custom log driver you could do that, but that would be external to compose.

@geekscrapy

This comment has been minimized.

Show comment
Hide comment
@geekscrapy

geekscrapy Jan 16, 2016

+1 for being able to flush the logs...

geekscrapy commented Jan 16, 2016

+1 for being able to flush the logs...

@kongchen

This comment has been minimized.

Show comment
Hide comment
@kongchen

kongchen commented Jan 20, 2016

+1

@ObjReponse

This comment has been minimized.

Show comment
Hide comment
@ObjReponse

ObjReponse commented Jan 22, 2016

+1

@eduardoleal

This comment has been minimized.

Show comment
Hide comment

eduardoleal commented Feb 13, 2017

+1

@Joshfindit

This comment has been minimized.

Show comment
Hide comment
@Joshfindit

Joshfindit Feb 13, 2017

Tangentially related suggestion:

When doing docker-compose logs -f, automatically default to --tail=30 (or any other number that's reasonable)

Joshfindit commented Feb 13, 2017

Tangentially related suggestion:

When doing docker-compose logs -f, automatically default to --tail=30 (or any other number that's reasonable)

@piotr-s-brainhub

This comment has been minimized.

Show comment
Hide comment
@piotr-s-brainhub

piotr-s-brainhub Feb 13, 2017

docker-compose logs -f is not enough because with large number of logs displaying it takes a very long time

piotr-s-brainhub commented Feb 13, 2017

docker-compose logs -f is not enough because with large number of logs displaying it takes a very long time

@phlegx

This comment has been minimized.

Show comment
Hide comment
@phlegx

phlegx commented Mar 8, 2017

+1

@luchesar

This comment has been minimized.

Show comment
Hide comment
@luchesar

luchesar commented Mar 10, 2017

+1

@edwardmp

This comment has been minimized.

Show comment
Hide comment
@edwardmp

edwardmp commented Mar 11, 2017

+1

@rabahs

This comment has been minimized.

Show comment
Hide comment
@rabahs

rabahs commented Mar 17, 2017

+1

@htdttb

This comment has been minimized.

Show comment
Hide comment
@htdttb

htdttb commented Mar 21, 2017

+1

@atedja

This comment has been minimized.

Show comment
Hide comment
@atedja

atedja commented Mar 22, 2017

+1

@ndoulgeridis

This comment has been minimized.

Show comment
Hide comment

ndoulgeridis commented Mar 22, 2017

+1

@ahmetkucuk

This comment has been minimized.

Show comment
Hide comment
@ahmetkucuk

ahmetkucuk commented Mar 24, 2017

+1

@sychevsky

This comment has been minimized.

Show comment
Hide comment
@sychevsky

sychevsky commented Mar 24, 2017

+1

@OzzyCzech

This comment has been minimized.

Show comment
Hide comment
@OzzyCzech

OzzyCzech commented Mar 24, 2017

+1

@JeanLescut

This comment has been minimized.

Show comment
Hide comment
@JeanLescut

JeanLescut Mar 29, 2017

Or there is this workaround also

logpath=`docker inspect --format='{{.LogPath}}' reveelium_metricsextraction_1` && mv $logpath $logpath".bckup"

JeanLescut commented Mar 29, 2017

Or there is this workaround also

logpath=`docker inspect --format='{{.LogPath}}' reveelium_metricsextraction_1` && mv $logpath $logpath".bckup"
@ReFruity

This comment has been minimized.

Show comment
Hide comment
@ReFruity

ReFruity Mar 29, 2017

Please, stop commenting +1. It makes this thread really hard to read and extract valuable information. There is a thumbs up button on OP post for this.

ReFruity commented Mar 29, 2017

Please, stop commenting +1. It makes this thread really hard to read and extract valuable information. There is a thumbs up button on OP post for this.

@avindra

This comment has been minimized.

Show comment
Hide comment
@avindra

avindra Apr 3, 2017

Nothing here works for me with Docker for mac. I've hacked together something that works based on reading this thread and the docker forums however.

The problem with D4M, is that on mac, you need to actually run the commands on the xhyve vm. So here's what I came up with. Add these two functions to your .bash_profile.

Important: dont forget to start a new shell or reload your profile before continuing.


Now, docker-logs-clean looks like this:

#!/bin/bash -e

if [[ -z $1 ]]; then
    echo "No container specified"
    exit 1
fi

logFile=$(docker inspect -f '{{.LogPath}}' $1 2> /dev/null)

echo -n "Cleaning ${logFile}... "
d4mexec << EOF
> $logFile
EOF
echo "done"

Note that I am not rming the log file, but just doing >, which will fully truncate the file.

avindra commented Apr 3, 2017

Nothing here works for me with Docker for mac. I've hacked together something that works based on reading this thread and the docker forums however.

The problem with D4M, is that on mac, you need to actually run the commands on the xhyve vm. So here's what I came up with. Add these two functions to your .bash_profile.

Important: dont forget to start a new shell or reload your profile before continuing.


Now, docker-logs-clean looks like this:

#!/bin/bash -e

if [[ -z $1 ]]; then
    echo "No container specified"
    exit 1
fi

logFile=$(docker inspect -f '{{.LogPath}}' $1 2> /dev/null)

echo -n "Cleaning ${logFile}... "
d4mexec << EOF
> $logFile
EOF
echo "done"

Note that I am not rming the log file, but just doing >, which will fully truncate the file.

@tgjamin

This comment has been minimized.

Show comment
Hide comment
@tgjamin

tgjamin May 5, 2017

FWIW in docker-compose version 2 the feature to limit log file sizes:

version: '2'
services:
  my-service:
    image: nginx:alpine
    restart: always
    logging:
      # limit logs retained on host to 25MB
      driver: "json-file"
      options:
        max-size: "500k"
        max-file: "50"

It is not documented on the docker website very well, and this may be useful to others.

tgjamin commented May 5, 2017

FWIW in docker-compose version 2 the feature to limit log file sizes:

version: '2'
services:
  my-service:
    image: nginx:alpine
    restart: always
    logging:
      # limit logs retained on host to 25MB
      driver: "json-file"
      options:
        max-size: "500k"
        max-file: "50"

It is not documented on the docker website very well, and this may be useful to others.

cesine added a commit to cesine/summit-workshop that referenced this issue Jul 26, 2017

adding docker clean up for more debuggability by docker novices
removing "-d" so that docker novices can see the errors while debugging
docker/compose#1083 (comment)

cesine added a commit to cesine/summit-workshop that referenced this issue Jul 26, 2017

adding docker clean up for more debuggability by docker novices
removing "-d" so that docker novices can see the errors while debugging
docker/compose#1083 (comment)

cesine added a commit to cesine/summit-workshop that referenced this issue Jul 26, 2017

adding docker clean up for more debuggability by docker novices
removing "-d" so that docker novices can see the errors while debugging
docker/compose#1083 (comment)

cesine added a commit to cesine/summit-workshop that referenced this issue Jul 26, 2017

adding docker clean up for more debuggability by docker novices
removing "-d" so that docker novices can see the errors while debugging
docker/compose#1083 (comment)

cesine added a commit to cesine/summit-workshop that referenced this issue Jul 27, 2017

adding docker clean up for more debuggability by docker novices
removing "-d" so that docker novices can see the errors while debugging
docker/compose#1083 (comment)

cesine added a commit to cesine/summit-workshop that referenced this issue Jul 27, 2017

adding docker clean up for more debuggability by docker novices
removing "-d" so that docker novices can see the errors while debugging
docker/compose#1083 (comment)

cesine added a commit to cesine/summit-workshop that referenced this issue Jul 27, 2017

adding docker clean up for more debuggability by docker novices
removing "-d" so that docker novices can see the errors while debugging
docker/compose#1083 (comment)
@heetap

This comment has been minimized.

Show comment
Hide comment
@heetap

heetap commented Aug 14, 2017

+1

@sintetico82

This comment has been minimized.

Show comment
Hide comment

sintetico82 commented Dec 15, 2017

+1

@d13g0s0uz4

This comment has been minimized.

Show comment
Hide comment
@d13g0s0uz4

d13g0s0uz4 commented Jan 11, 2018

+1

@ccamacho

This comment has been minimized.

Show comment
Hide comment
@ccamacho

ccamacho commented Jan 19, 2018

+1

@MPJHorner

This comment has been minimized.

Show comment
Hide comment
@MPJHorner

MPJHorner commented Jan 22, 2018

+1

@WhiteBearSpirit

This comment has been minimized.

Show comment
Hide comment
@WhiteBearSpirit

WhiteBearSpirit Feb 9, 2018

I made a simple script docker_clear_log.sh:
sudo truncate -s 0 $(docker inspect --format='{{.LogPath}}' $1)
Usage: ./docker_clear_log.sh [Name-or-ID]
Should work if you have sudo rights and log_driver: "json-file" setting for your docker (default).

WhiteBearSpirit commented Feb 9, 2018

I made a simple script docker_clear_log.sh:
sudo truncate -s 0 $(docker inspect --format='{{.LogPath}}' $1)
Usage: ./docker_clear_log.sh [Name-or-ID]
Should work if you have sudo rights and log_driver: "json-file" setting for your docker (default).

@gsugambit

This comment has been minimized.

Show comment
Hide comment
@gsugambit

gsugambit Mar 5, 2018

+1

I also tried to temp solution and it appears to work

gsugambit commented Mar 5, 2018

+1

I also tried to temp solution and it appears to work

@educatedwarrior

This comment has been minimized.

Show comment
Hide comment
@educatedwarrior

educatedwarrior Jun 23, 2018

The original issue was never resolved and the issue was closed anyway. Interesting. @djessup , how do you like those apples?

educatedwarrior commented Jun 23, 2018

The original issue was never resolved and the issue was closed anyway. Interesting. @djessup , how do you like those apples?

@dhakehurst

This comment has been minimized.

Show comment
Hide comment
@dhakehurst

dhakehurst Jul 11, 2018

would still be nice to have a docker-compose logs --clean

dhakehurst commented Jul 11, 2018

would still be nice to have a docker-compose logs --clean

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment