add feature to log command to support only getting stdout or stderr as well as both

[kencochrane]

right now the logs command shows both the standard out and standard error combined, it would be nice if you can pass a flag and see just stdout or stderr. If no flag default to both.

This is helpful if you have a process that has lots of logs, and you really only want to see error messages.

[carlhoerberg]

why not just normal redirection? > /dev/null to only see stderr and 2>/dev/null to only see stdout?

[progrium]

yeah, let's be a good unix app and not reinvent standard infrastructure.

[shykes]

I agree this seems like the way to go.

What about in the future remote API - is there interest in getting
programmatic access to stdout without stderr or vice-versa?

On Mon, Mar 11, 2013 at 7:00 PM, Jeff Lindsay notifications@github.comwrote:

yeah, let's be a good unix app and not reinvent standard infrastructure.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/45#issuecomment-14754641
.

[progrium]

Oh I think absolutely.

On Mon, Mar 11, 2013 at 9:12 PM, Solomon Hykes notifications@github.comwrote:

I agree this seems like the way to go.

What about in the future remote API - is there interest in getting
programmatic access to stdout without stderr or vice-versa?

On Mon, Mar 11, 2013 at 7:00 PM, Jeff Lindsay notifications@github.comwrote:

yeah, let's be a good unix app and not reinvent standard infrastructure.

—
Reply to this email directly or view it on GitHub<
https://github.com/dotcloud/docker/issues/45#issuecomment-14754641>
.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/45#issuecomment-14754936
.

Jeff Lindsay
http://progrium.com

[carlhoerberg]

i would not like to see logs through the remote api, imo it should just be for management, a simple human readable (think redis) api which you could telnet/nc to. using syslog or something should be much more efficient for log distribution, it shouldn't be a part of docker. if you mix logs in there you need a multiplexing/framing protocol..

On Tuesday 12 March 2013 at 10:12, Solomon Hykes wrote:

I agree this seems like the way to go.

What about in the future remote API - is there interest in getting
programmatic access to stdout without stderr or vice-versa?

On Mon, Mar 11, 2013 at 7:00 PM, Jeff Lindsay <notifications@github.com (mailto:notifications@github.com)>wrote:

yeah, let's be a good unix app and not reinvent standard infrastructure.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/45#issuecomment-14754641
.

—
Reply to this email directly or view it on GitHub (#45 (comment)).

[progrium]

I also agree with Carl and would love a simple human readable Redis-style
remote management interface. But I also want access to streams. So this is
how I think it should work. This is sort of how Heroku works behind the
scenes and was sort of how the old Python Docker prototype worked that I
worked on.

The remote interface wouldn't let you interact with a running container's
streams, but it could "mount" a process to a (random) port which you could
then connect to for raw socket access to a container's stdin/stdout. Note
that sockets don't map directly to stdin/stdout/stderr, but that in a
terminal you always ultimately only have two anyway. So you can control
whether you see stdout or stderr by including redirection within the
command argument of "docker run".

I personally prefer to think in terms of more general "streams" as opposed
to "logs", and this provides a very simple and accessible interface that
doesn't bother with the slightly more complicated and archaic but also
"log"-oriented model that you'd get implementing something like syslog.

Does that make sense?

On Mon, Mar 11, 2013 at 9:19 PM, Carl Hörberg notifications@github.comwrote:

i would not like to see logs through the remote api, imo it should just be
for management, a simple human readable (think redis) api which you could
telnet/nc to. using syslog or something should be much more efficient for
log distribution, it shouldn't be a part of docker. if you mix logs in
there you need a multiplexing/framing protocol..

On Tuesday 12 March 2013 at 10:12, Solomon Hykes wrote:

I agree this seems like the way to go.

What about in the future remote API - is there interest in getting
programmatic access to stdout without stderr or vice-versa?

On Mon, Mar 11, 2013 at 7:00 PM, Jeff Lindsay <notifications@github.com(mailto:
notifications@github.com)>wrote:

yeah, let's be a good unix app and not reinvent standard
infrastructure.

—
Reply to this email directly or view it on GitHub<
https://github.com/dotcloud/docker/issues/45#issuecomment-14754641>
.

—
Reply to this email directly or view it on GitHub (
#45 (comment)).

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/45#issuecomment-14755084
.

Jeff Lindsay
http://progrium.com

[shykes]

Yes, it does make sense to think in terms of streams. In addition to logs,
the API will need to expose other streams as well: metrics and activity
events, for example (not to mention inbound stdin streams for interactive
mode). And if possible it should be well integrated with the
request-response part of the API. So syslog is not a great primitive.
However it should be fairly easy to write a syslog adaptor which consumes
docker streams.

I have a specific design in mind for supporting streams in the api, based
strongly on our experience at dotCloud. It's not exactly what you describe,
Jeff, but it's close enough. Until I have time to write something down, I'm
happy to talk about it over beer :)

On Mon, Mar 11, 2013 at 7:39 PM, Jeff Lindsay notifications@github.comwrote:

I also agree with Carl and would love a simple human readable Redis-style
remote management interface. But I also want access to streams. So this is
how I think it should work. This is sort of how Heroku works behind the
scenes and was sort of how the old Python Docker prototype worked that I
worked on.

The remote interface wouldn't let you interact with a running container's
streams, but it could "mount" a process to a (random) port which you could
then connect to for raw socket access to a container's stdin/stdout. Note
that sockets don't map directly to stdin/stdout/stderr, but that in a
terminal you always ultimately only have two anyway. So you can control
whether you see stdout or stderr by including redirection within the
command argument of "docker run".

I personally prefer to think in terms of more general "streams" as opposed
to "logs", and this provides a very simple and accessible interface that
doesn't bother with the slightly more complicated and archaic but also
"log"-oriented model that you'd get implementing something like syslog.

Does that make sense?

On Mon, Mar 11, 2013 at 9:19 PM, Carl Hörberg notifications@github.comwrote:

i would not like to see logs through the remote api, imo it should just
be
for management, a simple human readable (think redis) api which you
could
telnet/nc to. using syslog or something should be much more efficient
for
log distribution, it shouldn't be a part of docker. if you mix logs in
there you need a multiplexing/framing protocol..

On Tuesday 12 March 2013 at 10:12, Solomon Hykes wrote:

I agree this seems like the way to go.

What about in the future remote API - is there interest in getting
programmatic access to stdout without stderr or vice-versa?

On Mon, Mar 11, 2013 at 7:00 PM, Jeff Lindsay <
notifications@github.com(mailto:
notifications@github.com)>wrote:

yeah, let's be a good unix app and not reinvent standard
infrastructure.

—
Reply to this email directly or view it on GitHub<
https://github.com/dotcloud/docker/issues/45#issuecomment-14754641>
.

—
Reply to this email directly or view it on GitHub (
#45 (comment)).

—
Reply to this email directly or view it on GitHub<
https://github.com/dotcloud/docker/issues/45#issuecomment-14755084>
.

Jeff Lindsay
http://progrium.com

—

Reply to this email directly or view it on GitHubhttps://github.com//issues/45#issuecomment-14755599
.

[carlhoerberg]

my idea was not to implement syslog, but rather just pipe the output to "logger" or something.. then rsyslog on the host machine can either log it or forward it..

[shykes]

Got it. Yes, we will make this possible.

On Mon, Mar 11, 2013 at 9:08 PM, Carl Hörberg notifications@github.comwrote:

my idea was not to implement syslog, but rather just pipe the output to
"logger" or something.. then rsyslog on the host machine can either log it
or forward it..

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/45#issuecomment-14757593
.

[srid]

+1 - treat logs as streams, not files.

@shykes

I have a specific design in mind for supporting streams in the api, based
strongly on our experience at dotCloud. [...] Until I have time to write
something down, [...]

i'm looking forward to this, as eventually i plan to integrate docker log streaming with logyard. at minimum, i'd expect something like the following for retrieving logs:

$ HOST=$(docker inspect eb345s | json NetworkSettings.IpAddress)
$ PORT=$(docker inspect eb345s | json Logs.Port)
$ nc -v $HOST $PORT
<< ... stream of process output ... >>

a simple tcp client is all that is required to read logs, and further process them (by sending it to external log aggregators, for instance).

[shykes]

This is addressed in #725 (don't mix stdout and stderr in docker run in attached mode)