Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docker outputs carriage return when using -t option #37366

Closed
sgleske-ias opened this issue Jun 28, 2018 · 11 comments
Closed

docker outputs carriage return when using -t option #37366

sgleske-ias opened this issue Jun 28, 2018 · 11 comments

Comments

@sgleske-ias
Copy link

Bug description

Docker outputs a carriage return when using docker run -t or docker exec -t. Similar to #8513

However, if I remove -t then there's no carriage return.

Steps to reproduce the issue:

Example of running without and with -t option.

$ docker run alpine /bin/sh -c echo | od -c
0000000   \n
0000001
$ docker run -t alpine /bin/sh -c echo | od -c
0000000   \r  \n
0000002

Describe the results you received:

od returned when running docker run -t alpine /bin/sh -c echo | od -c.

0000000   \r  \n
0000002

Describe the results you expected:

od should have returned when running docker run -t alpine /bin/sh -c echo | od -c.

0000000   \n
0000001

Additional information

Output of docker version:

Client:
 Version:      18.03.1-ce
 API version:  1.37
 Go version:   go1.9.5
 Git commit:   9ee9f40
 Built:        Thu Apr 26 07:13:02 2018
 OS/Arch:      darwin/amd64
 Experimental: false
 Orchestrator: swarm

Server:
 Engine:
  Version:      18.03.1-ce
  API version:  1.37 (minimum version 1.12)
  Go version:   go1.9.5
  Git commit:   9ee9f40
  Built:        Thu Apr 26 07:22:38 2018
  OS/Arch:      linux/amd64
  Experimental: true

Output of docker info:

Containers: 7
 Running: 2
 Paused: 0
 Stopped: 5
Images: 43
Server Version: 18.03.1-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host ipvlan macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 773c489c9c1b21a6d78b5c538cd395416ec50f88
runc version: 4fc53a81fb7c994640722ac585fa9ca548971871
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.9.87-linuxkit-aufs
Operating System: Docker for Mac
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 1.952GiB
Name: linuxkit-025000000001
ID: OPI3:5G2I:K3LS:U3RY:URAM:ZOJU:RUFS:LX5D:EV5C:ROV7:SXNV:7XYS
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
 File Descriptors: 31
 Goroutines: 51
 System Time: 2018-06-28T19:33:53.5527109Z
 EventsListeners: 2
HTTP Proxy: docker.for.mac.http.internal:3128
HTTPS Proxy: docker.for.mac.http.internal:3129
Registry: https://index.docker.io/v1/
Labels:
Experimental: true
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):

Using Docker for Mac. Mac OS X version sw_vers:

ProductName:	Mac OS X
ProductVersion:	10.12.6
BuildVersion:	16G1408
@thaJeztah
Copy link
Member

thaJeztah commented Jun 28, 2018

The reason you get a carriage return is that when specifying -t, a TTY is attached, and will (by default) convert newlines to \r\n; you can find more information in this thread; #8513 (comment)

If you disable this option in the shell, those carriage returns won't appear;

docker run -t --rm alpine /bin/sh -c "stty -onlcr && echo" | od -c
0000000  \n
0000001

So, the TL;DR is that it's the TTY adding the carriage return, not Docker.

I'll close this issue, because this is not a bug, but feel free to continue the conversation

@sgleske-ias
Copy link
Author

Hmm, I didn't realize that and must have glossed over that comment. Thanks for clearing it up.

@thaJeztah
Copy link
Member

I was surprised at the time as well (\r\n is a bit odd for Linux); haven't digged yet for "why", but probably some weird historic reason haha

@samrocketman
Copy link

samrocketman commented Jun 29, 2018

My OSS user handle is @samrocketman (just mentioning since I commit issues to this project from that handle as well).

edit: I'm also @sgleske-ias

samrocketman added a commit to samrocketman/jenkins-bootstrap-shared that referenced this issue Jun 29, 2018
Using the TTY means that the password included a carriage return.  This
was because of the `-t` option and not specific to docker.

See also:

- moby/moby#37366
- moby/moby#8513 (comment)
@Silvanoc
Copy link

Silvanoc commented Sep 2, 2019

I know that this issue is closed, but just for the sake of completeness and to help @thaJeztah better understanding it.

Many terminals (including Linux VT) actually need the CR (\r), otherwise you get the "staircase effect" (next text just a line down but not at the beginning).

This in independent from the convention how EOL is stored in files, what we usually see carriage return and newline related to.

@thaJeztah
Copy link
Member

Thanks @Silvanoc, that makes sense!

@hqmonaro
Copy link

hqmonaro commented Aug 1, 2021

If you capturing output via command substitution and looping through it then an alternative is to let read strip off the carriage returns for you by passing the ANSI-C quoted $'\r' escape sequence for carriage return as the delimiter to read.

Default newline delimiter:

while read -r file; do echo $file | cat -vet; done <<< $(podman exec -t s3-proxy-backend find /etc/httpd/conf.d -type f) 
/etc/httpd/conf.d/ssl.conf^M$
/etc/httpd/conf.d/mod_security.conf^M$

Carriage return delimiter:

while read -rd $'\r' file; do echo $file | cat -vet; done <<< $(podman exec -t s3-proxy-backend find /etc/httpd/conf.d -type f)
/etc/httpd/conf.d/ssl.conf$
/etc/httpd/conf.d/mod_security.conf$

Of course you can simply not allocate a TTY and no terminal formatting will be performed:

for file in $(podman exec s3-proxy-backend find /etc/httpd/conf.d -type f); do echo $file | cat -vet; done 
/etc/httpd/conf.d/ssl.conf$
/etc/httpd/conf.d/mod_security.conf$

@mikaelstaldal
Copy link

Linux shells (e.g. bash) handles this by detecting if the output is a TTY or not (e.g. redirected to a file), and act accordingly. Why cannot docker run do the same, i.e. detect if the output of docker is a TTY and only then enable --tty?

@thaJeztah
Copy link
Member

The container itself has a TTY attached, which means that software inside the container is acting accordingly, and the container's output streams are returned over the API "as-is". There may be (and likely will be) valid use-cases for this scenario, where the exact output of the container must be captured.

Removing these extra newlines when presenting won't be possible, or at least: not without making the "assumption" that all of them were added because of how these binaries inside the container change their behaviour. For example, if the output of the container intentionally prints a CRLF, then there's no way for docker to know which one of these were "intentional", and which of those were added (due to the TTY being attached);

docker run -it --rm alpine sh -c 'echo -e "hello\r\nworld"; echo "hello"; echo "world"' | od -c
0000000    h   e   l   l   o  \r  \r  \n   w   o   r   l   d  \r  \n   h
0000020    e   l   l   o  \r  \n   w   o   r   l   d  \r  \n
0000035

Of course it would still be possible to use (e.g.) dos2unix to convert them all;

docker run -it --rm alpine sh -c 'echo -e "hello\r\nworld\nhello"; echo "world"' | dos2unix | od -c
0000000    h   e   l   l   o  \r  \n   w   o   r   l   d  \n   h   e   l
0000020    l   o  \n   w   o   r   l   d  \n
0000031

@thaJeztah
Copy link
Member

thaJeztah commented Apr 5, 2023

Oh! Probably bad example because I used a shell, and I see the shell here converts my original \r\n to \r\r\n. So that would be the case for "if a shell was used and did conversion", but that's not the same with other binaries.

@mikaelstaldal
Copy link

The container itself has a TTY attached, which means that software inside the container is acting accordingly, and the container's output streams are returned over the API "as-is".

But docker run behaves differently depending on whether you specify --tty or not. What I would like is that it enables TTY when running:

$ docker run first_image

but not when running:

$ docker run second_image > somefile.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

7 participants