Docker CLI inserts many unwanted control characters when exec a command inside a container - Windows Server 2019

[javiertuya]

Description

When executing a command inside a container (e.g. a ping), the command includes many unwanted control characters at the output:

• if executed from powershell it makes the console scroll to the end of previous output before sending the output
• if executed from powershell ISE we can see a lot of control chars, which make the output unreadable

Steps to reproduce the issue:

1. From a powershell window execute three commands, eg: clear, docker version and docker exec -it container-name cmd /c ping localhost
2. From powershell ISE execute the same commands

Describe the results you received:

From powershell window (note that output form docker version becomes hidden at the console):

From powershell ISE(need two screenshots to include all ouput):

Describe the results you expected:
This is the result from windows server 2016 (running a different version of docker)

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

PS C:\Users\Administrador> docker version
Client:
 Version:           18.09.0
 API version:       1.39
 Go version:        go1.10.3
 Git commit:        33a45cd0a2
 Built:             unknown-buildtime
 OS/Arch:           windows/amd64
 Experimental:      false

Server:
 Engine:
  Version:          18.09.0
  API version:      1.39 (minimum version 1.24)
  Go version:       go1.10.3
  Git commit:       33a45cd0a2
  Built:            11/07/2018 00:24:12
  OS/Arch:          windows/amd64
  Experimental:     false

Output of docker info:

PS C:\Users\Administrador> docker info
Containers: 2
 Running: 1
 Paused: 0
 Stopped: 1
Images: 81
Server Version: 18.09.0
Storage Driver: windowsfilter
 Windows: 
Logging Driver: json-file
Plugins:
 Volume: local
 Network: ics l2bridge l2tunnel nat null overlay transparent
 Log: awslogs etwlogs fluentd gelf json-file local logentries splunk syslog
Swarm: inactive
Default Isolation: process
Kernel Version: 10.0 17763 (17763.1.amd64fre.rs5_release.180914-1434)
Operating System: Windows Server 2019 Standard Version 1809 (OS Build 17763.195)
OSType: windows
Architecture: x86_64
CPUs: 24
Total Memory: 1.95GiB
Name: CONT1
ID: BO5J:USNI:JWCV:ZMCA:VPY7:TIAE:4KBQ:EVBN:LIBU:FM6C:44GU:SJA4
Docker Root Dir: C:\ProgramData\docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):
Hyper-V virtual machine

[thaJeztah]

If I'm not mistaken, when running an interactive container, the raw stdout/stderr of the container is used, so those control-characters are coming from the container itself; not sure if there's something we can do in that case 🤔

@johnstep @olljanat @cpuguy83

[javiertuya]

I've found an old related issue: #26911.
Should the CLI pass some TERM capabilities to the container in order to get an adequate output?

[thaJeztah]

You can try if that works by passing -e TERM to the docker run command; setting an environment-variable without specifying a value (and no =) will take the value from the current environment;

docker run -it --rm busybox sh -c 'echo $TERM' 
xterm

docker run -it --rm -e TERM  busybox sh -c 'echo $TERM'
xterm-color

[javiertuya]

The container is windows, so it is not expected to understand the TERM environment variable. Nevertheless, I've tried it and get the same result

[thaJeztah]

There's some older issues around terminal emulation on Windows (see #37002 and issues linked from that). I think those concentrated around this part in the codebase

moby/pkg/term/term_windows.go

Lines 27 to 88 in ea65a0b

	// StdStreams returns the standard streams (stdin, stdout, stderr).
	func StdStreams() (stdIn io.ReadCloser, stdOut, stdErr io.Writer) {
	// Turn on VT handling on all std handles, if possible. This might
	// fail, in which case we will fall back to terminal emulation.
	var emulateStdin, emulateStdout, emulateStderr bool
	fd := os.Stdin.Fd()
	if mode, err := winterm.GetConsoleMode(fd); err == nil {
	// Validate that winterm.ENABLE_VIRTUAL_TERMINAL_INPUT is supported, but do not set it.
	if err = winterm.SetConsoleMode(fd, mode|winterm.ENABLE_VIRTUAL_TERMINAL_INPUT); err != nil {
	emulateStdin = true
	} else {
	vtInputSupported = true
	}
	// Unconditionally set the console mode back even on failure because SetConsoleMode
	// remembers invalid bits on input handles.
	winterm.SetConsoleMode(fd, mode)
	}
	fd = os.Stdout.Fd()
	if mode, err := winterm.GetConsoleMode(fd); err == nil {
	// Validate winterm.DISABLE_NEWLINE_AUTO_RETURN is supported, but do not set it.
	if err = winterm.SetConsoleMode(fd, mode|winterm.ENABLE_VIRTUAL_TERMINAL_PROCESSING|winterm.DISABLE_NEWLINE_AUTO_RETURN); err != nil {
	emulateStdout = true
	} else {
	winterm.SetConsoleMode(fd, mode|winterm.ENABLE_VIRTUAL_TERMINAL_PROCESSING)
	}
	}
	fd = os.Stderr.Fd()
	if mode, err := winterm.GetConsoleMode(fd); err == nil {
	// Validate winterm.DISABLE_NEWLINE_AUTO_RETURN is supported, but do not set it.
	if err = winterm.SetConsoleMode(fd, mode|winterm.ENABLE_VIRTUAL_TERMINAL_PROCESSING|winterm.DISABLE_NEWLINE_AUTO_RETURN); err != nil {
	emulateStderr = true
	} else {
	winterm.SetConsoleMode(fd, mode|winterm.ENABLE_VIRTUAL_TERMINAL_PROCESSING)
	}
	}
	// Temporarily use STD_INPUT_HANDLE, STD_OUTPUT_HANDLE and
	// STD_ERROR_HANDLE from syscall rather than x/sys/windows as long as
	// go-ansiterm hasn't switch to x/sys/windows.
	// TODO: switch back to x/sys/windows once go-ansiterm has switched
	if emulateStdin {
	stdIn = windowsconsole.NewAnsiReader(syscall.STD_INPUT_HANDLE)
	} else {
	stdIn = os.Stdin
	}
	if emulateStdout {
	stdOut = windowsconsole.NewAnsiWriter(syscall.STD_OUTPUT_HANDLE)
	} else {
	stdOut = os.Stdout
	}
	if emulateStderr {
	stdErr = windowsconsole.NewAnsiWriter(syscall.STD_ERROR_HANDLE)
	} else {
	stdErr = os.Stderr
	}
	return
	}

I'm not running Docker on Windows myself, so it's tricky for me to reproduce 😅 , but possibly one of the other people I pinged above may be able to check if there's anything that can be done here

[olljanat]

@javiertuya I highly recommend everyone to use only core version of Windows Server on Docker nodes because of lot of other weaknesses (look example what I have said over year go: #27588 (comment) )

All our servers are so I'm not able to reproduce this one on them but I can see that it happens on Windows 10, version 18.03 + Docker CE 18.09 too.

How ever IMO this is more likely bug on PowerShell ISE than on Docker so I would recommend to contact Microsoft about it or use some other editor.

[javiertuya]

@olljanat Thank you. I assume that latest docker version changed this behaviour, that can't be handled by a dumb terminal, like powershell ISE (I run docker 18.03.1-ce on linux). I have checked that Visual Studio Code does not have this problem.

[olljanat]

@javiertuya yes that is most probably the case.

Can we close this one?

[henvic]

Hi, anyone knows anything else about this (e.g., anyone contact Microsoft regarding PowerShell ISE)? We tested it, and it failed on some Windows installations but not others.

We are piping Docker exec output though WebSocket and our clients reported this exact issue (just to let you know we run Docker on Linux nodes).

[DaweiCai]

I met this issue too. following is the repro step:

1. Run one windows container like : docker run -it testwindowsimage cmd
2. run command "docker exec -it f7654312232 cmd /c ping localhost > test.txt"
3. in test.txt we will see:
   �[H�]0;C:\Windows\system32\cmd.exe ��[?25h�[?25l�[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C
   �[80X�[80C�[H�[?25h�[?25l
   Pinging f7465cb4cd21 [::1] with 32 bytes of data:
   Reply from ::�[37X�[37C�[3;14H�[?25h�[?25l1: time<1ms
   �[?25h�[?25lRe ��[?25h�[?25lply from ::1: time<1ms
   �[?25h�[?25lR ��[?25h�[?25leply from ::1: time<1ms
   �[?25h�[?25lR ��[?25h�[?25l
   Reply from ::1: time<1ms�[33X�[33C
   �[57X�[57C

@javiertuya and @thaJeztah do you know how to solve this issue?

But if we only output it to terminal instead of redirect the output looks fine.

[thaJeztah]

Looks like the same issue as #30137. I left a comment there, linking to some possible workarounds; #30137 (comment)

These answers may help if you're using PowerShell; https://stackoverflow.com/a/51681675 https://stackoverflow.com/a/51683794
I see there's some discussion around ANSI escape sequences in the PowerShell issue tracker here; PowerShell/PowerShell#13071

As to

in test.txt we will see:

That is expected; if the container's output has control characters / escape sequences, those are handled by the terminal in one case, and in the other case are piped directly to a text-file, so there's nothing in between to handle those control characters.