x/crypto/ssh/terminal: ReadPassword does not support umlauts on Windows

[levrik]

What version of Go are you using (go version)?

go version go1.9.2 windows/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

set GOARCH=amd64
set GOBIN=
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\development\go
set GORACE=
set GOROOT=C:\Go
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0
set CXX=g++
set CGO_ENABLED=1
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config

What did you do?

Execute this on Gist on Windows (https://gist.github.com/levrik/99210259065679da20c3c66ef383eea9) and enter some umlauts in the password prompt (e.g. ö, ü or ä).

What did you expect to see?

(Screenshot taken on go version go1.9.2 darwin/amd64)

What did you see instead?

This was discovered when a colleague tried to use a new version of an internal command line tool and it didn't accept the correct password anymore (password was supplied via command line flag before).

[bradfitz]

/cc @alexbrainman

[levrik]

Just looked a bit more into this issue. Basically the problem is that Go expects UTF-8 (or some different encoding) but the Windows console uses Code Page xxx. Which code page exactly seems to depend on the configured language (for example US English uses 437, German uses 850).

Using the following workaround for now. Seems to work fine on German and English Windows systems. But I'm not sure if it's a good idea to always use the US English Code Page 437. It should respect the current code page of the system.

// ReadPassword reads a line of input from a terminal without local echo.  This
// is commonly used for inputting passwords and other sensitive data. The slice
// returned does not include the \n.
func ReadPassword(fd int) ([]byte, error) {
	var st uint32
	if err := windows.GetConsoleMode(windows.Handle(fd), &st); err != nil {
		return nil, err
	}
	old := st

	st &^= (windows.ENABLE_ECHO_INPUT)
	st |= (windows.ENABLE_PROCESSED_INPUT | windows.ENABLE_LINE_INPUT | windows.ENABLE_PROCESSED_OUTPUT)
	if err := windows.SetConsoleMode(windows.Handle(fd), st); err != nil {
		return nil, err
	}

	defer func() {
		windows.SetConsoleMode(windows.Handle(fd), old)
	}()

+	decodedFd := transform.NewReader(passwordReader(fd), charmap.CodePage437.NewDecoder())
-	return readPasswordLine(passwordReader(fd))
+	return readPasswordLine(decodedFd)
}

[mattn]

As far as I can see, x/crypto/ssh/terminal#ReadPassword take a string argument not int.

[mattn]

Oh, sorry. My bad. it is implemented in util_windows.go

[mattn]

This should fix the issue.

diff --git a/ssh/terminal/util_windows.go b/ssh/terminal/util_windows.go
index 60979cc..e7dc952 100644
--- a/ssh/terminal/util_windows.go
+++ b/ssh/terminal/util_windows.go
@@ -17,6 +17,8 @@
 package terminal
 
 import (
+	"os"
+
 	"golang.org/x/sys/windows"
 )
 
@@ -98,5 +100,5 @@ func ReadPassword(fd int) ([]byte, error) {
 		windows.SetConsoleMode(windows.Handle(fd), old)
 	}()
 
-	return readPasswordLine(passwordReader(fd))
+	return readPasswordLine(os.NewFile(uintptr(fd), "sysfile"))
 }

[gopherbot]

Change https://golang.org/cl/79335 mentions this issue: ssh/terminal: read password from os.File

[egonelbre]

EDIT: nevermind didn't see the previous updates.

When you read from stdin it uses default codepage. Printing the byte-sequence verifies this.

GetConsoleCP (https://docs.microsoft.com/en-us/windows/console/getconsolecp) can be used to get the current code page. SetConsoleCP can be used to switch to code page 65001 (utf8) -- but, initial testing only resulted getting 0 for characters. One possibility is to lookup the active code-page and translate after reading the buffer. However, I suspect that Code Page can change during typing.

readPasswordLine seems to need adjusting for multi-byte characters as well.

On a laptop's Nordic keyboard when set on Windows XP to a Danish layout typing in cmd.exe set to Code Page 850 the umlaut "¨" followed by "a" or "o", will render respectively "ä" and "ö", the characters "æ", "ø" , and "å" will also render utilising those settings, but none of the above render using levrik 's code sample.

The code sample doesn't seem to run properly utilising mintty.exe.

[levrik]

EDIT: Nevermind, @forskning was referring to the initial bug report

@forskning I think the German umlauts are included in the English code page so it works fine. As I said, it was just a quick workaround for us (we only have German and English Windows systems).

[bronze1man]

I think you guys need a chinese/japanese version win7 to test the real console code page problem.
I think you should use windows api MultiByteToWideChar and GetConsoleCP to solve your problem.
Implement another MultiByteToWideChar in golang will make the binary very large if you include gbk support.

By the way code page 65001 (utf8) on chinese win7 is an useless stuff. It looks like code page 437. But it will crash the process(include ipconfig.exe) if you want to output some chinese char.

[alexbrainman]

@levrik did you try https://golang.org/cl/79335 ? Does it fixes your problem?
I think it should work for you.

Thank you.

Alex

@alexbrainman I confirm after applying CL 79335 and running cmd.exe set to Code Page 850

that with the keyboard layout set to DE that ö, ü, ä, and ß, all render

and with the keyboard layout set to DA that ä, ö, æ, ø, and å, all render