New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fmt: Scanf works differently on Windows and Linux #23562

Open
ans-ashkan opened this Issue Jan 26, 2018 · 13 comments

Comments

Projects
None yet
10 participants
@ans-ashkan

ans-ashkan commented Jan 26, 2018

What version of Go are you using (go version)?

go version go1.9.3 windows/amd64 and go version go1.9.3 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

on Windows:

set GOARCH=amd64
set GOBIN=
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=D:\golang
set GORACE=
set GOROOT=C:\Go
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\ANS_AS~1\AppData\Local\Temp\go-build251374523=/tmp/go-build -gno-record-gcc-switches
set CXX=g++
set CGO_ENABLED=1
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config

on Linux:

GOARCH="amd64"
GOBIN=""
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/ans_ashkan/go"
GORACE=""
GOROOT="/usr/local/go"
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build871792404=/tmp/go-build -gno-record-gcc-switches"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"

What did you do?

package main

import "fmt"

func main() {
	var firstString string
	var secondString string

	fmt.Printf("Please enter first string:\n")
	fmt.Scanf("%s", &firstString)
	fmt.Printf("Please enter second string:\n")
	n, err := fmt.Scanf("%s", &secondString)
	fmt.Println(err)
}

What did you expect to see?

Same behavior on Windows and Linux.
Either it should capture secondString on Windows and Linux,
or reject it on both operating systems.

What did you see instead?

on Windows: unexpected newline.
on Linux: no errors.

@ans-ashkan ans-ashkan changed the title from Different code behavior between Windows and Linux to Different code behavior on Windows and Linux Jan 26, 2018

@alexbrainman alexbrainman changed the title from Different code behavior on Windows and Linux to fmt: Scanf works differently on Windows and Linux Jan 27, 2018

@alexbrainman

This comment has been minimized.

Member

alexbrainman commented Jan 27, 2018

@ans-ashkan thank you for raising this issue.

I can reproduce what you see. I added some debugging:

package main

import (
	"fmt"
	"os"
)

type myReader struct{}

func (r myReader) Read(p []byte) (n int, err error) {
	n, err = os.Stdin.Read(p)
	fmt.Fprintf(os.Stderr, "DEBUG: %q %v\n", p[:n], err)
	return n, err
}

func main() {
	input := myReader{}
	var firstString string
	var secondString string
	fmt.Printf("Please enter first string:\n")
	fmt.Fscanf(input, "%s", &firstString)
	fmt.Printf("Please enter second string:\n")
	_, err := fmt.Fscanf(input, "%s", &secondString)
	fmt.Println(err)
}

if I run this program, I see:

C:\>u:\test
Please enter first string:
first
DEBUG: "f" <nil>
DEBUG: "i" <nil>
DEBUG: "r" <nil>
DEBUG: "s" <nil>
DEBUG: "t" <nil>
DEBUG: "\r" <nil>
Please enter second string:
DEBUG: "\n" <nil>
unexpected newline

C:\>

on Windows. Windows has \r\n as line delimiters, instead of \n on Linux. I am not sure what to do here. Leaving for others to decide.

Alex

@rasky

This comment has been minimized.

Member

rasky commented Jan 27, 2018

The documentation of the package says: In all the scanning functions, a carriage return followed immediately by a newline is treated as a plain newline (\r\n means the same as \n).

I would think this is a bug but maybe a fmt expert should confirm. /cc @robpike @martisch

@ans-ashkan

This comment has been minimized.

ans-ashkan commented Jan 27, 2018

@rasky I'm confused, if the problem is \r\n then what is happening in this playground. (this is using Fscanf though that is what Scanf does under the hood)

@robpike

This comment has been minimized.

Contributor

robpike commented Jan 27, 2018

The scanning code in fmt is cursed. cc @rsc

@rsc rsc added the NeedsFix label Jan 29, 2018

@rsc

This comment has been minimized.

Contributor

rsc commented Jan 29, 2018

We all agree that the \n after \r should be eaten by the first Scanf.

@ans-ashkan

This comment has been minimized.

ans-ashkan commented Mar 20, 2018

I want to fix this but I'm confused. take a look at this:

package main

import (
	"bytes"
	"fmt"
	"io"
)

var r io.Reader = bytes.NewReader([]byte("first\nsecond\n"))

func main() {
	var firstString string
	var secondString string

	fmt.Printf("Please enter first string:\n")
	fmt.Fscanf(r, "%s", &firstString)
	fmt.Printf("Please enter second string:\n")
	n, err := fmt.Fscanf(r, "%s", &secondString)
	fmt.Printf("firstString: %s, secondString: %s\n", firstString, secondString)
	fmt.Println("n =", n, "err =", err)
}

playground

@rsc you said that \n after \r should be eaten. but what is this behavior in above code?

@mattn

This comment has been minimized.

Member

mattn commented Mar 20, 2018

fmt.Fscanf doesn't read newline so you should do:

https://play.golang.org/p/S15xHVSzUnn

@ans-ashkan

This comment has been minimized.

ans-ashkan commented Mar 20, 2018

@mattn , but fmt.Scanf does read new lines and the code for fmt.Scanf is:

func Scanf(format string, a ...interface{}) (n int, err error) {
	return Fscanf(os.Stdin, format, a...)
}

so fmt.Scanf calls fmt.Fscanf and it reads newlines.

@mattn

This comment has been minimized.

Member

mattn commented Mar 20, 2018

Hmm, fare enough.

diff --git a/src/fmt/scan.go b/src/fmt/scan.go
index ae79e39dee..f3f5bb49f0 100644
--- a/src/fmt/scan.go
+++ b/src/fmt/scan.go
@@ -139,7 +139,7 @@ func Fscanln(r io.Reader, a ...interface{}) (n int, err error) {
 // returns the number of items successfully parsed.
 // Newlines in the input must match newlines in the format.
 func Fscanf(r io.Reader, format string, a ...interface{}) (n int, err error) {
-	s, old := newScanState(r, false, false)
+	s, old := newScanState(r, true, false)
 	n, err = s.doScanf(format, a)
 	s.free(old)
 	return

Anyone know why this part is "true"?

@ans-ashkan

This comment has been minimized.

ans-ashkan commented Mar 20, 2018

Anyone know why this part is "true"?

Which branch/commit?, I don't see "true" there. this is what it looks like in my version:

func Fscanf(r io.Reader, format string, a ...interface{}) (n int, err error) {
	s, old := newScanState(r, false, false)
	n, err = s.doScanf(format, a)
	s.free(old)
	return
}
@mattn

This comment has been minimized.

Member

mattn commented Mar 20, 2018

Sorry, it's my typo. I want to know why this is false?

@andybons andybons added this to the Go1.11 milestone Mar 26, 2018

@gopherbot

This comment has been minimized.

gopherbot commented May 1, 2018

Change https://golang.org/cl/110595 mentions this issue: fmt: Scanning newlines should work the same way on Windows and Linux

@peterGo

This comment has been minimized.

Contributor

peterGo commented May 1, 2018

The test case used to open this issue, after CL https://golang.org/cl/110595 is applied,

i23562.go:

package main

import "fmt"

func main() {
	var firstString string
	var secondString string

	fmt.Printf("Please enter first string:\n")
	n, err := fmt.Scanf("%s", &firstString)
	fmt.Println(n, err, firstString)
	fmt.Printf("Please enter second string:\n")
	n, err = fmt.Scanf("%s", &secondString)
	fmt.Println(n, err, secondString)
}

/*

Linux:

$ go run i23562.go
Please enter first string:
first
1 <nil> first
Please enter second string:
second
1 <nil> second
$ 

Windows:

>go run i23562.go
Please enter first string:
first
1 <nil> first
Please enter second string:
second
1 <nil> second
>

*/

@ianlancetaylor ianlancetaylor modified the milestones: Go1.11, Go1.12 Jun 29, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment