Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

internal/poll: Reading data piped through os.Stdin hangs on Windows version #22024

Closed
jakans opened this issue Sep 25, 2017 · 61 comments
Closed

internal/poll: Reading data piped through os.Stdin hangs on Windows version #22024

jakans opened this issue Sep 25, 2017 · 61 comments

Comments

@jakans
Copy link

@jakans jakans commented Sep 25, 2017

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go1.9 darwin/amd64, but cross-compiling for PC/CYGWIN

Does this issue reproduce with the latest release?

Yes, issue is only on go1.9 and Windows, go1.8 is fine on all platforms

What operating system and processor architecture are you using (go env)?

Windows under CYGWIN. (Go compiler is not installed on target machines.)

What did you do?

curl -s "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=pubmed&id=26422376&version=2.0" |
go run pipehangs.go

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

Source code of pipehangs.go is inserted here:

package main

import (
"fmt"
"io/ioutil"
"os"
)

func main() {

str, err := ioutil.ReadAll(os.Stdin)
fmt.Fprintf(os.Stderr, "n: %d\n", len(str))
if err != nil {
	fmt.Fprintf(os.Stderr, "err: %s\n", err)
}

}

File attached, with .txt suffix added to get past data format test:

pipehangs.go.txt

What did you expect to see?

n: 3910

What did you see instead?

Program hangs.

It's fine if you pipe a file:

cat testfile.txt | go run pipehangs.go

or redirect stdin:

go run pipehangs.go < testfile.txt

Only reading a network result under Windows and go1.9 causes it to hang.

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Sep 25, 2017

Probably has something to do with the poller changes, but I don't know what.

CC @alexbrainman

@alexbrainman

This comment has been minimized.

Copy link
Member

@alexbrainman alexbrainman commented Sep 26, 2017

@jakans your program works with Windows type command:

c:\Users\Alex\dev\src\issue\go\22024>type pipehangs.go
package main

import (
        "fmt"
        "io/ioutil"
        "os"
)

func main() {

        str, err := ioutil.ReadAll(os.Stdin)
        fmt.Fprintf(os.Stderr, "n: %d\n", len(str))
        if err != nil {
                fmt.Fprintf(os.Stderr, "err: %s\n", err)
        }
}

c:\Users\Alex\dev\src\issue\go\22024>type pipehangs.go | go run pipehangs.go
n: 215

c:\Users\Alex\dev\src\issue\go\22024>

I do not have curl program installed on my computer to test your scenario. How do I install it?
But even after I install curl program, I am not sure how to debug this - it could be curl program that hangs for all we know.

What are you trying to do? If you just want to download some file from the Internet, Go standard library have plenty of tools to do that.

Alex

@forskning

This comment has been minimized.

Copy link

@forskning forskning commented Sep 26, 2017

MSYS (with separately downloaded cURL)

$ curl.exe --version
curl 7.50.2 (i386-pc-win32) libcurl/7.50.2 OpenSSL/1.0.2e zlib/1.2.8
Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtsp
smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile NTLM SSL libz

$ go.exe version
go version go1.9 windows/386

$ curl.exe -s \
> "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=pubmed&id=26422376&version=2.0" | \
> go.exe run pipehangs.go
n: 0

$ 

MSYS2

http://nurmi-labs.blogspot.com/2016/11/git.html

$ curl.exe --version
curl 7.50.1 (i686-w64-mingw32) libcurl/7.50.1 OpenSSL/1.0.2h zlib/1.2.8 libidn/1
.33 libssh2/1.7.0 nghttp2/1.13.0 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s
 rtmp rtsp scp sftp smtp smtps telnet tftp
Features: IDN IPv6 Largefile SSPI Kerberos SPNEGO NTLM SSL libz TLS-SRP HTTP2 Me
talink

$ go.exe version
go version go1.9 windows/386

$ curl.exe -s \
> "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=pubmed&id=26422376&version=2.0" | \
> go.exe run pipehangs.go
n: 3910

$
@ktye

This comment has been minimized.

Copy link

@ktye ktye commented Sep 26, 2017

Does it hang also, if you run it from cmd.exe?
It could have something to do with the terminal (cygwin, msys/mintty vs console).
For me on mintty, pipe sometimes work, sometimes hang. On cmd.exe they always work.

@forskning

This comment has been minimized.

Copy link

@forskning forskning commented Sep 26, 2017

the content of #22024 (comment) indicates NO outout from curl 7.50.2

in both examples from that comment the Windows Console had been utilised

for what its worth here is MSYS2's mintty.exe running a C Shell

% curl.exe -s \
? "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=pubmed&id=26422376&version=2.0" | \
? go.exe run pipehangs.go
n: 3910
%

You don't mention what cURL version you are using, or if you've run your cURL command without the pipe.

@alexbrainman

This comment has been minimized.

Copy link
Member

@alexbrainman alexbrainman commented Sep 26, 2017

@forskning thanks for testing. Looks like "go run pipehangs.go" does not hang in either of your scenarios.

I wonder why curl 7.50.2 does not output any data. What happens if you redirect curl output to a file? Do you get anything written to the file?

Thank you

Alex

@forskning

This comment has been minimized.

Copy link

@forskning forskning commented Sep 26, 2017

@alexbrainman I should have looked into that prior to adding the MSYS content

http://nurmi-labs.blogspot.com/2015/11/bcc55.html

It was Dirk Paehl's cURL 7.50.2 (Download WITH SUPPORT SSL) I used for a Borland/Dmake compile of perl-5.10.1.tar.gz, the CPAN setup, and, as those were http addresses, I never added the SSL support files.

Perl/perl5@378eeda#diff-65539b463d4890c68be9c1e3de589c4d

You can read about the events which led up to the "The Borland Chainsaw Massacre" here:

http://www.xray.mpe.mpg.de/mailing-lists/perl5-porters/2011-09/msg00034.html

@as

This comment has been minimized.

Copy link
Contributor

@as as commented Sep 26, 2017

Unable to reproduce with various versions of curl 7.55.1.
n: 0 when curl fails to initialize; but a hang never occurs.

http://www.paehl.com/open_source/?CURL_7.55.1

set URL=https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=pubmed^&id=26422376^&version=2.0

curl -s "%URL%" | go run pipehangs.go  & echo curl_755_1
n: 0
curl_755_1

No crash, instant return

curl -s "%URL%" | go run pipehangs.go  & echo curl_755_1_ssh2_ssl_sspi
n: 0
curl_755_1_ssh2_ssl_sspi

image

curl -s "%URL%" | go run pipehangs.go & echo curl_755_1
n: 3910
curl_755_1
curl -s "%URL%" | go run pipehangs.go  & echo curl_755_1_rtmp_ssh2_ssl_sspi
n: 0
curl_755_1_rtmp_ssh2_ssl_sspi

image

go get github.com/as/torgo/hget
go run %GOPATH%\src\github.com\as\torgo\hget\hget.go "%URL%" | go run pipehangs.go
n: 3910
@forskning

This comment has been minimized.

Copy link

@forskning forskning commented Sep 27, 2017

@jakans

Does your curl command when run without the pipe output the xml to the console?

@alexbrainman

This comment has been minimized.

Copy link
Member

@alexbrainman alexbrainman commented Sep 27, 2017

@alexbrainman I should have looked into that prior to adding the MSYS content
...

Simply speaking you curl.exe was broken. Cool. Thank you for explaining.

Unable to reproduce with various versions of curl 7.55.1.
n: 0 when curl fails to initialize; but a hang never occurs.

Thank you @as for trying.

Alex

@forskning

This comment has been minimized.

Copy link

@forskning forskning commented Sep 27, 2017

moot point

curl_750_2_ssl (without adding the SSL support files) returns output for http not https

$ curl.exe -s "http://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=pubmed&id=26422376&version=2.0"
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>301 Moved Permanently</title>
</head><body>
<h1>Moved Permanently</h1>
<p>The document has moved <a href="https://eutils.ncbi.nlm.nih.gov/entrez/eutils
/esummary.fcgi?db=pubmed&amp;id=26422376&amp;version=2.0">here</a>.</p>
</body></html>

$ curl.exe -s "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/esummary.fcgi?db=pubmed&id=26422376&version=2.0"

$ curl.exe --version
curl 7.50.2 (i386-pc-win32) libcurl/7.50.2 OpenSSL/1.0.2e zlib/1.2.8
Protocols: dict file ftp ftps gopher http https imap imaps ldap pop3 pop3s rtsp
smb smbs smtp smtps telnet tftp
Features: AsynchDNS IPv6 Largefile NTLM SSL libz

$

"That they be not forced to sue the lawe, wrapped with so infinite crickes and moot poyntes."

Laurence Humphrey - 1563

@jakans

This comment has been minimized.

Copy link
Author

@jakans jakans commented Sep 27, 2017

The problem was discovered by users of Entrez Direct (https://www.ncbi.nlm.nih.gov/books/NBK179288/) who had installed the latest release on their PCs. While most of EDirect is implemented as a Perl wrapper to NCBI's URL-based Entrez Utilities (https://www.ncbi.nlm.nih.gov/books/NBK25501/), the xtract component is written in Go, and it failed to read pipes from the upstream Perl steps. I quickly reverted to the Go 1.8 xtract binary for CYGWIN (Mac and Linux versions are fine under Go 1.9), and then isolated the problem and created the minimal pipehangs.go program for debugging.

Entrez Direct provides a simple way for non-programmers to do sophisticated data mining in PubMed and other interconnected NCBI databases. The target audience includes scientists, medical librarians, bibliometric researchers, and academic administrators, as well as computational biologists who are looking for an easy way to get specific information from our databases. Users supply query details in command-line arguments. Separate steps in the query are connected by Unix pipes. Everything else is handled behind the scenes, with no additional coding required.

The sample ad hoc EDirect query shown below searches for journal articles in PubMed and retrieves the records in a defined XML form. The XML data set is piped to xtract, which limits results to journals published in the U.S. and then visits each author, printing one author name per line. This output is piped to a standard script that produces a frequency table of publications per author:

esearch -db pubmed -query "rattlesnake phospholipase" |
efetch -format xml |
xtract -pattern PubmedArticle
-if MedlineJournalInfo/Country -equals "United States"
-block Author -tab "\n" -sep " " -element LastName,Initials |
sort-uniq-count-rank

9 Wells MA
8 Marangoni S
8 Toyama MH
6 Dennis EA
5 Bon C
5 Kézdy FJ
5 Sigler PB
5 Soares AM
4 Carlini CR
4 Faure G
4 Francischetti IM
4 Guimarães JA
4 HANAHAN DJ
4 Heinrikson RL
...

To answer the earlier question, curl.exe --version produces:
curl 7.43.0 (x86_64-unknown-cygwin) libcurl/7.43.0 OpenSSL/1.0.2d zlib/1.2.8 libidn/1.29 libssh2/1.5.0

@jakans

This comment has been minimized.

Copy link
Author

@jakans jakans commented Sep 27, 2017

The latest xtract.go source code now has an undocumented -echo command to help with debugging. xtract -sample will send a small sample XML file to stdout. xtract -echo will read from stdin, using the same method that is hanging when reading from a pipe. To obtain the source code without doing the full EDirect installation you can run:

ftp ftp://ftp.ncbi.nlm.nih.gov/entrez/entrezdirect/edirect.tar.gz
gunzip -c edirect.tar.gz | tar xf -
rm edirect.tar.gz

To build the PC binary using a Go 1.9 compiler on any platform, run:

cd edirect
env GOOS=windows GOARCH=386 go build -o testxtract -v xtract.go
chmod +x testxtract

On a PC under CYGWIN, the following will print a sample XML record:

./testxtract -sample

Redirecting stdout will save it to a file:

./testxtract -sample > sample.xml

Piping it to another instance of testxtract and reading with the -echo command will fail:

./testxtract -sample | ./testxtract -echo

However, redirecting stdin from an existing file will work:

./testxtract -echo < sample.xml

The actual code being tested is excerpted here:

in := os.Stdin

// test reading from input pipe or file (undocumented)
if args[0] == "-echo" {
	const XMLBUFSIZE = 65536 + 16384
	buffr := make([]byte, XMLBUFSIZE)
	for {
		n, err := in.Read(buffr)
		if n == 0 {
			break
		}
		if err != nil {
			fmt.Fprintf(os.Stderr, "err: %s\n", err)
			break
		}
		fmt.Fprintf(os.Stdout, "%s", buffr[:n])
	}
	return
}
@as

This comment has been minimized.

Copy link
Contributor

@as as commented Sep 27, 2017

It would be nice to see the dataflow through the pipe. Can you run your pipeline through pv? If you're using Windows you can use the implementation here:

go get github.com/as/torgo/pv
hget http://badurl.com | %GOBIN%\pv > nul
pv: t=1   b=0.000 B   Δ=0.000 B/s  a=0.000 B/s
pv: t=2   b=0.000 B   Δ=0.000 B/s  a=0.000 B/s
pv: t=3   b=0.000 B   Δ=0.000 B/s  a=0.000 B/s
pv: t=4   b=0.000 B   Δ=0.000 B/s  a=0.000 B/s
pv: t=5   b=0.000 B   Δ=0.000 B/s  a=0.000 B/s
pv: t=6   b=0.000 B   Δ=0.000 B/s  a=0.000 B/s
pv: t=7   b=0.000 B   Δ=0.000 B/s  a=0.000 B/s
pv: t=8   b=0.000 B   Δ=0.000 B/s  a=0.000 B/s
pv: t=9   b=0.000 B   Δ=0.000 B/s  a=0.000 B/s
pv: t=10   b=0.000 B   Δ=0.000 B/s  a=0.000 B/s
pv: t=11   b=0.000 B   Δ=0.000 B/s  a=0.000 B/s
2017/09/27 14:00:15 Get http://badurl.com: dial tcp: lookup badurl.com: no such host
pv: t=11   b=0.000 B   Δ=0.000 B/s  a=0.000 B/s

./testxtract -sample | pv | ./testxtract -echo

@alexbrainman

This comment has been minimized.

Copy link
Member

@alexbrainman alexbrainman commented Sep 28, 2017

To obtain the source code without doing the full EDirect installation you can run:

ftp ftp://ftp.ncbi.nlm.nih.gov/entrez/entrezdirect/edirect.tar.gz
gunzip -c edirect.tar.gz | tar xf -
rm edirect.tar.gz

I did that.

To build the PC binary using a Go 1.9 compiler on any platform, run:

cd edirect
env GOOS=windows GOARCH=386 go build -o testxtract -v xtract.go
chmod +x testxtract

I did that.

On a PC under CYGWIN,

I don't have CYGWIN, I used cmd.exe program.

the following will print a sample XML record:

./testxtract -sample

I did that. It worked.

Redirecting stdout will save it to a file:

./testxtract -sample > sample.xml

I did that. It worked.

Piping it to another instance of testxtract and reading with the -echo command will fail:

./testxtract -sample | ./testxtract -echo

I did that. It worked.

However, redirecting stdin from an existing file will work:

./testxtract -echo < sample.xml

I did that. It worked.

The actual code being tested is excerpted here:

I think your code has a bug. When in.Read(buffr) returns with n == 0 you break out of the read loop. But there could be more data coming. The Reader documentation says https://golang.org/pkg/io/#Reader about that: "... Callers should treat a return of 0 and nil as indicating that nothing happened; in particular it does not indicate EOF. ...".

Perhaps your CYGWIN shell program (unlike my cmd.exe) inserts 0 length writes into the pipe between 2 programs.

Alex

@forskning

This comment has been minimized.

Copy link

@forskning forskning commented Sep 28, 2017

In the event anyone decides to pursue testing the Cygnus port.

A search of the current curl and perl implementations follows.

https://cygwin.com/cgi-bin2/package-grep.cgi

curl-7.55.1-1
perl-5.24.1-1

Pour faire l'Archiviste il faut être un homme intelligent.

Un homme intelligent ne fait pas l'Archiviste.

@jakans

This comment has been minimized.

Copy link
Author

@jakans jakans commented Sep 28, 2017

I've modified the -echo code to report if "n == 0" without breaking out of the loop, but the test had been added merely to suppress the EOF error message. The actual function that calls Read only looks for "err != nil" to determine end of data.

To further investigate, I added a -read command to specifically test the actual read block function. These are in the xtract.go that is now on the ftp site. The relevant code section is:

// DEBUGGING

// test reading from input pipe or file (undocumented)
if args[0] == "-echo" {
	const XMLBUFSIZE = 65536 + 16384
	buffr := make([]byte, XMLBUFSIZE)
	for {
		n, err := in.Read(buffr)
		if err != nil {
			fmt.Fprintf(os.Stderr, "ERR: %s, N: %d\n", err, n)
			break
		}
		if n == 0 {
			fmt.Fprintf(os.Stderr, "N: zero\n")
			continue
		}
		fmt.Fprintf(os.Stdout, "%s", buffr[:n])
	}
	return
}

// CREATE XML BLOCK READER FROM STDIN OR FILE

rdr := NewXMLReader(in, doCompress, doCleanup, doStrict || doMixed)
if rdr == nil {
	fmt.Fprintf(os.Stderr, "\nERROR: Unable to create XML Block Reader\n")
	os.Exit(1)
}

// DEBUGGING

// test reading blocks from xml reader (undocumented)
if args[0] == "-read" {
	for {
		str := rdr.NextBlock()
		if str == "" {
			fmt.Fprintf(os.Stderr, "\n\nSTR: empty\n")
			break
		}
		fmt.Fprintf(os.Stdout, "%s", str)
	}
	fmt.Fprintf(os.Stdout, "\n")
	return
}

I asked my colleague with a PC to try:

testxtract -sample | testxtract -echo

and:

testxtract -sample | testxtract -read

and he got wildly inconsistent results. Sometimes it would hang, sometimes it wouldn't. Inserting pv in between would either hang, print one "pv:" line and then the expected output, or print "pv:" lines forever.

My expert-of-last-resort colleague got involved, and quickly found that he could force consistent hangs by replacing:

testxtract -sample

in the above commands with:

(sleep 1; testxtract -sample)

That was immediately confirmed by the other colleague.

Do you know of any change between Go 1.8 and Go 1.9 that might explain this sort of timing dependency with pipes on the PC?

@as

This comment has been minimized.

Copy link
Contributor

@as as commented Sep 28, 2017

		n, err := in.Read(buffr)
		if err != nil {

Err can be io.EOF, and n > 0, but we break out of the loop so that data is never processed

			fmt.Fprintf(os.Stderr, "ERR: %s, N: %d\n", err, n)
			break
		}
		if n == 0 {
			fmt.Fprintf(os.Stderr, "N: zero\n")
			continue
		}
		fmt.Fprintf(os.Stdout, "%s", buffr[:n])
@as

This comment has been minimized.

Copy link
Contributor

@as as commented Sep 28, 2017

Just because err != nil doesn't mean you're done processing the data, it may be the case that n > 0. The test err == io.EOF tells you that the reader is done, so err != nil it says nothing about the data it copied to the slice for you.

for {
   n, err := fd.Read(p)
   if err != nil && err != io.EOF{
       // hard stop; a real error happens
       break
   }
   datain <- append([]byte{}, p[:n]...) // edited here 
   if err == io.EOF{
     break
  }
}

Above is a cautious way to do this, i.e., don't process the bytes if a real error occurred. The go doc io.Reader recommends p[:n] is handled regardless of what err is though.

@jakans

This comment has been minimized.

Copy link
Author

@jakans jakans commented Sep 28, 2017

I had noticed this in the documentation, but in my early tests err != nil always came with n == 0. To be robust against future implementation changes, I've modified the code, but I still prefer to err on the side of caution and ignore bytes that come with a real error. It will now print any non-EOF error message to Stderr, at least making the user aware that something is amiss. The -echo test code was also changed, with the relevant section shown below:

		n, err := in.Read(buffr)
		if err != nil {
			if err != io.EOF {
				fmt.Fprintf(os.Stderr, "ERR: %s, N: %d\n", err, n)
				break
			}
			if n == 0 {
				// EOF and no more data
				break
			}
		}
		if n == 0 {
			fmt.Fprintf(os.Stderr, "N: zero\n")
			continue
		}
		fmt.Fprintf(os.Stdout, "%s", buffr[:n])

The source code on the ftp site has been updated, but I'll allow some time for in-house testing before making a new release of the precompiled binaries.

Thanks for the advice.

@as

This comment has been minimized.

Copy link
Contributor

@as as commented Sep 29, 2017

Not a problem, but just FYI my example and yours are semantically similar - they don't handle data if there's a real error. This is because p[:n], when n==0 is a valid zero-length slice. The disadvantage is that it doesn't guard against non-conforming implementations of io.Reader that return -1.

Below I used n >= 0 and not n > 0 because sometimes a zero-length read carries information through the virtue of the read call itself, which I believe @alexbrainman mentioned somewhere earlier in the thread. It's up to you whether you need this information or not.

// Edit: as pointed out below, this is still incorrect as the data should 
// be processed regardless of error's value
for {
   n, err := fd.Read(p)
   if err != nil && err != io.EOF{
       // hard stop; a real error happens
       break
    }
   if n >= 0{
         // handle data in p[:n]
   }
   if err == io.EOF{
     break
  }
}
@alexbrainman

This comment has been minimized.

Copy link
Member

@alexbrainman alexbrainman commented Sep 29, 2017

@jakans and @as, I think, you are still wrong with your code. As per io.Reader documentation: "... When Read encounters an error or end-of-file condition after successfully reading n > 0 bytes, it returns the number of bytes read. ...". So if Read returns n > 0, you must process p[:n] bytes regardless what the err is. And you don't, because you break out of for loop.

@jakans are you still having a problem? If yes, I take it you can reproduce it with a Go program (without curl). Is that correct? If so, can you, show us that program, how you run it, what the program outputs, and why you think the output is wrong. If we need CYGWIN to reproduce, tell us where to get it and how to install it. Thank you.

Alex

@as

This comment has been minimized.

Copy link
Contributor

@as as commented Sep 29, 2017

I agree it is incorrect, but a mild amount of APIs outside of stdlib return mangled data on a non-EOF error. I would love for this to work everywhere, it a lot easier on the eyes too.

for {
	n, err = r.Read(b)
	if n > 0 {
		// process b[:n]
	}
	if err != nil {
		break
	}
}
@forskning

This comment has been minimized.

Copy link

@forskning forskning commented Sep 29, 2017

presuming from #22024 (comment)

the hardware utilised is 64bit

https://cygwin.com/install.html

Installing and Updating Cygwin for 64-bit versions of Windows

@jakans

This comment has been minimized.

Copy link
Author

@jakans jakans commented Sep 29, 2017

That is the CYGWIN address I was about to pass along.

The current xtract.go source code has the modified -echo command that can handle n == 0 blocks. You can download it by running:

ftp ftp://ftp.ncbi.nlm.nih.gov/entrez/entrezdirect/edirect.tar.gz
gunzip -c edirect.tar.gz | tar xf -
rm edirect.tar.gz

and build the PC executable with:

env GOOS=windows GOARCH=386 go build -o testxtract -v xtract.go
chmod +x testxtract

You should be able to reproduce the hanging problem when built under Go 1.9 by running:

(sleep 1; testxtract -sample) | testxtract -echo

and:

(sleep 1; testxtract -sample) | pv | testxtract -echo

For reference, here is the current -echo source code:

if args[0] == "-echo" {
	const XMLBUFSIZE = 65536 + 16384
	buffr := make([]byte, XMLBUFSIZE)
	for {
		n, err := in.Read(buffr)
		if err != nil {
			if err != io.EOF {
				fmt.Fprintf(os.Stderr, "ERR: %s, N: %d\n", err, n)
				break
			}
			if n == 0 {
				// EOF and no more data
				break
			}
		}
		if n == 0 {
			fmt.Fprintf(os.Stderr, "N: zero\n")
			continue
		}
		fmt.Fprintf(os.Stdout, "%s", buffr[:n])
	}
	return
}
@forskning

This comment has been minimized.

Copy link

@forskning forskning commented Sep 29, 2017

@jakans

If CYGWIN is needed to reproduce, beyond the minimal base packages, what added packages would be relevant?

Not pertinent to this thread, but IIRC historically bi-directional named pipes have been problematic on CYGWIN.

@jakans

This comment has been minimized.

Copy link
Author

@jakans jakans commented Sep 29, 2017

I'm told that the base system should be sufficient.

@jakans

This comment has been minimized.

Copy link
Author

@jakans jakans commented Oct 11, 2017

Thanks for testing it, and of course for all the work to get to this point. Since this fixes the problem, are you and the other developers satisfied with the change, and will be in 1.9.2 and future versions?

@as

This comment has been minimized.

Copy link
Contributor

@as as commented Oct 11, 2017

No problem. This issue did not affect me personally (yet), I just wanted to help see what the problem was and fix it. I am happy that @alexbrainman got the poller to behave, and time will tell if the change is satisfying.

@forskning

This comment has been minimized.

Copy link

@forskning forskning commented Oct 11, 2017

https://groups.google.com/forum/#!topic/golang-dev/naQlgeWVtvA

@jakans I believe Go 1.9.2 is the subject of the above discussion.

@dsnet dsnet changed the title Reading data piped through os.Stdin hangs on Windows version internal/poll: Reading data piped through os.Stdin hangs on Windows version Oct 11, 2017
@alexbrainman

This comment has been minimized.

Copy link
Member

@alexbrainman alexbrainman commented Oct 12, 2017

are you and the other developers satisfied with the change,

I satisfy myself that https://go-review.googlesource.com/#/c/go/+/69871 change is not doing bad things. I wouldn't know if it fixes your problem, because I could not reproduce it. But @as checked that. So I will take his word.

and will be in 1.9.2 and future versions?

I suggested that change for go1.9.2 on
https://groups.google.com/forum/#!topic/golang-dev/naQlgeWVtvA

Alex

@ianlancetaylor ianlancetaylor modified the milestones: Go1.10, Go1.9.2 Oct 12, 2017
@gopherbot gopherbot closed this in 862fb86 Oct 12, 2017
@rsc rsc reopened this Oct 13, 2017
@rsc

This comment has been minimized.

Copy link
Contributor

@rsc rsc commented Oct 13, 2017

CL 69871 OK for Go 1.9.2.

@rsc

This comment has been minimized.

Copy link
Contributor

@rsc rsc commented Oct 15, 2017

It looks like if we are going to fix this for Go 1.9.2, the patch will apply cleanly only if we first also fix #22149, which it looks like we should fix anyway.

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Oct 15, 2017

Change https://golang.org/cl/70990 mentions this issue: [release-branch.go1.9] internal/poll: only call SetFileCompletionNotificationModes for sockets

@alexbrainman

This comment has been minimized.

Copy link
Member

@alexbrainman alexbrainman commented Oct 15, 2017

It looks like if we are going to fix this for Go 1.9.2, the patch will apply cleanly only if we first also fix #22149, which it looks like we should fix anyway.

SGTM to also fix #22149

Alex

@as

This comment has been minimized.

Copy link
Contributor

@as as commented Oct 15, 2017

Since I still have the environment to reproduce this issue, I can re-test any intermediate versions if necessary.

@junegunn junegunn mentioned this issue Oct 15, 2017
6 of 6 tasks complete
@rsc

This comment has been minimized.

Copy link
Contributor

@rsc rsc commented Oct 16, 2017

CL 53530 OK for Go 1.9.2.
CL 65810 OK for Go 1.9.2.
CL 69871 OK for Go 1.9.2.

gopherbot pushed a commit that referenced this issue Oct 25, 2017
…ficationModes for sockets

CL 36799 made SetFileCompletionNotificationModes to be called for
file handles. I don't think it is correct. Revert that change.

Fixes #22024
Fixes #22207

Change-Id: I26260e8a727131cffbf60958d79eca2457495554
Reviewed-on: https://go-review.googlesource.com/69871
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-on: https://go-review.googlesource.com/70990
Run-TryBot: Russ Cox <rsc@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
@rsc

This comment has been minimized.

Copy link
Contributor

@rsc rsc commented Oct 26, 2017

go1.9.2 has been packaged and includes:

The release is posted at golang.org/dl.

— golang.org/x/build/cmd/releasebot, Oct 26 21:09:18 UTC

@VladimirAlexiev

This comment has been minimized.

Copy link

@VladimirAlexiev VladimirAlexiev commented Apr 4, 2018

tomnomnom/gron#32 occurs on Go 1.9 but not on 1.10.
The symptom is that the program doesn't read stdin redirect.

@golang golang locked and limited conversation to collaborators Apr 4, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
9 participants
You can’t perform that action at this time.