Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net/http: FileServer on WSL fails with wget and curl for files over ~4MB #27128

Open
upsampled opened this issue Aug 21, 2018 · 3 comments
Open

net/http: FileServer on WSL fails with wget and curl for files over ~4MB #27128

upsampled opened this issue Aug 21, 2018 · 3 comments
Labels
Milestone

Comments

@upsampled
Copy link

@upsampled upsampled commented Aug 21, 2018

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

go version go1.10.3 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build493773858=/tmp/go-build -gno-record-gcc-switches"

using WSL

Ran the same test on a CentOS Machine and I did not see this problem

What did you do?

Attempted to server files via HTTP that an application using curl can download.

package main

import (
        "log"
        "net/http"
)

func main() {
        fs := http.FileServer(http.Dir("./"))
        http.Handle("/repo/", http.StripPrefix("/repo/", fs))

        log.Println("Listening...")
        http.ListenAndServe(":80", nil)
}

What did you expect to see?

Curl to download the files.

What did you see instead?

The file server worked for file that were roughly less than 5M, but failed for larger files with curl error 18. The curl command line tool also reproduces this error:

curl http://<IP>/repo/<FILE> --output <FILE>
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                                    Dload  Upload   Total   Spent     Left  Speed
 56 7257k   56 4096k    0     0      4970k      0       0:00:01 --:--:--  0:00:01 4970k
curl: (18) transfer closed with 3237034 bytes remaining to read

Wget (busybox version) also has the same problem:

wget http://<IP>/repo/<FILE> -O <FILE>
Connecting to <IP> (<IP>:80)
pa-config-1-r0.corte   6% |**********                                                      |   460k  0:00:14 ETA
<STOPS and the file is only ~4M>

However using pythons web server, wget and curl both work
sudo python3 -m http.server 80

@bradfitz

This comment has been minimized.

Copy link
Contributor

@bradfitz bradfitz commented Aug 22, 2018

The WSL team want you to file bugs to them about any differences from actual Linux.

We generally don't investigate these things, or only do some when it appears we might be violating Linux or POSIX APIs in undocumented ways, or do so out of curiosity.

But please file a WSL bug and link it here.

We probably won't be investigating this. It's WSL's job to act like Linux and they don't want projects treating them like its own port.

@hullarb

This comment has been minimized.

Copy link

@hullarb hullarb commented Jun 18, 2019

i run into the same issue and while debugging it i saw that CopyN in the fileserver returns EOF early:
https://golang.org/src/net/http/fs.go#L296
(changing that line to readall and write works just fine if you need a quick workaround)

@bradfitz i wondered if the ignored error could be checked and logged to make easier to debug issues happening there?

hullarb added a commit to hullarb/go that referenced this issue Aug 27, 2019
Logging the error returned by CopyN could help finding the root cause
of issues like golang#27128.
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Aug 27, 2019

Change https://golang.org/cl/191971 mentions this issue: net/http: log error if fileserver fails to copy the file content

hullarb added a commit to hullarb/go that referenced this issue Sep 12, 2019
The errors returned during copying the file content in ServeContent,
ServeFile and FileServer were ignored to avoid excessive, meaningless
logging. This commit introduces error filtering and logging to ensure
that the errors occurring during reading the content are logged.

Updates golang#27128
hullarb added a commit to hullarb/go that referenced this issue Sep 12, 2019
The errors returned during copying the file content in ServeContent,
ServeFile and FileServer were ignored to avoid excessive, meaningless
logging. This commit introduces error filtering and logging to ensure
that the errors occurring during reading the content are logged.

Updates golang#27128
hullarb added a commit to hullarb/go that referenced this issue Sep 12, 2019
The errors returned during copying the file content in ServeContent,
ServeFile and FileServer were ignored to avoid excessive, meaningless
logging. This commit introduces error filtering and logging to ensure
that the errors occurring during reading the content are logged.

Updates golang#27128
hullarb added a commit to hullarb/go that referenced this issue Oct 24, 2019
The errors returned during copying the file content in ServeContent,
ServeFile and FileServer were ignored to avoid excessive, meaningless
logging. This commit introduces error filtering and logging to ensure
that the errors occurring during reading the content are logged.

Updates golang#27128
hullarb added a commit to hullarb/go that referenced this issue Oct 24, 2019
Errors returned during copying the file content in ServeContent,
ServeFile and FileServer were ignored to avoid excessive, meaningless
logging. This commit introduces error filtering and logging to ensure
that the errors occurring during reading the content are logged.

Updates golang#27128
hullarb added a commit to hullarb/go that referenced this issue Oct 30, 2019
Errors returned during copying the file content in ServeContent,
ServeFile and FileServer were ignored to avoid excessive, meaningless
logging. This commit introduces error filtering and logging to ensure
that the errors occurring during reading the content are logged.

Updates golang#27128
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.