net/http: ServeFile adds garbage at end of file - Stat() gets wrong size from Mac OS

[spekary]

Please answer these questions before submitting your issue. Thanks!

What did you do?

Called http.ServeFile on a javascript file with a ".js" ending in Mac OS, pointing to a file that is 40010 bytes long. This is confirmed by both counting the bytes using a text editor, and seeing the size in the Mac GetInfo window in the Finder.

What did you expect to see?

The file sent as is.

What did you see instead?

The file is sent, but 6 extra garbage bytes are tacked on to the end. Stepping in to ServeFile, the cuplrit appears to be that Stat() is showing a file size of 40016, and so ServeFile goes ahead and reports the files size in the http header as 40016, and also reads the buffer past the end of the file.

System details

go version go1.10 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/shannon/Library/Caches/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Sites/home/goradd"
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/gk/9w8fcc7c8xng8g001s6zggs80000gp/T/go-build314896890=/tmp/go-build -gno-record-gcc-switches -fno-common"
GOROOT/bin/go version: go version go1.10 darwin/amd64
GOROOT/bin/go tool compile -V: compile version go1.10
uname -v: Darwin Kernel Version 17.4.0: Sun Dec 17 09:19:54 PST 2017; root:xnu-4570.41.2~1/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.13.3
BuildVersion:	17D102
lldb --version: lldb-900.0.64
  Swift-4.0

This was also a problem in go 1.9.1

[peczenyj]

can you use cat -A or hexdump -C and check the last bytes?

you file may have non-printable chars in the end

does your file have some BOM / utf8 marker? is there any change to provide the original file + md5 / sha1 to look inside?

[spekary]

Doing some more investigation, the hexdump seems to have repaired the file. Very odd problem. I will keep monitoring to see if I can reproduce, but now it looks like some kind of filesystem anomally and not go related.