net: TCP short writes to MariaDB

[johndoughnut]

What version of Go are you using (go version)?

go version go1.10.3 windows/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\Aviad\AppData\Local\go-build
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\Aviad\go
set GORACE=
set GOROOT=C:\Go
set GOTMPDIR=
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Users\Aviad\AppData\Local\Temp\go-build035776318=/tmp/go-build -gno-record-gcc-switches

What did you do?

1. Create a TCP socket to mariadb db server.
2. Wrap the socket with bufio.Writer
3. Write() and Flush() to the buffered writer.

What did you expect to see?

Either bufio.Writer should keep writing until everything is written, or provide means to recover from the error and retry.

What did you see instead?

Got an ErrShortWrite from Flush(). Cannot recover from this error.

bufio.Writer: https://github.com/golang/go/blame/master/src/bufio/bufio.go#L577

This may be an issue with 10.x - happened after upgrade.

[bradfitz]

bufio is not and should not be in the business of "retrying". If your underlying Writer fails, game over.

The real question is why you're getting short writes to TCP connections on Windows.

How can we reproduce this? Do you have code?

[johndoughnut]

Reproduction is not trivial. Perhaps the issue is with localhost TCP connections. It may take some time to prepare a clean reproduction.

What is the rationale for making short writes a sticky error? I would guess many io.Writer implementations could produce short writes.

[bradfitz]

What is the rationale for making short writes a sticky error?

All write errors are a sticky error.

I would guess many io.Writer implementations could produce short writes.

They should not. That's an error. The io.Writer interface is not like io.Reader. A Reader permits short reads. A Writer does not.

[johndoughnut]

After more investigations, it appears that the cause for this error was concurrent writes to bufio.Writer.
If possible, it would be nice to detect this situation and report it using a different error.

[bradfitz]

We do have tools to detect such cases:

https://blog.golang.org/race-detector
https://golang.org/doc/articles/race_detector.html

I'll close this. Thanks for the update.