Skip to content

x/net/http2: Connection reset incorrectly in high load performance test #26956

@crowfrog

Description

@crowfrog

What version of Go are you using (go version)?

golang:1.10.0 linux

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

container: rhel
kubenet:

What did you do?

func (sc *serverConn) wroteFrame(res frameWriteResult) {
...
			sc.closeStream(st, errHandlerComplete)                  <=== _code1_
		}
...
	wr.replyToWriter(res.err)                                               <=== _code2_
...
}

In some cases, the server goroutine will be switch out execution context between code1 and code2.
If the handler goroutine switch back to execution context in these cases, that will make writeDataFromHandler() (writeHeaders() also?) failed and set responseWriterState to dirty.

func (sc *serverConn) writeDataFromHandler(stream *stream, data []byte, endStream bool) error {
...
	case <-stream.cw:
		// If both ch and stream.cw were ready (as might
		// happen on the final Write after an http.Handler
		// ends), prefer the write result. Otherwise this
		// might just be us successfully closing the stream.
		// The writeFrameAsync and serve goroutines guarantee
		// that the ch send will happen before the stream.cw
		// close.
		select {
		case err = <-ch:
			frameWriteDone = true
		default:
			return errStreamClosed
		}
	}
...
}
func (rws *responseWriterState) writeChunk(p []byte) (n int, err error) {
...
		if err := rws.conn.writeDataFromHandler(rws.stream, p, endStream); err != nil {
			rws.dirty = true
			return 0, err
		}
...
}

This issue can be reproduced easily if add sc.logf() between code1 and code2 in lower load performace test.
PS: I think in wroteFrame(), we shouldn'd call "wr.replyToWriter(res.err)" when closing stream with error. It will cause writeDataFromHandler() exit with no error. But in fact this stream/responseWriterState is already on incorrect status.

What did you expect to see?

no connection reset in load test

What did you see instead?

connecttion reset and traffic failed beofre new connection set up

Metadata

Metadata

Assignees

No one assigned

    Labels

    NeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions