Ensure FlowControlled data frames will be correctly removed from the …

[normanmaurer]

…flow-controller when a write error happens.

Motivation:

When a write error happens during writing of flowcontrolled data frames we miss to correctly detect this in the write loop which may result in an infinite loop as we will never detect that the frame should be removed from the queue.

Modifications:

• When we fail a flowcontrolled data frame we ensure that the next frame.write(...) call will signal back that the whole frame was handled and so can be removed.
• Add unit test.

Result:

Fixes #8707.

[rkapsi]

@normanmaurer LGTM. I went through a few memory dumps I had saved (September 5, 2018 being oldest) and the condition queue.isEmpty() is true in all cases. That should do the trick. Best Netty release ever!

[Scottmitch]

this has potentially interesting implications on returning bytes to flow control. consider adding a unit test that involves the real flow control and that the flow control window is properly tracked.

Also consider writing a test that would demonstrate the original issue (infinite loop) to verify the scenario doesn't re-occur. The interactions between the encoder, flow controller, and pipeline events may lead to interesting combinations which is not obvious if this will resolve the original issue. Http2ConnectionRounttripTest#writeOfEmptyReleasedBufferQueuedInFlowControllerShouldFail hits the error condition, I wonder if we can enhance this test or do something similar to exercise the desired condition.

[normanmaurer]

@Scottmitch will check and see if I can add more tests. That said I think it should work as expected as we only reset after write is called and we failed it before.

[normanmaurer]

@rkapsi @Scottmitch I was able to add a unit test that would result in the endless loop before the fix:

3d2cff4

[rkapsi]

Wonder if we want to move the addListener() after the write(...) as well. Right now it's skewing the stack should the remove(...) complete the promise.

[normanmaurer]

Not sure I understand... can you show me some code ?

[rkapsi]

Something like that...

ChannelPromise writePromise = ctx.newPromise();
ctx.write(queue.remove(0, writePromise), writePromise)
    .addListener(this);

Otherwise if queue.remove(0, writePromise) was to complete the promise it'd potentially notify the listener first, followed by calling ctx.write(...). Stack (traces) will look strange. Not sure there would be other side effects due to the order in which things execute but I think you want the listener to get notified after the write call returns.

[normanmaurer]

Got it... I would like to investigate as a followup here. Trying to keep changes as minimal as possible atm

[normanmaurer]

@netty-bot test this please

[normanmaurer]

@netty-bot test this please

[carl-mastrangelo]

LGTM

[carl-mastrangelo]

LGTM