Fix performance of HTTP server (for strict entities at least)

[jrudolph]

i.e. only for Strict entities where we can now completely get rid of per-request materialization.

This includes a fix for #19240 and also the corresponding issue on the rendering side.

This is just a quick try which makes a few tests fail probably because some boundary conditions need to be fixed properly.

With this patch performance is up to ~ 90k RPS (from 15k without the patch) on my machine.

/cc @sirthias

[akka-ci]

Test FAILed.

[akka-ci]

Test FAILed.

[jrudolph]

Cleaned up and rebased (also I added an issue for the second part: #19352).

It is interesting that you don't need to actually touch the stage functionality (on both sides). The only thing that is needed is some slight additions to the already existing GraphStages that handle the substreams properly. I wonder if some form of this would even be generic enough to be included in GraphStages themselves as helper methods.

On the rendering side the only thing needed would be an emit(Source) which would transfer all elements from the Source to the given Outlet.

On the parsing side, something similar is needed, i.e. which creates a substream, wraps that substream (in an HttpEntity in that case) and pushes it and then transfers input elements to that substream as long as (or until) some predicate holds.

[akka-ci]

Test FAILed.

[jrudolph]

Here is the current benchmark with and without this PR:

RPS	Fusing enabled	What
23k	no	akka-http 2.0.1
23k	yes	akka-http 2.0.1
67k	no	This PR
115k	yes	This PR

It turns out that fusing has no effect in the original version because basically all the time is spent in materialization. Only after per-request materialization was removed the benefits of fusing can actually reap their benefits.

[jrudolph]

The test failure was in akka.stream.scaladsl.QueueSinkSpec, so most likely unrelated.

[ktoso]

Those look very promising, and also seem to overlap with what Roland observed before xmas when benchmarking (that things did not make much of a difference then). Thanks a lot!
I'll want to read the first commit here in detail before merging, will do that soon :)

[ktoso]

PLS BUILD

[akka-ci]

Test PASSed.

[ktoso]

Promising to (1st commit, 2nd already done) review (and hope to merge) today.

[ktoso]

Finally read through the entire thing, looks good! Thanks a lot and sorry for the delay @jrudolph!

[jrudolph]

Cool, thanks for reviewing and merging, @ktoso.