#827 #1679 Improve performance of Request Mapper

[ggnaegi]

Fixes #1679 #827

• RequestMapper performance issue: ToByteArray(request.Body) #1679
• Low performance even with logging disabled #827

Proposed Changes

• implementing custom HttpContent, largely inspired by the yarp project.
  https://github.com/microsoft/reverse-proxy/blob/main/src/ReverseProxy/Forwarder/StreamCopyHttpContent.cs

/// Conclusion: by overriding HttpContent.SerializeToStreamAsync,
/// we have full control over pumping bytes to the target stream for all protocols
/// (except Web Sockets, which is handled separately).

@raman-m @RaynaldM I have written a new benchmark, called payload benchmarks (part of PR too). We send various payloads (empty, 25, 50, 100, 1000, 5000, 10000, 20000 json objects), which involves the use of RequestMapper, and evaluate performance and memory usage. I checked that the target client was receiving the payload.

@raman-m @weichaohh As a colleague pointed out, I didn't address the issue of smaller payloads with a length < than the Default Buffer Size. Therefore I have added smaller payloads to the benchmark's payloads. Some are smaller than the Large Object Heap threshold, 85KB. So I modified the copy logic, and made sure that the default buffer size was only used when the promised / announced content length was >= than the Default Buffer Size.

First I have checked the performance of the develop branch code:

Figure 1: develop branch code

And then I checked the results of this PR

Figure 2:PR code

As you can see, we have a performance gain, but also lower memory usage.
Worst case: 92 MB vs 977 KB for the 20'000 objects payload

[ggnaegi]

@raman-m this is largely inspired by yarp.

[raman-m]

⚠️ Current build 2178 has failed! This is well-known issue #1706

[ggnaegi]

@RaynaldM @raman-m I will provide some benchmarks...

[raman-m]

@RaynaldM @raman-m I will provide some benchmarks...

That will be awesome to attach benchmark results of old version code and new one. Thanks!
Screenshots are better to attach.

[raman-m]

I'll review this PR after documentation release... OK?

[ggnaegi]

I'll review this PR after documentation release... OK?

Yeah fine, I will check the performance with 1-2 benchmarks first.

[ggnaegi]

@raman-m Ok, code is now review ready.

[raman-m]

Will this PR fix #749 ?

[raman-m]

Ready for delivery! ✔️

Thanks Gui for your efforts! 😍
This will be the brilliant feature of November'23 release! 💎

[RaynaldM]

We've had this PR in production for 5 days (plus a trial run a few days earlier), it's working well, and we haven't had any Out of Memory issues, or even a slightly lower memory footprint from the app.

[raman-m]

@RaynaldM commented on Dec 14

🆗 Good to know! 👍
Please, watch the system in production further... OOM and Docker container vs K8s pods restarts also?!