Improve Efficiency of MapStreaming

[vigith]

Summary

Today, we have the following constraints

• only an element can be processed at a time
• we create a map-stream (gRPC) for every batch (meaning streaming inside unary)
• our batch is a blocking batch and will not proceed if we have a slow message
• Read, Execute, Forward, Ack is an ordered (sync) workflow for a batch, we need to make it out-of-order non-blocking
• move away from blocking batch to bounded concurrency (X in a batch to always have X in the backlog)

We need to remove these constraints without affecting the correctness or completeness property.

Use Cases

• Better throughput
• Rewrite Map on top of MapStreaming

---

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.