[exporterhelper] Fix race in concurrency check in batch sender leading to smaller batch sizes

[carsonip]

Description:

Although activeRequests is atomic, it is possible for 2 arriving
requests to both increment activeRequests, and when entering the
critical section of bs.activeRequests.Load() >= bs.concurrencyLimit,
both times it evaluates to true. The correct behavior should be that
only the 2nd check is true.

Remove the workaround in tests that bypassed the bug.

---

Even with this change, the results are slightly better but still depend on goroutine scheduling.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 90.96%. Comparing base (cc485e0) to head (88358c8).

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #9761   +/-   ##
=======================================
  Coverage   90.96%   90.96%           
=======================================
  Files         353      353           
  Lines       18626    18628    +2     
=======================================
+ Hits        16943    16945    +2     
  Misses       1356     1356           
  Partials      327      327           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[dmitryax]

Thanks for the fix, @carsonip! I'm curious to know more about your adoption of this API. Any feedback is welcome.

[carsonip]

@dmitryax not sure if you caught my latest edit. In my tests, with the bugfix, the behavior is slightly better, but there are still a lot of requests with just 1 item. It boils down to goroutine scheduling. It may require a bigger change in how queue sender and batch sender interact with each other to eliminate this issue.

I'm curious to know more about your adoption of this API. Any feedback is welcome.

I'm still experimenting with it. Batch sender blocking the send() function with <-batch.done is quite a nice workaround to avoid immediately calling the callback in queue sender which deletes items from persistent queue. Maybe I can give a better summary of our usage later!