Relax Segment Operation Throttler Permit limitation to zero#18366
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #18366 +/- ##
============================================
+ Coverage 63.43% 63.46% +0.03%
Complexity 1683 1683
============================================
Files 3253 3253
Lines 198841 198884 +43
Branches 30795 30798 +3
============================================
+ Hits 126136 126223 +87
+ Misses 62625 62589 -36
+ Partials 10080 10072 -8
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
| } | ||
|
|
||
| @Test(timeOut = 10_000) | ||
| public void testZeroPermitsBlocksAcquireUntilPermitsRestored() |
There was a problem hiding this comment.
I don't like this test, it has a lot of design smells IMO and could be flaky. We shouldn't be polling on Thread.State.WAITING, it's an internal thread state and not a synchronization primitive. Also this comment:
// Wait until the acquirer is parked inside acquire(). availablePermits() goes to -1 once it
// has decremented but not yet been granted a permit (AdjustableSemaphore behavior), so poll
// for that condition.
doesn't seem to match the code? Can't we just check the throttler queue length or something instead? This test feels like overkill for what's being asserted.
There was a problem hiding this comment.
I think the blocking part is trivial, might as well remove the test that tests blocking. I'll keep the test to verify the unblocking part.
|
Opened a docs follow-up PR for this change: pinot-contrib/pinot-docs#783 |
Description
Currently we disallow users to set throttler permit to zero (e.g.
pinot.server.max.segment.preprocess.parallelism=0). Relaxing this limitation so that these throttlers can be used as kill switches functionally, to disable certain segment operations.