-
Notifications
You must be signed in to change notification settings - Fork 4.5k
[WIP] Swap to a random number generator that doesn't use atomics/synchronization on hot path #25551
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
…ation since the Java Random and ThreadLocalRandom are thread safe and use atomics and/or synchonization internally. This is for apache#21250
.../java/harness/src/main/java/org/apache/beam/fn/harness/data/PCollectionConsumerRegistry.java
Outdated
Show resolved
Hide resolved
…a/PCollectionConsumerRegistry.java
Codecov Report
@@ Coverage Diff @@
## master #25551 +/- ##
=======================================
Coverage 72.65% 72.65%
=======================================
Files 758 758
Lines 100681 100681
=======================================
Hits 73154 73154
Misses 26118 26118
Partials 1409 1409
Flags with carried forward coverage won't be shown. Click here to find out more.
📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more |
|
Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment |
|
Run Java PreCommit |
|
Run Java_PVR_Flink_Batch PreCommit |
|
Run Python_Runners PreCommit |
|
Run SQL_Java11 PreCommit |
|
Run SQL_Java17 PreCommit |
| // samplingCutoff / samplingTokenUpperBound. This algorithm may be refined | ||
| // later. | ||
| samplingToken = Math.min(samplingToken + 1, SAMPLING_TOKEN_UPPER_BOUND); | ||
| return randomGenerator.nextInt(samplingToken) < SAMPLING_CUTOFF; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Even if we switch to a thread-local RNG, I'm concerned that calling nextInt on every single value will be more expensive than doing the fast reservoir sampling algorithm that we had here before.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah this is still WIP and marked it as so. I'm still trying to root cause why the existing implementation shows up on FlexWordCount profiles.
|
This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the dev@beam.apache.org list. Thank you for your contributions. |
|
This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. |
The Java Random and ThreadLocalRandom are thread safe and use atomics and/or synchonization internally.
This is for #21250
Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:
addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, commentfixes #<ISSUE NUMBER>instead.CHANGES.mdwith noteworthy changes.See the Contributor Guide for more tips on how to make review process smoother.
To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md
GitHub Actions Tests Status (on master branch)
See CI.md for more information about GitHub Actions CI.