GEODE-6424: Greatly improves statistic counter storage throughput.#3204
Merged
jake-at-work merged 5 commits intoapache:developfrom Feb 20, 2019
Merged
GEODE-6424: Greatly improves statistic counter storage throughput.#3204jake-at-work merged 5 commits intoapache:developfrom
jake-at-work merged 5 commits intoapache:developfrom
Conversation
Reduces thread contention by using LongAdder and DoubleAdder to store counters. Benchmarking (on specific hardware) showed Atomic50StatisticsImpl could perform about 41M increments/second regardless of the number of threads updating the counter. Poor use of volatile memory access and CAS operations created unnecessary contention. The replacement, StripedStatisticsImpl, uses LongAdder and DoubleAdder to reduce contention. The same benchmark showed the throughput in increments/second scale nearly linearly up to the physical hardware threads of the host, seeing values as high as 2.8B increments/second on a 36 thread host.
jake-at-work
requested review from
kirklund,
metatype,
nabarunnag and
upthewaterspout
Contributor
Author
|
See https://issues.apache.org/jira/browse/GEODE-6424 for benchmarking details. |
LocalStatisticsImpl benchmarks even worse that Atomic50StatisticsImpl. It is tightly integrated with OS stats.
| public static void compareStatArchiveFiles(final File expectedStatArchiveFile, | ||
| final File actualStatArchiveFile) throws IOException { | ||
| System.out.println(actualStatArchiveFile); | ||
| System.out.println(expectedStatArchiveFile); |
Contributor
There was a problem hiding this comment.
Are these new System.out.printlns intentional?
Contributor
Author
There was a problem hiding this comment.
Whoops! Didn’t mean to check that in. Thanks!
upthewaterspout
approved these changes
Feb 19, 2019
jake-at-work
added a commit
that referenced
this pull request
Feb 20, 2019
…3204) Reduces thread contention by using LongAdder and DoubleAdder to store counters. Benchmarking (on specific hardware) showed Atomic50StatisticsImpl could perform about 41M increments/second regardless of the number of threads updating the counter. Poor use of volatile memory access and CAS operations created unnecessary contention. The replacement, StripedStatisticsImpl, uses LongAdder and DoubleAdder to reduce contention. The same benchmark showed the throughput in increments/second scale nearly linearly up to the physical hardware threads of the host, seeing values as high as 2.8B increments/second on a 36 thread host. LocalStatisticsImpl benchmarks even worse that Atomic50StatisticsImpl. It is tightly integrated with OS stats. (cherry picked from commit 888d2b2)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reduces thread contention by using LongAdder and DoubleAdder to store
counters. Benchmarking (on specific hardware) showed
Atomic50StatisticsImpl could perform about 41M increments/second
regardless of the number of threads updating the counter. Poor use of
volatile memory access and CAS operations created unnecessary
contention. The replacement, StripedStatisticsImpl, uses LongAdder and
DoubleAdder to reduce contention. The same benchmark showed the
throughput in increments/second scale nearly linearly up to the
physical hardware threads of the host, seeing values as high as 2.8B
increments/second on a 36 thread host.
Thank you for submitting a contribution to Apache Geode.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
For all changes:
Is there a JIRA ticket associated with this PR? Is it referenced in the commit message?
Has your PR been rebased against the latest commit within the target branch (typically
develop)?Is your initial contribution a single, squashed commit?
Does
gradlew buildrun cleanly?Have you written or updated unit tests to verify your changes?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and
submit an update to your PR as soon as possible. If you need help, please send an
email to dev@geode.apache.org.