-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[improve][broker] Support shrink for ConcurrentSortedLongPairSet #15354
Merged
Technoboy-
merged 4 commits into
apache:master
from
Technoboy-:fix-sorted-long-pair-set
Apr 28, 2022
Merged
[improve][broker] Support shrink for ConcurrentSortedLongPairSet #15354
Technoboy-
merged 4 commits into
apache:master
from
Technoboy-:fix-sorted-long-pair-set
Apr 28, 2022
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Technoboy-
added
area/broker
doc-not-needed
Your PR changes do not impact docs
release/2.9.3
release/2.8.4
release/2.10.1
labels
Apr 27, 2022
Technoboy-
changed the title
[improve][broker] Support shrink for ConcurrentSortedLongPairSet
[improve][broker] Support shrink for ConcurrentSortedLongPairSet
Apr 27, 2022
codelipenghui
approved these changes
Apr 27, 2022
Technoboy-
requested review from
315157973,
congbobo184,
gaoran10,
gaozhangmin and
Jason918
April 28, 2022 01:44
mattisonchao
approved these changes
Apr 28, 2022
Technoboy-
force-pushed
the
fix-sorted-long-pair-set
branch
from
April 28, 2022 07:25
88fd930
to
1ae7474
Compare
codelipenghui
approved these changes
Apr 28, 2022
/pulsarbot run-failure-checks |
mattisonchao
approved these changes
Apr 28, 2022
RobertIndie
approved these changes
Apr 28, 2022
RobertIndie
added
the
type/enhancement
The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages
label
Apr 28, 2022
gaoran10
reviewed
Apr 28, 2022
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, left a trivial comment.
...ar-common/src/main/java/org/apache/pulsar/common/util/collections/ConcurrentLongPairSet.java
Outdated
Show resolved
Hide resolved
nodece
approved these changes
Apr 28, 2022
shibd
approved these changes
Apr 28, 2022
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
codelipenghui
pushed a commit
that referenced
this pull request
Apr 28, 2022
) (cherry picked from commit 24d4d76)
codelipenghui
pushed a commit
that referenced
this pull request
Apr 28, 2022
) (cherry picked from commit 24d4d76)
codelipenghui
pushed a commit
that referenced
this pull request
Apr 29, 2022
) (cherry picked from commit 24d4d76)
nicoloboschi
pushed a commit
to datastax/pulsar
that referenced
this pull request
May 4, 2022
…che#15354) (cherry picked from commit 24d4d76) (cherry picked from commit 9e8c324)
nicoloboschi
pushed a commit
to datastax/pulsar
that referenced
this pull request
May 9, 2022
…che#15354) (cherry picked from commit 24d4d76) (cherry picked from commit f712861)
12 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
area/broker
cherry-picked/branch-2.8
Archived: 2.8 is end of life
cherry-picked/branch-2.9
Archived: 2.9 is end of life
cherry-picked/branch-2.10
doc-not-needed
Your PR changes do not impact docs
release/2.8.4
release/2.9.3
release/2.10.1
type/enhancement
The enhancements for the existing features or docs. e.g. reduce memory usage of the delayed messages
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
Sometimes the messages sent to consumers are delayed over 100ms and we find that the CPU is wasted on :
Then we find there are many empty datas in the map from dump :
ConcurrentSortedLongPairSet implements based on
ConcurrentLongPairSet
, andConcurrentLongPairSet
supports shrink, so support shrink for ConcurrentSortedLongPairSet to avoid iterator the empty data.But the original
removeIf
has a bug ifautoShrink=true
, because when iterator the section, the internaltable
may change afterremove
, so result in some data not removed.Reproduce Test:
Modification
removeIf
and add related test.Documentation
no-need-doc
(Please explain why)