-
Notifications
You must be signed in to change notification settings - Fork 553
perf(s3stream): compute compaction delay using min timestamp instead of sorting #2984
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
perf(s3stream): compute compaction delay using min timestamp instead of sorting #2984
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR optimizes the computation of compactionDelayTime by replacing a sort-based approach with a single-pass minimum finding algorithm. This improves performance when calculating the delay time metric for compaction monitoring.
Key Changes:
- Replaces
List.sort()followed byget(0)with a manual iteration to find the minimumcommittedTimestamp - Eliminates unnecessary sorting overhead when only the minimum value is needed
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
s3stream/src/main/java/com/automq/stream/s3/compact/CompactionManager.java
Show resolved
Hide resolved
|
Thank you for your contribution @JasirVoriya . It looks fine overall, but I noticed there are some CI errors that could be resolved. |
Prevent delay calculation from using Long.MAX_VALUE; return early. Also remove unused import.
Fixed it, thanks for the heads-up @Gezi-lzq . |
Gezi-lzq
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
woshigaopp
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Mixficsol
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Issue Title
Performance: avoid sorting when computing
compactionDelayTime; use min timestampDescription
In
CompactionManager.start()’s periodic monitor task,compactionDelayTimeis computed by sorting server objects bycommittedTimestampand then taking the first element. This sorting is unnecessary because we only need the minimum timestamp. Sorting costs O(n log n) time; finding the minimum is O(n). With large object lists, the sort introduces avoidable CPU overhead and potential GC pressure.Affected Code
s3stream/src/main/java/com/automq/stream/s3/compact/CompactionManager.javastart()Benefits
Behavior
committedTimestamp.datais null/empty or an exception occurs, delay is set to0.Validation
s3streammodule:./gradlew :s3stream:compileJava -x testshould pass.Committer Checklist (excluded from commit message)