Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(storage): iter metrics should include init duration #3462

Merged
merged 6 commits into from
Jun 28, 2022

Conversation

skyzh
Copy link
Contributor

@skyzh skyzh commented Jun 25, 2022

Signed-off-by: Alex Chi iskyzh@gmail.com

I hereby agree to the terms of the Singularity Data, Inc. Contributor License Agreement.

What's changed and what's your intention?

Previously we only take scan duration into account, which is not accurate enough to reflect the real time taken by each iterator.

Checklist

  • I have written necessary docs and comments
  • I have added necessary unit tests and integration tests
  • All checks passed in ./risedev check (or alias, ./risedev c)

Refer to a related PR or issue link (optional)

Signed-off-by: Alex Chi <iskyzh@gmail.com>
@skyzh skyzh requested a review from BugenZhao June 25, 2022 12:07
@github-actions github-actions bot added the type/fix Bug fix label Jun 25, 2022
Signed-off-by: Alex Chi <iskyzh@gmail.com>
Signed-off-by: Alex Chi <iskyzh@gmail.com>
@skyzh
Copy link
Contributor Author

skyzh commented Jun 25, 2022

Also we're using mininstant (thanks to @breezewish for recommendation!) for Instant, which should reduce overhead of collecting time metrics.

@skyzh skyzh added the mergify/can-merge Indicates that the PR can be added to the merge queue label Jun 25, 2022
@codecov
Copy link

codecov bot commented Jun 25, 2022

Codecov Report

Merging #3462 (e1d9086) into main (6596612) will decrease coverage by 0.00%.
The diff coverage is 56.25%.

@@            Coverage Diff             @@
##             main    #3462      +/-   ##
==========================================
- Coverage   74.48%   74.48%   -0.01%     
==========================================
  Files         770      770              
  Lines      108126   108140      +14     
==========================================
+ Hits        80542    80546       +4     
- Misses      27584    27594      +10     
Flag Coverage Δ
rust 74.48% <56.25%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
src/storage/src/monitor/monitored_store.rs 1.88% <0.00%> (-0.06%) ⬇️
src/storage/src/monitor/state_store_metrics.rs 86.38% <100.00%> (+0.30%) ⬆️
src/frontend/src/expr/utils.rs 98.74% <0.00%> (-0.51%) ⬇️
src/connector/src/filesystem/file_common.rs 80.35% <0.00%> (-0.45%) ⬇️
src/meta/src/rpc/server.rs 79.95% <0.00%> (-0.24%) ⬇️
src/meta/src/manager/id.rs 94.94% <0.00%> (ø)

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@skyzh
Copy link
Contributor Author

skyzh commented Jun 25, 2022

image

One interesting finding is that iterator creation + fetching first block is taking much more time than actual scanning, which indicates some issues. maybe related to #3459

Signed-off-by: Alex Chi <iskyzh@gmail.com>
@mergify mergify bot merged commit cb84b42 into main Jun 28, 2022
@mergify mergify bot deleted the skyzh/fix-iter-metrics branch June 28, 2022 09:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
mergify/can-merge Indicates that the PR can be added to the merge queue type/fix Bug fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants