New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
In-memory engine: collector iteration related metrics #16735
Conversation
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
[REVIEW NOTIFICATION] This pull request has been approved by:
To complete the pull request process, please ask the reviewers in the list to review by filling The full list of commands accepted by this bot can be found here. Reviewer can indicate their review by submitting an approval review. |
EK: KvEngine, | ||
EC: RangeCacheEngine, | ||
{ | ||
collector: Either< |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to have HybridEngineIterMetricsCollector
again?
It seems to me as long as the caller of HybridEngineIterMetricsCollector
has the MetricsExt instance
, it can call the relevant APIs directly.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Disk engine collector and range cache engine collector are different and have their own implementation (RocksIterMetricsCollector
and RangeCacheIterMetricsCollector
).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, understood. But for one HybridEngineIterMetricsCollector instance, it's either the RocksDB or RangeCache, which have the same interface. So why don't we use this same interface directly instead of a wrapper.
@@ -117,8 +117,19 @@ pub trait RefIterable { | |||
fn iter(&self, opts: IterOptions) -> Result<Self::Iterator<'_>>; | |||
} | |||
|
|||
pub trait IterMetricsCollector { | |||
fn internal_delete_skipped_count(&self) -> usize; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Should we include total scanned count as well? Otherwise we don't know the percentage of delete skipped or internal skipped.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I will leave a todo and we should add these metrics when we need. This PR is to make the in-memory engine returns iteration related metrics like what did in RocksEngine which only include delete skipped count.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Some comments, overall LGTM.
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
/run-test retry=7 |
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
{ | ||
type Collector = HybridEngineIterMetricsCollector<EK, EC>; | ||
|
||
fn metrics_collector(&self) -> Self::Collector { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
IMO, here we don't need to wrap a collector object HybridEngineIterMetricsCollector
at all.
Instead we can do this:
fn metrics_collector(&self) -> impl IterMetricsCollector
The current code is kind of manual implementation of Polymorphism.
What do you think @afeinberg
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think we can use -> impl IterMetricsCollector
here. impl trait is static dispatch, which determines the return type at compile time. However, what we need is dynamic dispatch here and determines the return type according to the type of self.iter. We can use Box<dyn IterMetricsCollector>
here but the performance should be poorer.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No. impl IterMetricsCollector is dynamic dispatch
@SpadeA-Tang
/merge |
@SpadeA-Tang: It seems you want to merge this PR, I will help you trigger all the tests: /run-all-tests You only need to trigger
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository. |
This pull request has been accepted and is ready to merge. Commit hash: 789faf6
|
What is changed and how it works?
Issue Number: Ref #16141
What's Changed:
Collector iteration related metrics just like what does in rocksdb.
Related changes
pingcap/docs
/pingcap/docs-cn
:Check List
Tests
Side effects
Release note