[FLINK-19130] [core] Expose metrics for backpressure - #142
Closed
tzulitai wants to merge 9 commits into
Closed
Conversation
This commit extends the FunctionTypeMetrics interface and introduces a new FunctionDispatcherMetrics interface with a goal to expose the following backpressure-related metrics: - Number of blocked addresses (per function type) - Number of inflight async ops (per function type + per-operator)
…ypeMetricsRepository This commit introduces a new scoped-down interface FunctionTypeMetricsRepository that the existing StatefulFunctionsRepository now implements. This interface will be provided to components that needs access to per-function metrics.
This extends the ObjectContainer to set alias keys that returns the same instance. This is required for the StatefulFunctionsRepository, since we want to share the same instance for different keys (depending on which interface we expose to different components).
Now that we have multiple types of metrics with different scopes (per-function / per-operator), this interface should be renamed to convey that it is a factory specific for per-function scoped metrics.
Contributor
Author
|
Discussion point: Strictly speaking, the per-operator @igalshilman what do you think about this? |
As a preparation to expose FunctionTypeMetrics to functions for internal axcess only, AsyncWaiter is renamed to a more general-purpose name "InternalContext" so that it makes sense to add more internal-only context methods there.
Contributor
Author
|
Update: |
igalshilman
approved these changes
Sep 8, 2020
igalshilman
left a comment
igalshilman
left a comment
Contributor
There was a problem hiding this comment.
Thanks @tzulitai, LGTM!
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR adds in total 2 new metrics related to backpressure:
We also rejected to add the following metric, since after some discussion it doesn't seem to add much value:
This was not added, with the assumption that users would really only want to care about that some address has reached the maximum request batch size and was being blocked.
Verification
I verified this by running the Python Greeter example, with the following modifications to let backpressure happen more easily:
You can see the following metric charts in the Flink Web UI:
Brief changelog
FunctionTypeMetricsinterface to include the new metrics, and adds a newFunctionDispatcherMetricsinterface for per-operator metrics.FunctionTypeMetricsRepositoryand letStatefulFunctionsRepositoryextend it. Components that need to access function metrics will be passed this interface.ObjectContainerDI utility so that we can share same instance across different object labels.AsyncSinkto expose backpressure metrics.