[autoscaler] Additional Autoscaler Metrics #16198
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ijrsvt
reviewed
Jun 2, 2021
ijrsvt
reviewed
Jun 2, 2021
|
@ijrsvt ah sorry, I still have the problem where I don't have auth to merge my own PRs. If you're good with this you can merge when you can go forward with the merge, otherwise we can wait for more reviewers |
|
@ckw017 Let's wait and see if @AmeerHajAli wants to say anythin! |
AmeerHajAli
reviewed
Jun 3, 2021
| @@ -332,6 +338,7 @@ def _update(self): | |||
| for node_id in nodes: | |||
There was a problem hiding this comment.
Should we add here a tracking of recovering nodes?
AmeerHajAli
reviewed
Jun 3, 2021
AmeerHajAli
reviewed
Jun 3, 2021
AmeerHajAli
added
the
@author-action-required
ckw017
removed
the
@author-action-required
|
added in a member to NodeUpdaterThread that's true if the thread was started to recovery an unhealthy node, and metrics:
|
ijrsvt
changed the title
mwtian
pushed a commit
that referenced
this pull request
Jun 5, 2021
mwtian
pushed a commit
that referenced
this pull request
Jun 5, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
Followup to #16066. Adds the following metrics
worker_launch_time(histogram), measured as time for provider.create_node to executeworker_update_time(histogram), measured as time for NodeLauncherThread.run() to successfully executeupdating_nodes(gauge), length of StandardAutoscaler.updatersfailed_create_nodes(counter), incremented whenever by number of nodes in batch whenever provider.create_node throws an exceptionfailed_updates(counter), tracks StandardAutoscaler.num_failed_updatessuccessful_updates(counter) tracks StandardAutoscaler.num_successful_updatesRelated issue number
Checks
scripts/format.shto lint the changes in this PR.