[release] 2.8 release perf logs #40571
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
vitsai
added
P0
|
w00h00 thank you, I'll leave this up for your team to review then. Also they normally try something like this #29615 to make it easier to review. |
|
|
Diff here: #40572 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
For |
|
|
Yeah, but given the release branch is just a prefix of the master branch, I belive it's more of flakiness. Could we rerun it? |
|
Will run again after cherry picks, should give more signal on those edge of distribution values |
|
The new ones from today. Still waiting on @GeneDer for 2.7.1 release logs to compare against (instead of 2.7.0) |
|
The dashboard related latency for 
The dashboard latency for 
|
|
Variance for 
|
|
|
Signed-off-by: vitsai <victoria@anyscale.com>
Signed-off-by: vitsai <vitsai@cs.stanford.edu>
|
This one is against 2.7.1 |
|
|
|
I believe there's some regression in The test is testing submitting of 1M tasks from the driver (which overloads the task backend buffer on the driver worker), given other more realistic tests like 
|
This was referenced Oct 25, 2023
|
8 tasks
|
|
@rickyyx Still seeing 1000000_queued_time |
|
I think it would be variance: 177 is on the low end, while 199 is on the high end for this metric: 
|
|
So on the same commit:
I think we should ignore this as variance. |
|
|
This seems regression since we had a run on 2.8.0 with non-regressed perf just before: https://buildkite.com/ray-project/release-tests-branch/builds/2365#018b843e-61ff-4d6d-880e-0a10b6b122fa |
|
And others metrics look like variance. |
rickyyx
approved these changes
Nov 1, 2023
rkooo567
approved these changes
Nov 2, 2023
| 119.28968447369434 | ||
| ], | ||
| "1_n_actor_calls_async": [ | ||
| 9581.728569086026, |
| { | ||
| "perf_metric_name": "single_client_tasks_async", | ||
| "perf_metric_type": "THROUGHPUT", | ||
| "perf_metric_value": 8643.833466025399 |
| @@ -0,0 +1,13 @@ | |||
| { | |||
| "broadcast_time": 82.940892212, | |||
There was a problem hiding this comment.
this has a slight regression
| { | ||
| "perf_metric_name": "stage_2_avg_iteration_time", | ||
| "perf_metric_type": "LATENCY", | ||
| "perf_metric_value": 68.77781887054444 |
|
is this still relevant and a release-blocker for ray29? |
|
@vitsai should i merge this? |
ujjawal-khare
pushed a commit
to ujjawal-khare-27/ray
that referenced
this pull request
Nov 29, 2023
Signed-off-by: vitsai <victoria@anyscale.com> Signed-off-by: vitsai <vitsai@cs.stanford.edu>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
Related issue number
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.method in Tune, I've added it in
doc/source/tune/api/under thecorresponding
.rstfile.