Add perf metrics for 2.44.0 #51427
Merged
Add perf metrics for 2.44.0 #51427
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
khluu
commented
Mar 17, 2025
khluu
commented
Signed-off-by: Lonnie Liu <lonnie@anyscale.com>
Collaborator
Noise
Noise |
jjyao
approved these changes
Mar 17, 2025
dhakshin32
pushed a commit
to dhakshin32/ray
that referenced
this pull request
Mar 27, 2025
``` REGRESSION 9.35%: client__get_calls (THROUGHPUT) regresses from 1094.7883444776185 to 992.4456902391204 in microbenchmark.json REGRESSION 7.87%: tasks_per_second (THROUGHPUT) regresses from 399.43954902981744 to 367.9840802358416 in benchmarks/many_tasks.json REGRESSION 6.60%: multi_client_put_gigabytes (THROUGHPUT) regresses from 43.246981615749526 to 40.39150444280067 in microbenchmark.json REGRESSION 5.16%: client__tasks_and_put_batch (THROUGHPUT) regresses from 14341.529664523765 to 13601.436104861408 in microbenchmark.json REGRESSION 5.03%: 1_1_actor_calls_concurrent (THROUGHPUT) regresses from 5402.532852540871 to 5130.570133178275 in microbenchmark.json REGRESSION 4.83%: 1_1_actor_calls_async (THROUGHPUT) regresses from 8588.075503140139 to 8173.653446206568 in microbenchmark.json REGRESSION 4.71%: single_client_tasks_and_get_batch (THROUGHPUT) regresses from 6.116479739439202 to 5.828378076935622 in microbenchmark.json REGRESSION 4.06%: single_client_get_calls_Plasma_Store (THROUGHPUT) regresses from 10975.200393255369 to 10529.193272608605 in microbenchmark.json REGRESSION 3.71%: client__tasks_and_get_batch (THROUGHPUT) regresses from 0.9551721070094008 to 0.9197513826205774 in microbenchmark.json REGRESSION 3.25%: 1_1_actor_calls_sync (THROUGHPUT) regresses from 2024.9514970549762 to 1959.1925407193576 in microbenchmark.json REGRESSION 2.78%: single_client_put_gigabytes (THROUGHPUT) regresses from 18.30617444315663 to 17.79739662942353 in microbenchmark.json REGRESSION 1.46%: client__1_1_actor_calls_async (THROUGHPUT) regresses from 1057.2932167754398 to 1041.8730021547178 in microbenchmark.json REGRESSION 1.32%: 1_n_actor_calls_async (THROUGHPUT) regresses from 8168.440029557936 to 8060.698907411474 in microbenchmark.json REGRESSION 1.19%: single_client_tasks_sync (THROUGHPUT) regresses from 981.51641421362 to 969.8384217890384 in microbenchmark.json REGRESSION 0.89%: client__1_1_actor_calls_concurrent (THROUGHPUT) regresses from 1056.4662855748954 to 1047.1016344870811 in microbenchmark.json REGRESSION 0.58%: actors_per_second (THROUGHPUT) regresses from 591.3775923644333 to 587.9457127979538 in benchmarks/many_actors.json REGRESSION 0.56%: 1_1_async_actor_calls_sync (THROUGHPUT) regresses from 1434.2085547024217 to 1426.2018801386466 in microbenchmark.json REGRESSION 116.92%: dashboard_p50_latency_ms (LATENCY) regresses from 32.123 to 69.681 in benchmarks/many_actors.json REGRESSION 59.07%: dashboard_p99_latency_ms (LATENCY) regresses from 589.9 to 938.359 in benchmarks/many_tasks.json REGRESSION 57.53%: dashboard_p95_latency_ms (LATENCY) regresses from 398.245 to 627.361 in benchmarks/many_tasks.json REGRESSION 53.36%: dashboard_p50_latency_ms (LATENCY) regresses from 89.962 to 137.963 in benchmarks/many_tasks.json REGRESSION 37.60%: dashboard_p99_latency_ms (LATENCY) regresses from 3067.405 to 4220.801 in benchmarks/many_actors.json REGRESSION 12.91%: stage_0_time (LATENCY) regresses from 6.343268156051636 to 7.161974191665649 in stress_tests/stress_test_many_tasks.json REGRESSION 10.77%: dashboard_p95_latency_ms (LATENCY) regresses from 2575.96 to 2853.454 in benchmarks/many_actors.json REGRESSION 6.85%: dashboard_p99_latency_ms (LATENCY) regresses from 252.85 to 270.166 in benchmarks/many_pgs.json REGRESSION 2.52%: 10000_get_time (LATENCY) regresses from 23.620077062999997 to 24.215384834000005 in scalability/single_node.json REGRESSION 2.22%: avg_iteration_time (LATENCY) regresses from 1.1939783954620362 to 1.220467975139618 in stress_tests/stress_test_dead_actors.json REGRESSION 1.80%: stage_3_time (LATENCY) regresses from 1829.902144908905 to 1862.925583600998 in stress_tests/stress_test_many_tasks.json REGRESSION 1.73%: 1000000_queued_time (LATENCY) regresses from 191.976472028 to 195.30269835 in scalability/single_node.json REGRESSION 1.56%: time_to_broadcast_1073741824_bytes_to_50_nodes (LATENCY) regresses from 17.602684142 to 17.87641767999999 in scalability/object_store.json REGRESSION 1.12%: 10000_args_time (LATENCY) regresses from 18.656692702999997 to 18.865748501 in scalability/single_node.json REGRESSION 0.60%: stage_2_avg_iteration_time (LATENCY) regresses from 39.46649179458618 to 39.70143375396729 in stress_tests/stress_test_many_tasks.json REGRESSION 0.45%: 107374182400_large_object_time (LATENCY) regresses from 29.23165342300001 to 29.36276392100001 in scalability/single_node.json ``` Signed-off-by: Lonnie Liu <lonnie@anyscale.com> Co-authored-by: Lonnie Liu <lonnie@anyscale.com> Signed-off-by: Dhakshin Suriakannu <d_suriakannu@apple.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.