Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix DynamicRayRunner performance issues. Warmup more works in benchmarking script. #430

Merged
merged 18 commits into from
Jan 10, 2023

Conversation

xcharleslin
Copy link
Contributor

@xcharleslin xcharleslin commented Jan 7, 2023

DynamicRayRunner is now similar or faster than RayRunner on all questions of TPCH.

Benchmarking changes:

  • Add support for Ray's profiler in TPCH benchmarks. To use, set RAY_PROFILING=1. Official docs here.
  • Warm up more workers: one worker per CPU. Currently, we warm up one per node.

DynamicRayRunner changes:

  • Correctly track task parallelism. Before, we were overcounting tasks and letting CPUs idle.
  • Add spread strategy for reduce tasks.
  • Add logger statements and profiling wrapper.

TODO: refactor for cleanliness

@xcharleslin xcharleslin force-pushed the charles/spread branch 2 times, most recently from caf7a70 to 7e2a2cb Compare January 9, 2023 19:42
@xcharleslin xcharleslin marked this pull request as ready for review January 10, 2023 01:50
@xcharleslin xcharleslin changed the title Charles/spread Fix DynamicRayRunner performance issues. Warmup more works in benchmarking script. Jan 10, 2023
@xcharleslin
Copy link
Contributor Author

Main branch DynamicRayRunner:

Q1: 3.4603s     +1.94% vs RayRunner
Q2: 2.613255s  -40.59% vs RayRunner
Q3: 3.399833s  +40.57% vs RayRunner
Q4: 4.338846s   +5.49% vs RayRunner
Q5: 6.704545s  +58.23% vs RayRunner
Q6: 0.646179s   -2.78% vs RayRunner
Q7: 3.411119s  +31.45% vs RayRunner
Q8: 5.630377s  +41.63% vs RayRunner
Q9: 9.025225s  +54.58% vs RayRunner
Q10: 3.034456s +24.08% vs RayRunner

Total: 42s (+24% vs RayRunner, 34s)

This branch DynamicRayRunner:

Q1: 3.543052s    +4.38% vs RayRunner
Q2: 1.247474s   -71.64% vs RayRunner
Q3: 2.248951s    -7.01% vs RayRunner
Q4: 3.777719s    -8.15% vs RayRunner
Q5: 4.163764s    -1.73% vs RayRunner
Q6: 0.623651s    -6.17% vs RayRunner
Q7: 2.457648s    -5.29% vs RayRunner
Q8: 4.081357s    +2.67% vs RayRunner
Q9: 5.232197s   -10.38% vs RayRunner
Q10: 2.200268s  -10.03% vs RayRunner

Total: 30s (-13% vs RayRunner, 34s)

@xcharleslin xcharleslin merged commit 25028f1 into main Jan 10, 2023
@xcharleslin xcharleslin deleted the charles/spread branch January 10, 2023 02:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant