Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lazy load help info #14305

Merged
merged 5 commits into from Jan 29, 2022
Merged

Lazy load help info #14305

merged 5 commits into from Jan 29, 2022

Conversation

kaos
Copy link
Member

@kaos kaos commented Jan 29, 2022

In order to get a more snappy feeling from the CLI help system, only load the information when it is actually referenced, rather than loading everything up-front.

The performance impact is not huge compared to main, but this opens up for adding more data without having to pay the cost in time up-front, for things like rules and api types info (see #14227)

On main 1e1fdab:

$ hyperfine --warmup=2 --runs=10 './pants help-advanced docker'
Benchmark 1: ./pants help-advanced docker
  Time (mean ± σ):      1.648 s ±  0.023 s    [User: 0.758 s, System: 0.396 s]
  Range (min … max):    1.624 s …  1.695 s    10 runs
 

This branch:

$ hyperfine --warmup=2 --runs=10 './pants help-advanced docker'
Benchmark 1: ./pants help-advanced docker
  Time (mean ± σ):      1.497 s ±  0.018 s    [User: 0.758 s, System: 0.386 s]
  Range (min … max):    1.481 s …  1.541 s    10 runs

And for help-all the difference is negligible:

# main
$ hyperfine --warmup=2 --runs=10 './pants help-all'
Benchmark 1: ./pants help-all
  Time (mean ± σ):      1.722 s ±  0.027 s    [User: 0.766 s, System: 0.387 s]
  Range (min … max):    1.691 s …  1.779 s    10 runs

# This branch
$ hyperfine --warmup=2 --runs=10 './pants help-all'
Benchmark 1: ./pants help-all
  Time (mean ± σ):      1.711 s ±  0.057 s    [User: 0.764 s, System: 0.388 s]
  Range (min … max):    1.631 s …  1.799 s    10 runs

There are a few places with "ugly" lazy loading code for the rule infos, which will be refactored and cleaned up in #14227

# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
Copy link
Contributor

@Eric-Arellano Eric-Arellano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Brilliant! That performance speedup is non-trivial 🚀

Please wait to merge till #14302 lands to help Yi out.

src/python/pants/help/help_info_extracter.py Outdated Show resolved Hide resolved
src/python/pants/help/help_info_extracter.py Outdated Show resolved Hide resolved
src/python/pants/help/help_info_extracter.py Outdated Show resolved Hide resolved
src/python/pants/help/help_info_extracter.py Show resolved Hide resolved
# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
# Rust tests and lints will be skipped. Delete if not intended.
[ci skip-rust]

# Building wheels and fs_util will be skipped. Delete if not intended.
[ci skip-build-wheels]
@kaos kaos enabled auto-merge (squash) January 29, 2022 20:09
@kaos kaos merged commit 27f23db into pantsbuild:main Jan 29, 2022
@kaos kaos deleted the help_load_time branch January 29, 2022 22:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants