Skip to content

perf(pm): cap install worker pressure#2996

Closed
elrrrrrrr wants to merge 1 commit into
exp/pm-install-clone-batch-completionsfrom
exp/pm-install-p3-limit-tuning
Closed

perf(pm): cap install worker pressure#2996
elrrrrrrr wants to merge 1 commit into
exp/pm-install-clone-batch-completionsfrom
exp/pm-install-p3-limit-tuning

Conversation

@elrrrrrrr
Copy link
Copy Markdown
Contributor

@elrrrrrrr elrrrrrrr commented May 19, 2026

Summary

Experiment for p3 cold install scheduling pressure.

  • cap install clone workers at 4 instead of scaling to 6/10 on larger runners
  • cap extract concurrency at 4 instead of 8
  • keep the scheduler architecture unchanged; only tune worker pressure around clone/extract materialization

Verification

  • cargo fmt
  • cargo test -p utoo-pm install_scheduler
  • cargo clippy -p utoo-pm --all-targets -- -D warnings --no-deps

GHA Benchmark

Conclusion: negative on GHA; do not integrate as-is.

Compared with #2989 latest Linux/npmjs run (26102983933), this worker-cap experiment (26105816781) regresses p0 and p3:

phase #2989 utoo #2996 utoo note
p0 full cold 7.34s, 65.1K/45.7K ctx 9.85s, 82.6K/52.8K ctx regression
p1 resolve 2.39s, 14.9K/19.3K ctx 2.77s, 15.5K/19.8K ctx run/network noise, no benefit
p3 cold install 5.30s, 52.4K/33.8K ctx 5.68s, 62.3K/40.6K ctx regression
p4 warm link 2.20s, 7.9K/4.6K ctx 2.17s, 7.9K/4.6K ctx neutral

The cap can improve one high-ctx internal environment, but GHA does not support making it the default.

@elrrrrrrr elrrrrrrr added A-Pkg Manager Area: Package Manager benchmark Run pm-bench on PR labels May 19, 2026
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request reduces the concurrency and worker limits for installation tasks, capping them at a maximum of 4. The changes affect the clone_worker_limit and extract_concurrency_limit functions, and the unit tests have been updated accordingly. A review comment suggests simplifying the clone_worker_limit implementation, as the current logic is unnecessarily complex and mathematically equivalent to a simple clamp operation.

Comment on lines 660 to +663
clone_limit
.saturating_div(2)
.saturating_add(2)
.clamp(1, clone_limit.max(1))
.clamp(1, clone_limit.clamp(1, 4))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The logic in clone_worker_limit has become unnecessarily complex with the nested clamping. Since the goal is to cap the worker limit at 4 while ensuring it doesn't exceed the total clone_limit (and stays at least 1), the entire expression is mathematically equivalent to clone_limit.clamp(1, 4).

For all $x \ge 1$, the formula $\lfloor x/2 \rfloor + 2$ is always greater than or equal to $\min(x, 4)$, meaning the previous scaling logic is now completely shadowed by the new cap. Simplifying this improves readability and maintainability.

Suggested change
clone_limit
.saturating_div(2)
.saturating_add(2)
.clamp(1, clone_limit.max(1))
.clamp(1, clone_limit.clamp(1, 4))
clone_limit.clamp(1, 4)

@elrrrrrrr
Copy link
Copy Markdown
Contributor Author

Closing this draft as a negative experiment: the GHA AB regressed p0 and p3 versus #2989, so the worker cap should not be integrated as-is.

@elrrrrrrr elrrrrrrr closed this May 19, 2026
@github-actions
Copy link
Copy Markdown

📊 pm-bench-phases · e6f9edc · linux (ubuntu-latest)

Workflow run — ant-design

PMs: utoo (this branch) · utoo-npm (latest published) · bun (latest)

npmjs.org

p0_full_cold

PM wall ±σ user sys RSS pgMinor
bun 15.43s 2.74s 9.92s 9.81s 571M 323.4K
utoo-next 10.44s 0.51s 10.70s 13.08s 984M 119.6K
utoo-npm 13.02s 2.75s 10.95s 13.37s 976M 121.0K
utoo 0.00s 0.00s 7.28s 6.23s 795M 117.0K
PM vCtx iCtx netRX netTX cache node_mod lock
bun 34.0K 12.0K 1.19G 7M 1.86G 1.75G 1M
utoo-next 169.1K 114.6K 1.16G 6M 1.71G 1.70G 2M
utoo-npm 184.2K 128.3K 1.16G 6M 1.71G 1.70G 2M
utoo 37.3K 17.5K 688M 5M 958M 952M -

p1_resolve

PM wall ±σ user sys RSS pgMinor
bun 10.26s 3.29s 3.70s 1.14s 465M 217.5K
utoo-next 6.62s 1.91s 5.72s 2.28s 610M 88.0K
utoo-npm 8.01s 2.24s 5.80s 2.26s 597M 81.4K
utoo 8.49s 1.95s 6.11s 1.73s 632M 120.7K
PM vCtx iCtx netRX netTX cache node_mod lock
bun 19.8K 1.6K 202M 3M 107M - 1M
utoo-next 98.0K 79.6K 200M 3M 7M 3M 2M
utoo-npm 100.7K 79.3K 200M 3M 7M 3M 2M
utoo 21.6K 18.6K 201M 3M 7M 3M 2M

p3_cold_install

PM wall ±σ user sys RSS pgMinor
bun 9.66s 4.94s 6.41s 10.11s 425M 184.1K
utoo-next 13.40s 2.25s 5.15s 11.38s 424M 61.3K
utoo-npm 11.41s 1.54s 5.25s 11.35s 442M 60.4K
utoo 10.02s 1.61s 5.14s 9.54s 462M 52.3K
PM vCtx iCtx netRX netTX cache node_mod lock
bun 10.4K 9.8K 1014M 5M 1.76G 1.76G 1M
utoo-next 140.0K 52.7K 990M 4M 1.70G 1.70G 2M
utoo-npm 135.0K 62.3K 990M 4M 1.70G 1.70G 2M
utoo 79.2K 44.5K 990M 4M 1.70G 1.70G 2M

p4_warm_link

PM wall ±σ user sys RSS pgMinor
bun 3.36s 0.07s 0.20s 2.34s 135M 33.4K
utoo-next 2.53s 0.15s 0.52s 3.77s 79M 18.1K
utoo-npm 2.34s 0.05s 0.48s 3.77s 78M 17.9K
utoo 2.17s 0.05s 0.34s 3.32s 50M 11.1K
PM vCtx iCtx netRX netTX cache node_mod lock
bun 547 24 5M 21K 1.91G 1.73G 1M
utoo-next 41.8K 18.7K 7K 10K 1.70G 1.70G 2M
utoo-npm 40.4K 17.3K 6K 5K 1.70G 1.70G 2M
utoo 8.1K 4.5K 7K 9K 1.71G 1.70G 2M

npmmirror.com: no output captured.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-Pkg Manager Area: Package Manager benchmark Run pm-bench on PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant