Skip to content

mut and par-each optimizations for performance#18566

Merged
fdncred merged 12 commits into
nushell:mainfrom
fdncred:rendering_optimization
Jul 14, 2026
Merged

mut and par-each optimizations for performance#18566
fdncred merged 12 commits into
nushell:mainfrom
fdncred:rendering_optimization

Conversation

@fdncred

@fdncred fdncred commented Jul 10, 2026

Copy link
Copy Markdown
Contributor

Description

This PR has a few changed to increase performance with par-each and mut assignments.

Two performance optimizations targeting common hot paths in Nushell: mut field assignment (cloning reduction via get_var_mut) and par-each (thread-pool reuse + captures-only stack clone).

I also added a few benchmarks.

User-facing changes (Release notes)

  • Improved performance of mut variable field assignment (e.g., $list.field = value, $table.0 = value): the variable is now mutated in-place on the evaluation stack instead of being cloned, modified, and stored back. Large list variables see the most benefit — up to 70× faster for 100k-element lists.

  • Improved performance of repeated par-each calls: custom thread pools (--threads N) are cached by thread count and reused, eliminating OS thread spawn overhead on repeated calls. The streaming (unordered) path now only clones the closure's captured variables instead of the entire stack. Combined, these reduce par-each per-call overhead by up to 33% when calling par-each many times sequentially.

Additional notes

@fdncred fdncred added notes:ready Indicates Ready for Release notes notes:other Noted in "Other changes" section labels Jul 10, 2026
@fdncred fdncred added notes:perf Performance related PRs and removed notes:other Noted in "Other changes" section labels Jul 14, 2026
@fdncred
fdncred merged commit 00065c6 into nushell:main Jul 14, 2026
14 checks passed
@fdncred
fdncred deleted the rendering_optimization branch July 14, 2026 20:02
@github-actions github-actions Bot added this to the v0.115.0 milestone Jul 14, 2026
@Tyarel8

Tyarel8 commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Using latest nightly with a command like glob "~/scoop/apps/{uv}/**/*.exe" | wrap name | par-each {} (I have remove the code inside the par-each since the bug happens even if it's empty), it hangs almost always, I haven't bisected the issue, but I'd guess this pr is the cause.

@fdncred

fdncred commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

i'll look into it, thanks.

@fdncred

fdncred commented Jul 16, 2026

Copy link
Copy Markdown
Contributor Author

@Tyarel8 can you try out #18628 ?

@Tyarel8

Tyarel8 commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

Works for me👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

notes:perf Performance related PRs notes:ready Indicates Ready for Release notes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants