Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

improve rolling_{min/max/sum/mean} prerformance ~3.4x #3444

Merged
merged 4 commits into from
May 20, 2022

Conversation

ritchie46
Copy link
Member

Also improvements to rolling_min/ rolling_max kernels. They can maintain a rolling min/max state. No need to to do an O(n) operation at every new window.

We can do the same with sum. Will add that to this PR.

bench_1                 time:   [543.66 us 544.71 us 546.00 us]
                        change: [-70.677% -70.627% -70.585%] (p = 0.00 < 0.05)
                        Performance has improved.

@github-actions github-actions bot added the rust Related to Rust Polars label May 20, 2022
@codecov-commenter
Copy link

codecov-commenter commented May 20, 2022

Codecov Report

Merging #3444 (d246c45) into master (24ddb90) will increase coverage by 1.42%.
The diff coverage is 46.46%.

@@            Coverage Diff             @@
##           master    #3444      +/-   ##
==========================================
+ Coverage   62.08%   63.51%   +1.42%     
==========================================
  Files         396      398       +2     
  Lines       68188    66880    -1308     
==========================================
+ Hits        42337    42481     +144     
+ Misses      25851    24399    -1452     
Impacted Files Coverage Δ
polars/polars-arrow/src/kernels/rolling/nulls.rs 0.00% <0.00%> (ø)
...ars-arrow/src/kernels/rolling/quantile_no_nulls.rs 44.48% <0.00%> (ø)
polars/polars-arrow/src/lib.rs 0.00% <0.00%> (ø)
...-arrow/src/kernels/rolling/sum_min_max_no_nulls.rs 39.32% <39.32%> (ø)
.../polars-arrow/src/kernels/rolling/mean_no_nulls.rs 50.94% <50.94%> (ø)
...olars-core/src/chunked_array/ops/rolling_window.rs 84.24% <75.00%> (+47.90%) ⬆️
polars/polars-arrow/src/kernels/rolling/mod.rs 84.41% <91.66%> (+3.28%) ⬆️
...olars/polars-arrow/src/kernels/rolling/no_nulls.rs 80.69% <100.00%> (+20.64%) ⬆️
polars/polars-arrow/src/kernels/rolling/window.rs 26.14% <100.00%> (-4.77%) ⬇️
... and 44 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 24ddb90...d246c45. Read the comment docs.

@ritchie46 ritchie46 changed the title improve rolling_min/max prerformance ~3.4x improve rolling_{min/max/sum/mean} prerformance ~3.4x` May 20, 2022
@ritchie46 ritchie46 changed the title improve rolling_{min/max/sum/mean} prerformance ~3.4x` improve rolling_{min/max/sum/mean} prerformance ~3.4x May 20, 2022
@ritchie46 ritchie46 merged commit 24cc2ef into master May 20, 2022
@ritchie46 ritchie46 deleted the improve_rolling_min_max branch May 20, 2022 12:19
moritzwilksch pushed a commit to moritzwilksch/polars that referenced this pull request May 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
rust Related to Rust Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants