Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: Improve rolling_median algorithm #12704

Merged
merged 42 commits into from
Dec 7, 2023
Merged

perf: Improve rolling_median algorithm #12704

merged 42 commits into from
Dec 7, 2023

Conversation

ritchie46
Copy link
Member

@ritchie46 ritchie46 commented Nov 27, 2023

This removes quadratic worst case to O(nlogk).

closes #12609

0.19.x

polars,median,single,1000000,100,0.085
polars,median,single,1000000,1000,0.161
polars,median,single,1000000,10000,0.798
polars,median,single,10000000,100,0.849
polars,median,single,10000000,1000,1.663
polars,median,single,10000000,10000,8.597
polars,median,single,100000000,100,9.060
polars,median,single,100000000,1000,17.968
polars,median,single,100000000,10000,94.659

PR

polars,median,single,1000000,100,0.080
polars,median,single,1000000,1000,0.094
polars,median,single,1000000,10000,0.111
polars,median,single,10000000,100,0.777
polars,median,single,10000000,1000,0.943
polars,median,single,10000000,10000,1.120
polars,median,single,100000000,100,7.969
polars,median,single,100000000,1000,9.795
polars,median,single,100000000,10000,11.813

@ritchie46 ritchie46 requested a review from orlp as a code owner November 27, 2023 06:47
@ritchie46 ritchie46 marked this pull request as draft November 27, 2023 06:48
@ritchie46 ritchie46 changed the title WIP: improve rolling_median algorithm perf: improve rolling_median algorithm Dec 6, 2023
@github-actions github-actions bot added performance Performance issues or improvements python Related to Python Polars rust Related to Rust Polars labels Dec 6, 2023
@ritchie46 ritchie46 marked this pull request as ready for review December 6, 2023 18:20
@jangorecki
Copy link

jangorecki commented Dec 6, 2023

is documentation coming in this PR later on or it will some different PR? reading rust is not yet that fluent for me. I am interested about implementation used precisely.

@ritchie46 ritchie46 merged commit 7567759 into main Dec 7, 2023
25 checks passed
@ritchie46 ritchie46 deleted the rolling_median branch December 7, 2023 08:09
@stinodego stinodego changed the title perf: improve rolling_median algorithm perf: Improve rolling_median algorithm Dec 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
performance Performance issues or improvements python Related to Python Polars rust Related to Rust Polars
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Performance of rolling_median()
2 participants