Skip to content

Conversation

@Ted-Jiang
Copy link
Member

Which issue does this PR close?

Closes #3217 .

Rationale for this change

Before we use std::sort in update_batch of aggregate approx_percentile_cont
We can use arrow::compute::sort and merge_sorted_f64 replace merge_unsorted_f64

run bench get

aggregate_query_approx_percentile_cont_on_u64                                                                            
                        time:   [6.1686 ms 6.1978 ms 6.2367 ms]
                        change: [-8.4056% -7.5404% -6.5993%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 7 outliers among 100 measurements (7.00%)
  3 (3.00%) high mild
  4 (4.00%) high severe

aggregate_query_approx_percentile_cont_on_f32                                                                              
                        time:   [7.0710 ms 7.1209 ms 7.1847 ms]
                        change: [-7.5669% -6.6221% -5.5469%] (p = 0.00 < 0.05)
                        Performance has improved.
Found 13 outliers among 100 measurements (13.00%)
  4 (4.00%) high mild
  9 (9.00%) high severe

What changes are included in this PR?

Are there any user-facing changes?

@github-actions github-actions bot added core Core DataFusion crate physical-expr Changes to the physical-expr crates labels Aug 22, 2022
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice find @Ted-Jiang 👍

@Ted-Jiang Ted-Jiang closed this Aug 22, 2022
@Ted-Jiang Ted-Jiang reopened this Aug 22, 2022
@codecov-commenter
Copy link

Codecov Report

Merging #3219 (3c98138) into master (4838564) will increase coverage by 0.00%.
The diff coverage is 100.00%.

❗ Current head 3c98138 differs from pull request most recent head 77c9473. Consider uploading reports for the commit 77c9473 to get more accurate results

@@           Coverage Diff           @@
##           master    #3219   +/-   ##
=======================================
  Coverage   85.84%   85.84%           
=======================================
  Files         291      291           
  Lines       52909    52910    +1     
=======================================
+ Hits        45419    45420    +1     
  Misses       7490     7490           
Impacted Files Coverage Δ
...sical-expr/src/aggregate/approx_percentile_cont.rs 81.54% <100.00%> (+0.11%) ⬆️
datafusion/physical-expr/src/aggregate/tdigest.rs 91.25% <100.00%> (ø)
datafusion/core/src/physical_plan/metrics/value.rs 86.93% <0.00%> (-0.51%) ⬇️
datafusion/expr/src/logical_plan/plan.rs 78.00% <0.00%> (+0.17%) ⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

Copy link
Member

@andygrove andygrove left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

@andygrove andygrove merged commit f974e88 into apache:master Aug 22, 2022
@ursabot
Copy link

ursabot commented Aug 22, 2022

Benchmark runs are scheduled for baseline = f2313ae and contender = f974e88. f974e88 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ec2-t3-xlarge-us-east-2] ec2-t3-xlarge-us-east-2
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on test-mac-arm] test-mac-arm
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ursa-i9-9960x] ursa-i9-9960x
[Skipped ⚠️ Benchmarking of arrow-datafusion-commits is not supported on ursa-thinkcentre-m75q] ursa-thinkcentre-m75q
Buildkite builds:
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Core DataFusion crate physical-expr Changes to the physical-expr crates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Create bench for approx_percentile_cont aggregate

5 participants