Skip to content

feat: return inner array if sliced chunked array is only one chunk #2799

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
joseph-isaacs merged 3 commits into from
Mar 26, 2025
Merged

feat: return inner array if sliced chunked array is only one chunk #2799

joseph-isaacs merged 3 commits into from
Mar 26, 2025

Conversation

@joseph-isaacs
Copy link
Contributor

@joseph-isaacs joseph-isaacs commented Mar 26, 2025 •
edited
Loading

Seems unnecessary to allocate a new array?

@joseph-isaacs joseph-isaacs marked this pull request as ready for review March 26, 2025 14:20
@joseph-isaacs joseph-isaacs enabled auto-merge (squash) March 26, 2025 14:22
@joseph-isaacs joseph-isaacs changed the title feat: return inner array is slice chunked array how only one chunk feat: return inner array if sliced chunked array is only one chunk Mar 26, 2025
@joseph-isaacs joseph-isaacs added benchmark Run benchmarks on this branch benchmark-sql labels Mar 26, 2025
@github-actions github-actions bot removed benchmark-sql benchmark Run benchmarks on this branch labels Mar 26, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Mar 26, 2025

Benchmarks: random_access

Table of Results
name PR 8685885 base 5ae6f75 ratio (PR/base) unit
random-access/vortex-tokio-local-disk 2383674 2412615 0.988004 ns
random-access/parquet-tokio-local-disk 244169737 242121058 1.00846 ns

@github-actions
Copy link
Contributor

github-actions bot commented Mar 26, 2025 •
edited
Loading

Benchmarks: TPC-H on NVME

Table of Results
name PR 8685885 base 5ae6f75 ratio (PR/base) unit
tpch_q01/arrow 42450932 49281236 0.861402 ns
tpch_q02/arrow 44836100 49284429 0.909742 ns
tpch_q03/arrow 28738001 30694451 0.93626 ns
tpch_q04/arrow 21002603 22824394 0.920182 ns
tpch_q05/arrow 43297322 48485505 0.892995 ns
tpch_q06/arrow 10165391 11780720 0.862884 ns
tpch_q07/arrow 70007570 80245122 0.872422 ns
tpch_q08/arrow 52838420 57407049 0.920417 ns
tpch_q09/arrow 65580088 73546162 0.891686 ns
tpch_q10/arrow 43309951 48397412 0.894882 ns
tpch_q11/arrow 20580579 23856572 0.86268 ns
tpch_q12/arrow 28000095 31014389 0.90281 ns
tpch_q13/arrow 15077495 17617952 0.855803 ns
tpch_q14/arrow 13713923 13996662 0.9798 ns
tpch_q15/arrow 25023397 29450891 0.849665 ns
tpch_q16/arrow 19632005 22668394 0.866052 ns
tpch_q17/arrow 58811405 63730513 0.922814 ns
tpch_q18/arrow 92259632 111225416 0.829483 ns
tpch_q19/arrow 25717044 30042868 0.856012 ns
tpch_q20/arrow 30446733 34729405 0.876685 ns
tpch_q21/arrow 105987914 121308711 0.873704 ns
tpch_q22/arrow 13687404 16191367 0.845352 ns
tpch_q01/parquet 113738069 131756746 0.863243 ns
tpch_q02/parquet 107177420 122487155 0.875009 ns
tpch_q03/parquet 102276653 116904330 0.874875 ns
tpch_q04/parquet 59411606 64997222 0.914064 ns
tpch_q05/parquet 112267492 125864350 0.891972 ns
tpch_q06/parquet 26269117 28236845 0.930313 ns
tpch_q07/parquet 127460526 138427596 0.920774 ns
tpch_q08/parquet 154847914 173459331 0.892704 ns
tpch_q09/parquet 203049553 222512420 0.912531 ns
tpch_q10/parquet 125069182 143969964 0.868717 ns
tpch_q11/parquet 48467271 56400638 0.859339 ns
tpch_q12/parquet 91218383 100939475 0.903694 ns
tpch_q13/parquet 146938980 163693628 0.897646 ns
tpch_q14/parquet 54754582 48612109 1.12636 ns
tpch_q15/parquet 67250393 70746378 0.950584 ns
tpch_q16/parquet 49321823 55567564 0.887601 ns
tpch_q17/parquet 127381328 148746955 0.856363 ns
tpch_q18/parquet 185107751 204800804 0.903843 ns
tpch_q19/parquet 72237267 82889944 0.871484 ns
tpch_q20/parquet 93007067 104822693 0.88728 ns
tpch_q21/parquet 174511526 196462250 0.88827 ns
tpch_q22/parquet 46720650 54037504 0.864597 ns
tpch_q01/vortex-file-compressed 34636682 41078445 0.843184 ns
tpch_q02/vortex-file-compressed 51273222 58201886 0.880955 ns
tpch_q03/vortex-file-compressed 27969197 32039306 0.872965 ns
tpch_q04/vortex-file-compressed 17737388 19054681 0.930868 ns
tpch_q05/vortex-file-compressed 41493555 49283969 0.841928 ns
tpch_q06/vortex-file-compressed 9511876 14407746 0.660192 ns
tpch_q07/vortex-file-compressed 64323363 75905931 0.847409 ns
tpch_q08/vortex-file-compressed 49529549 57606757 0.859787 ns
tpch_q09/vortex-file-compressed 65441686 74428003 0.879262 ns
tpch_q10/vortex-file-compressed 47924774 55922953 0.856979 ns
tpch_q11/vortex-file-compressed 23679051 26349794 0.898643 ns
tpch_q12/vortex-file-compressed 23205295 24930127 0.930813 ns
tpch_q13/vortex-file-compressed 22684905 25966347 0.873627 ns
tpch_q14/vortex-file-compressed 11385535 13761559 0.827343 ns
tpch_q15/vortex-file-compressed 27572037 31351653 0.879444 ns
tpch_q16/vortex-file-compressed 25388590 30083940 0.843925 ns
tpch_q17/vortex-file-compressed 55360408 63317384 0.874332 ns
tpch_q18/vortex-file-compressed 82962221 98286344 0.844087 ns
tpch_q19/vortex-file-compressed 27902932 33530438 0.832167 ns
tpch_q20/vortex-file-compressed 34972228 39787681 0.878971 ns
tpch_q21/vortex-file-compressed 82094897 93799079 0.875221 ns
tpch_q22/vortex-file-compressed 26561006 30299580 0.876613 ns

@github-actions
Copy link
Contributor

github-actions bot commented Mar 26, 2025 •
edited
Loading

Benchmarks: TPC-H on S3

Table of Results
name PR 8685885 base 5ae6f75 ratio (PR/base) unit
tpch_q01/parquet 263312856 283786449 0.927856 ns
tpch_q02/parquet 678490681 728683628 0.931118 ns
tpch_q03/parquet 428988330 444808755 0.964433 ns
tpch_q04/parquet 232315330 235363580 0.987049 ns
tpch_q05/parquet 571402498 591327257 0.966305 ns
tpch_q06/parquet 176892039 184047904 0.96112 ns
tpch_q07/parquet 645835409 660986368 0.977078 ns
tpch_q08/parquet 776263841 809701394 0.958704 ns
tpch_q09/parquet 772500103 734811178 1.05129 ns
tpch_q10/parquet 534104676 554672197 0.96292 ns
tpch_q11/parquet 274600736 292837790 0.937723 ns
tpch_q12/parquet 281669904 299345947 0.940951 ns
tpch_q13/parquet 396117900 429621260 0.922017 ns
tpch_q14/parquet 253441985 264238632 0.959141 ns
tpch_q15/parquet 458693808 477950127 0.959711 ns
tpch_q16/parquet 277338572 289176943 0.959062 ns
tpch_q17/parquet 404210408 429381610 0.941378 ns
tpch_q18/parquet 537912122 581192994 0.925531 ns
tpch_q19/parquet 275262110 309948157 0.888091 ns
tpch_q20/parquet 518187957 533243918 0.971765 ns
tpch_q21/parquet 623921936 657664753 0.948693 ns
tpch_q22/parquet 263077897 295702845 0.88967 ns
tpch_q01/vortex-file-compressed 121988215 132385460 0.921462 ns
tpch_q02/vortex-file-compressed 315833280 358018762 0.88217 ns
tpch_q03/vortex-file-compressed 269621743 274476474 0.982313 ns
tpch_q04/vortex-file-compressed 199985447 203197871 0.984191 ns
tpch_q05/vortex-file-compressed 294213175 288009119 1.02154 ns
tpch_q06/vortex-file-compressed 87897929 90768398 0.968376 ns
tpch_q07/vortex-file-compressed 301678188 336185118 0.897357 ns
tpch_q08/vortex-file-compressed 385601448 410887898 0.938459 ns
tpch_q09/vortex-file-compressed 410553426 432810394 0.948576 ns
tpch_q10/vortex-file-compressed 385252014 403342708 0.955148 ns
tpch_q11/vortex-file-compressed 133435439 139778439 0.954621 ns
tpch_q12/vortex-file-compressed 157691895 159977154 0.985715 ns
tpch_q13/vortex-file-compressed 216475681 219475890 0.98633 ns
tpch_q14/vortex-file-compressed 178900860 174649151 1.02434 ns
tpch_q15/vortex-file-compressed 318361286 318379834 0.999942 ns
tpch_q16/vortex-file-compressed 117660772 118411916 0.993657 ns
tpch_q17/vortex-file-compressed 311775832 329236448 0.946966 ns
tpch_q18/vortex-file-compressed 398556228 428250450 0.930662 ns
tpch_q19/vortex-file-compressed 118839315 124896948 0.951499 ns
tpch_q20/vortex-file-compressed 244708466 259369049 0.943476 ns
tpch_q21/vortex-file-compressed 496100726 518835453 0.956181 ns
tpch_q22/vortex-file-compressed 136604169 152527454 0.895604 ns

@github-actions
Copy link
Contributor

github-actions bot commented Mar 26, 2025 •
edited
Loading

Benchmarks: Clickbench on NVME

Table of Results
name PR 8685885 base 5ae6f75 ratio (PR/base) unit
clickbench_q00/parquet 2546501 2313997 1.10048 ns
clickbench_q01/parquet 36532051 31608882 1.15575 ns
clickbench_q02/parquet 65824156 62119485 1.05964 ns
clickbench_q03/parquet 53300336 52059552 1.02383 ns
clickbench_q04/parquet 317723186 321637936 0.987829 ns
clickbench_q05/parquet 305670497 299947609 1.01908 ns
clickbench_q06/parquet 2356926 2210268 1.06635 ns
clickbench_q07/parquet 36466660 32813263 1.11134 ns
clickbench_q08/parquet 411784974 375908534 1.09544 ns
clickbench_q09/parquet 609774176 550209861 1.10826 ns
clickbench_q10/parquet 121421037 114537650 1.0601 ns
clickbench_q11/parquet 144338462 140972685 1.02388 ns
clickbench_q12/parquet 314082245 305941960 1.02661 ns
clickbench_q13/parquet 512112689 477695584 1.07205 ns
clickbench_q14/parquet 333832155 307554696 1.08544 ns
clickbench_q15/parquet 349345111 337782419 1.03423 ns
clickbench_q16/parquet 819228284 761975084 1.07514 ns
clickbench_q17/parquet 756441675 647762233 1.16778 ns
clickbench_q18/parquet 1534689549 1445629872 1.06161 ns
clickbench_q19/parquet 44969014 42763943 1.05156 ns
clickbench_q20/parquet 587085311 566916785 1.03558 ns
clickbench_q21/parquet 627144942 590339969 1.06235 ns
clickbench_q22/parquet 970991938 930086245 1.04398 ns
clickbench_q23/parquet 3900743002 3732414077 1.0451 ns
clickbench_q24/parquet 206482511 187565013 1.10086 ns
clickbench_q25/parquet 175599115 164560038 1.06708 ns
clickbench_q26/parquet 232597484 213767587 1.08809 ns
clickbench_q27/parquet 792671865 728972127 1.08738 ns
clickbench_q28/parquet 4421375272 4296392288 1.02909 ns
clickbench_q29/parquet 252023077 236298156 1.06655 ns
clickbench_q30/parquet 324299900 313436665 1.03466 ns
clickbench_q31/parquet 370045367 362938140 1.01958 ns
clickbench_q32/parquet 2005914407 1703738528 1.17736 ns
clickbench_q33/parquet 1553502614 1413507351 1.09904 ns
clickbench_q34/parquet 1506962610 1396698735 1.07895 ns
clickbench_q35/parquet 568561966 482690703 1.1779 ns
clickbench_q36/parquet 155398148 147444043 1.05395 ns
clickbench_q37/parquet 72787253 66198020 1.09954 ns
clickbench_q38/parquet 98775961 92506008 1.06778 ns
clickbench_q39/parquet 291679314 275161989 1.06003 ns
clickbench_q40/parquet 49111040 44370996 1.10683 ns
clickbench_q41/parquet 45762105 42508699 1.07654 ns
clickbench_q42/parquet 55466180 49236931 1.12652 ns
clickbench_q00/vortex-file-compressed 5477916 4725647 1.15919 ns
clickbench_q01/vortex-file-compressed 19697936 19134149 1.02946 ns
clickbench_q02/vortex-file-compressed 38734923 34180964 1.13323 ns
clickbench_q03/vortex-file-compressed 48741608 43075919 1.13153 ns
clickbench_q04/vortex-file-compressed 430843068 367293630 1.17302 ns
clickbench_q05/vortex-file-compressed 383531207 348428632 1.10075 ns
clickbench_q06/vortex-file-compressed 5344628 4268899 1.25199 ns
clickbench_q07/vortex-file-compressed 29691466 23572035 1.25961 ns
clickbench_q08/vortex-file-compressed 459522452 417437236 1.10082 ns
clickbench_q09/vortex-file-compressed 531359633 504928279 1.05235 ns
clickbench_q10/vortex-file-compressed 71878672 71691871 1.00261 ns
clickbench_q11/vortex-file-compressed 76615612 78199297 0.979748 ns
clickbench_q12/vortex-file-compressed 276835306 270176508 1.02465 ns
clickbench_q13/vortex-file-compressed 365077083 365463234 0.998943 ns
clickbench_q14/vortex-file-compressed 279707572 263613368 1.06105 ns
clickbench_q15/vortex-file-compressed 441696619 426265320 1.0362 ns
clickbench_q16/vortex-file-compressed 860437253 791493078 1.08711 ns
clickbench_q17/vortex-file-compressed 840088896 756282491 1.11081 ns
clickbench_q18/vortex-file-compressed 1326956261 1231208286 1.07777 ns
clickbench_q19/vortex-file-compressed 28781368 28793204 0.999589 ns
clickbench_q20/vortex-file-compressed 274981323 251848794 1.09185 ns
clickbench_q21/vortex-file-compressed 307537235 294594359 1.04393 ns
clickbench_q22/vortex-file-compressed 512562570 505685987 1.0136 ns
clickbench_q23/vortex-file-compressed 950868278 884325753 1.07525 ns
clickbench_q24/vortex-file-compressed 88684689 88406875 1.00314 ns
clickbench_q25/vortex-file-compressed 104594634 97436699 1.07346 ns
clickbench_q26/vortex-file-compressed 127049457 120538894 1.05401 ns
clickbench_q27/vortex-file-compressed 527534313 558383705 0.944752 ns
clickbench_q28/vortex-file-compressed 5275878463 5254025942 1.00416 ns
clickbench_q29/vortex-file-compressed 238455267 254003955 0.938786 ns
clickbench_q30/vortex-file-compressed 244218393 236762414 1.03149 ns
clickbench_q31/vortex-file-compressed 238835899 235367074 1.01474 ns
clickbench_q32/vortex-file-compressed 1396963868 1260391184 1.10836 ns
clickbench_q33/vortex-file-compressed 1318584262 1208582223 1.09102 ns
clickbench_q34/vortex-file-compressed 1267124627 1194292487 1.06098 ns
clickbench_q35/vortex-file-compressed 658319217 628192728 1.04796 ns
clickbench_q36/vortex-file-compressed 111349918 110112393 1.01124 ns
clickbench_q37/vortex-file-compressed 75501332 71794962 1.05162 ns
clickbench_q38/vortex-file-compressed 85332103 87940176 0.970343 ns
clickbench_q39/vortex-file-compressed 184488767 180555179 1.02179 ns
clickbench_q40/vortex-file-compressed 46019036 42808730 1.07499 ns
clickbench_q41/vortex-file-compressed 38873811 38220895 1.01708 ns
clickbench_q42/vortex-file-compressed 38777569 33955499 1.14201 ns

@joseph-isaacs joseph-isaacs merged commit 44da00f into develop Mar 26, 2025
41 of 42 checks passed
@joseph-isaacs joseph-isaacs deleted the ji/slice-chunk-turn-inner-if-size-is-one branch March 26, 2025 15:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@robert3005 robert3005 robert3005 approved these changes

@gatesn gatesn Awaiting requested review from gatesn

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@joseph-isaacs @robert3005