Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Various performance improvements #83

Merged
merged 24 commits into from
Oct 13, 2023
Merged

Various performance improvements #83

merged 24 commits into from
Oct 13, 2023

Conversation

sterrettm2
Copy link
Contributor

This merge request adds a bunch of performance enhancements. It changes:

The small sorting algorithm to be more efficient (b52e889)

Changes how the partitioning is done in a few small ways (16e35b0)

Changes how the array is shortened to be a multiple of the correct length (d617059)

Changes how pivots are selected for larger arrays (91928b6)

Increases the amount of prefetching done (d4ecb7e)

And some smaller changes, like some small changes to the parameters used.

Note that this was tested on a 7900x, so the 16-bit performance results should be ignored.

Benchmark                                                          Time             CPU      Time Old      Time New       CPU Old       CPU New
-----------------------------------------------------------------------------------------------------------------------------------------------
[simdsort vs. simdsort]/smallrandom_128/uint64_t                -0.1037         -0.1039          1184          1061          1186          1063
[simdsort vs. simdsort]/smallrandom_256/uint64_t                -0.1911         -0.1921          1695          1372          1701          1374
[simdsort vs. simdsort]/smallrandom_512/uint64_t                -0.2495         -0.2505          3212          2411          3220          2413
[simdsort vs. simdsort]/smallrandom_1k/uint64_t                 -0.2270         -0.2277          6093          4709          6102          4713
[simdsort vs. simdsort]/random_5k/uint64_t                      -0.3027         -0.3027         32216         22466         32229         22473
[simdsort vs. simdsort]/random_100k/uint64_t                    -0.3064         -0.3065        835972        579824        835950        579741
[simdsort vs. simdsort]/random_1m/uint64_t                      -0.2769         -0.2769      10400812       7520943      10399563       7520057
[simdsort vs. simdsort]/random_10m/uint64_t                     -0.2661         -0.2662     132986347      97592032     132971852      97577800
[simdsort vs. simdsort]/sorted_10k/uint64_t                     -0.3182         -0.3184         67646         46121         67678         46126
[simdsort vs. simdsort]/constant_10k/uint64_t                   -0.0176         -0.0180          3661          3596          3666          3600
[simdsort vs. simdsort]/reverse_10k/uint64_t                    -0.3199         -0.3201         67248         45736         67271         45740
[simdsort vs. simdsort]/smallrandom_128/int64_t                 -0.1059         -0.1056          1184          1059          1186          1061
[simdsort vs. simdsort]/smallrandom_256/int64_t                 -0.1944         -0.1949          1701          1370          1706          1373
[simdsort vs. simdsort]/smallrandom_512/int64_t                 -0.2506         -0.2517          3215          2409          3224          2413
[simdsort vs. simdsort]/smallrandom_1k/int64_t                  -0.2320         -0.2327          6117          4697          6127          4701
[simdsort vs. simdsort]/random_5k/int64_t                       -0.3099         -0.3099         32531         22449         32543         22459
[simdsort vs. simdsort]/random_100k/int64_t                     -0.3079         -0.3079        838363        580244        838384        580219
[simdsort vs. simdsort]/random_1m/int64_t                       -0.2812         -0.2813      10451223       7511916      10450858       7511274
[simdsort vs. simdsort]/random_10m/int64_t                      -0.2680         -0.2681     133372941      97623431     133362938      97611054
[simdsort vs. simdsort]/sorted_10k/int64_t                      -0.3243         -0.3244         68105         46018         68128         46026
[simdsort vs. simdsort]/constant_10k/int64_t                    -0.0197         -0.0198          3660          3588          3665          3593
[simdsort vs. simdsort]/reverse_10k/int64_t                     -0.3267         -0.3267         67893         45713         67904         45717
[simdsort vs. simdsort]/smallrandom_128/uint32_t                -0.0384         -0.0394           969           932           971           933
[simdsort vs. simdsort]/smallrandom_256/uint32_t                -0.0839         -0.0860          1157          1060          1161          1061
[simdsort vs. simdsort]/smallrandom_512/uint32_t                -0.2239         -0.2237          1785          1385          1788          1388
[simdsort vs. simdsort]/smallrandom_1k/uint32_t                 -0.1172         -0.1173          3044          2687          3048          2690
[simdsort vs. simdsort]/random_5k/uint32_t                      -0.2075         -0.2072         14428         11435         14417         11429
[simdsort vs. simdsort]/random_100k/uint32_t                    -0.4164         -0.4164        439127        256254        439082        256270
[simdsort vs. simdsort]/random_1m/uint32_t                      -0.3761         -0.3761       5525648       3447572       5524910       3447088
[simdsort vs. simdsort]/random_10m/uint32_t                     -0.3518         -0.3518      69094353      44789578      69088252      44781016
[simdsort vs. simdsort]/sorted_10k/uint32_t                     -0.3050         -0.3049         30179         20974         30188         20983
[simdsort vs. simdsort]/constant_10k/uint32_t                   -0.2282         -0.2274          2704          2087          2699          2085
[simdsort vs. simdsort]/reverse_10k/uint32_t                    -0.2915         -0.2913         30479         21596         30485         21604
[simdsort vs. simdsort]/smallrandom_128/int32_t                 -0.0383         -0.0388           970           933           972           935
[simdsort vs. simdsort]/smallrandom_256/int32_t                 -0.0842         -0.0865          1156          1059          1160          1060
[simdsort vs. simdsort]/smallrandom_512/int32_t                 -0.2247         -0.2247          1786          1384          1788          1386
[simdsort vs. simdsort]/smallrandom_1k/int32_t                  -0.1180         -0.1180          3047          2688          3049          2689
[simdsort vs. simdsort]/random_5k/int32_t                       -0.2087         -0.2083         14465         11447         14451         11441
[simdsort vs. simdsort]/random_100k/int32_t                     -0.4412         -0.4411        459693        256898        459702        256913
[simdsort vs. simdsort]/random_1m/int32_t                       -0.4040         -0.4041       5801112       3457252       5800462       3456667
[simdsort vs. simdsort]/random_10m/int32_t                      -0.3796         -0.3796      72248302      44824979      72240153      44817758
[simdsort vs. simdsort]/sorted_10k/int32_t                      -0.3083         -0.3082         30338         20986         30352         20997
[simdsort vs. simdsort]/constant_10k/int32_t                    -0.2198         -0.2194          2679          2090          2675          2088
[simdsort vs. simdsort]/reverse_10k/int32_t                     -0.2931         -0.2931         30600         21631         30613         21641
[simdsort vs. simdsort]/smallrandom_128/uint16_t                +0.0720         +0.0700          1433          1536          1437          1538
[simdsort vs. simdsort]/smallrandom_256/uint16_t                +0.0807         +0.0797          2175          2351          2179          2353
[simdsort vs. simdsort]/smallrandom_512/uint16_t                +0.0464         +0.0465          4122          4313          4124          4316
[simdsort vs. simdsort]/smallrandom_1k/uint16_t                 -0.1288         -0.1290         20403         17775         20427         17793
[simdsort vs. simdsort]/random_5k/uint16_t                      +0.0238         +0.0237        209875        214861        209886        214867
[simdsort vs. simdsort]/random_100k/uint16_t                    +0.0160         +0.0161       5740371       5832409       5739953       5832085
[simdsort vs. simdsort]/random_1m/uint16_t                      +0.0096         +0.0096      61461087      62050000      61456455      62044271
[simdsort vs. simdsort]/random_10m/uint16_t                     +0.0132         +0.0131     599895372     607800011     599833228     607709017
[simdsort vs. simdsort]/sorted_10k/uint16_t                     +0.0581         +0.0581         78611         83178         78612         83179
[simdsort vs. simdsort]/constant_10k/uint16_t                   +0.1349         +0.1348         62136         70519         62140         70518
[simdsort vs. simdsort]/reverse_10k/uint16_t                    +0.0853         +0.0854         63636         69062         63637         69069
[simdsort vs. simdsort]/smallrandom_128/int16_t                 +0.0539         +0.0542          1431          1508          1434          1511
[simdsort vs. simdsort]/smallrandom_256/int16_t                 +0.0796         +0.0796          2142          2312          2144          2314
[simdsort vs. simdsort]/smallrandom_512/int16_t                 +0.0484         +0.0482          3976          4168          3978          4170
[simdsort vs. simdsort]/smallrandom_1k/int16_t                  +0.0487         +0.0485         16956         17782         16980         17803
[simdsort vs. simdsort]/random_5k/int16_t                       +0.0177         +0.0176        210658        214383        210673        214391
[simdsort vs. simdsort]/random_100k/int16_t                     +0.0118         +0.0118       5753799       5821933       5753399       5821576
[simdsort vs. simdsort]/random_1m/int16_t                       +0.0090         +0.0091      61501369      62057405      61494820      62052658
[simdsort vs. simdsort]/random_10m/int16_t                      +0.0143         +0.0142     599188625     607748215     599111168     607641829
[simdsort vs. simdsort]/sorted_10k/int16_t                      +0.0372         +0.0373         78483         81407         78488         81413
[simdsort vs. simdsort]/constant_10k/int16_t                    +0.1352         +0.1352         61970         70349         61973         70349
[simdsort vs. simdsort]/reverse_10k/int16_t                     +0.0172         +0.0173         63997         65096         63993         65103
[simdsort vs. simdsort]/smallrandom_128/float                   -0.0822         -0.0831          1033           948          1035           949
[simdsort vs. simdsort]/smallrandom_256/float                   -0.1575         -0.1582          1250          1053          1252          1054
[simdsort vs. simdsort]/smallrandom_512/float                   -0.4735         -0.4728          2604          1371          2606          1374
[simdsort vs. simdsort]/smallrandom_1k/float                    -0.4098         -0.4083          3981          2350          3985          2358
[simdsort vs. simdsort]/random_5k/float                         -0.3888         -0.3885         17633         10777         17621         10776
[simdsort vs. simdsort]/random_100k/float                       -0.4964         -0.4963        507155        255406        507144        255430
[simdsort vs. simdsort]/random_1m/float                         -0.4476         -0.4477       6289875       3474452       6289004       3473610
[simdsort vs. simdsort]/random_10m/float                        -0.4088         -0.4088      78503262      46409045      78490290      46404562
[simdsort vs. simdsort]/sorted_10k/float                        -0.4333         -0.4331         36342         20596         36348         20605
[simdsort vs. simdsort]/constant_10k/float                      -0.1758         -0.1757          3782          3117          3787          3122
[simdsort vs. simdsort]/reverse_10k/float                       -0.4283         -0.4281         36931         21112         36935         21122
[simdsort vs. simdsort]/smallrandom_128/double                  -0.1016         -0.1014          1106           994          1108           995
[simdsort vs. simdsort]/smallrandom_256/double                  -0.1972         -0.1980          1534          1232          1539          1234
[simdsort vs. simdsort]/smallrandom_512/double                  -0.3095         -0.3089          2845          1965          2854          1972
[simdsort vs. simdsort]/smallrandom_1k/double                   -0.3407         -0.3400          5216          3439          5225          3448
[simdsort vs. simdsort]/random_5k/double                        -0.3107         -0.3108         26692         18399         26710         18408
[simdsort vs. simdsort]/random_100k/double                      -0.3524         -0.3524        750017        485719        750009        485690
[simdsort vs. simdsort]/random_1m/double                        -0.3168         -0.3168       9861228       6737332       9860038       6736280
[simdsort vs. simdsort]/random_10m/double                       -0.2849         -0.2849     127938404      91487464     127924136      91474156
[simdsort vs. simdsort]/sorted_10k/double                       -0.3839         -0.3839         57426         35382         57444         35390
[simdsort vs. simdsort]/constant_10k/double                     -0.0608         -0.0608          5378          5051          5383          5056
[simdsort vs. simdsort]/reverse_10k/double                      -0.3850         -0.3847         57398         35302         57408         35321
[simdsort vs. simdsort]/smallrandom_128/_Float16                -0.0244         -0.0244         21106         20591         21118         20603
[simdsort vs. simdsort]/smallrandom_256/_Float16                -0.0239         -0.0238         48497         47339         48505         47350
[simdsort vs. simdsort]/smallrandom_512/_Float16                -0.0227         -0.0227        106185        103778        106193        103787
[simdsort vs. simdsort]/smallrandom_1k/_Float16                 -0.0211         -0.0211        231702        226813        231702        226816
[simdsort vs. simdsort]/random_5k/_Float16                      -0.0193         -0.0193       1423840       1396377       1423760       1396331
[simdsort vs. simdsort]/random_100k/_Float16                    -0.0247         -0.0247      35021593      34155509      35019635      34153112
[simdsort vs. simdsort]/random_1m/_Float16                      -0.0220         -0.0221     381941308     373537153     381935917     373507960
[simdsort vs. simdsort]/random_10m/_Float16                     -0.0211         -0.0211    4361013075    4269155840    4360678081    4268780980
[simdsort vs. simdsort]/sorted_10k/_Float16                     -0.0391         -0.0390       2158440       2074104       2158259       2074038
[simdsort vs. simdsort]/constant_10k/_Float16                   -0.0063         -0.0063       1961500       1949136       1961389       1949099
[simdsort vs. simdsort]/reverse_10k/_Float16                    -0.0338         -0.0338       2053028       1983597       2052913       1983522
OVERALL_GEOMEAN                                                 -0.1883         -0.1883             0             0             0             0

@r-devulap
Copy link
Contributor

r-devulap commented Oct 4, 2023

On a TGL: Improves simdsort perf on 16-bit data by up to 1.25x

Comparing simdsort.*int16_t (from /x86-simd-sort/.bench/main/builddir/benchexe) to simdsort.*int16_t (from /x86-simd-sort/.bench/perf/builddir/benchexe)            [10/21285]
Benchmark                                                   Time             CPU      Time Old      Time New       CPU Old       CPU New
----------------------------------------------------------------------------------------------------------------------------------------
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.0643         -0.0615           417           390           419           393
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.1125         -0.1102           566           502           566           504
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.1769         -0.1768           917           755           919           757
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.1943         -0.1898          2193          1767          2188          1772
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.1488         -0.1492         12841         10930         12778         10872
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.1953         -0.1952        335721        270139        333690        268548
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.1135         -0.1098       4646583       4119337       4619227       4111873
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.1093         -0.1098      54836124      48840305      54474939      48494953
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.2720         -0.2720         29675         21603         29500         21477
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.0025         -0.0035          1899          1894          1892          1885
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.2795         -0.2795         29336         21136         29304         21113
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.0632         -0.0631           418           392           420           393
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.1114         -0.1143           566           503           568           503
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.1728         -0.1719           916           758           916           758
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.1977         -0.1949          2185          1753          2180          1755
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.1539         -0.1539         12816         10843         12808         10836
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.0694         -0.0693        329538        306657        329063        306253
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.1132         -0.1131       4634763       4110293       4626244       4102836
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.1155         -0.1153      54671881      48355572      54555529      48265035
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.2800         -0.2767         29662         21356         29489         21330
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.0361         -0.0397          1917          1847          1906          1830
[simdsort.*int16_t vs. simdsort.*int16_t]                -0.2835         -0.2836         29474         21118         29296         20989
[simdsort.*int16_t vs. simdsort.*int16_t]_pvalue          0.4887          0.4887      U Test, Repetitions: 22 vs 22
OVERALL_GEOMEAN                                          -0.1522         -0.1516             0             0             0             0

src/avx512-common-qsort.h Outdated Show resolved Hide resolved
src/avx512-common-qsort.h Outdated Show resolved Hide resolved
src/avx512-common-qsort.h Outdated Show resolved Hide resolved
src/avx512-common-qsort.h Outdated Show resolved Hide resolved
}
}
else {
X86_SIMD_SORT_UNROLL_LOOP(8)
for (int ii = 0; ii < num_unroll; ++ii) {
curr_vec[ii] = vtype::loadu(arr + left + ii * vtype::numlanes);
_mm_prefetch(arr + left + ii * vtype::numlanes
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ugh, the formatting from clang_format is ugly here. Might need to tweak some parameters in the _clang-format file.

src/avx512-common-qsort.h Outdated Show resolved Hide resolved
src/avx512-common-qsort.h Outdated Show resolved Hide resolved
src/xss-network-qsort.hpp Outdated Show resolved Hide resolved
src/xss-network-qsort.hpp Outdated Show resolved Hide resolved
@sterrettm2 sterrettm2 force-pushed the perfwork branch 2 times, most recently from 24701aa to dfa65db Compare October 10, 2023 17:42
src/xss-network-qsort.hpp Outdated Show resolved Hide resolved
src/xss-network-qsort.hpp Outdated Show resolved Hide resolved
src/xss-network-qsort.hpp Outdated Show resolved Hide resolved
src/xss-network-qsort.hpp Outdated Show resolved Hide resolved
src/xss-network-qsort.hpp Outdated Show resolved Hide resolved
src/xss-network-qsort.hpp Outdated Show resolved Hide resolved
src/xss-network-qsort.hpp Outdated Show resolved Hide resolved
src/avx512-common-qsort.h Outdated Show resolved Hide resolved
@r-devulap
Copy link
Contributor

Perf changes summary:

Qsort: On SKX, up-to 1.9x speed up for 32-bit and up-to 1.5x speed up for 64-bit data.

Benchmark                                                                    Time             CPU      Time Old      Time New       CPU Old       CPU New
---------------------------------------------------------------------------------------------------------------------------------------------------------
[simdsort.*random_100k vs. simdsort.*random_100k]/uint64_t                -0.2996         -0.2996        845220        591977        845176        591979
[simdsort.*random_100k vs. simdsort.*random_100k]/int64_t                 -0.3008         -0.3009        847856        592799        847930        592747
[simdsort.*random_100k vs. simdsort.*random_100k]/uint32_t                -0.3968         -0.3968        444567        268156        444563        268157
[simdsort.*random_100k vs. simdsort.*random_100k]/int32_t                 -0.3976         -0.3975        443295        267060        443310        267074
[simdsort.*random_100k vs. simdsort.*random_100k]/uint16_t                +0.1004         +0.1004       5808004       6391195       5807631       6390779
[simdsort.*random_100k vs. simdsort.*random_100k]/int16_t                 -0.0086         -0.0087       5788210       5738463       5788053       5737963
[simdsort.*random_100k vs. simdsort.*random_100k]/float                   -0.4789         -0.4789        503448        262355        503461        262358
[simdsort.*random_100k vs. simdsort.*random_100k]/double                  -0.3354         -0.3356        747113        496496        747174        496448
OVERALL_GEOMEAN                                                           -0.2864         -0.2864             0             0             0             0

Copy link
Contributor

@r-devulap r-devulap left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Thanks for the awesome work @sterrettm2!

@r-devulap r-devulap merged commit 0fad81f into intel:main Oct 13, 2023
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants