Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[stdlib] Optimize some Set operations #21300

Closed
wants to merge 11 commits into from

Conversation

lorentey
Copy link
Member

@lorentey lorentey commented Dec 13, 2018

This uses temporary bitmaps to considerably speed up some high-level Set operations.

Operation Speedup
Set.isSubset<S>(of:) 3x
Set.isStrictSubset<S>(of:) 3x
Set.isSuperset<S>(of:) 4x
Set.isStrictSuperset<S>(of:) 4x
Set.isDisjoint<S>(with:) 4x
Set.subtracting(_:) 1-4x
Set.filter(_:) 1-4x
Set.intersection<S>(_:) 4-6x
Set.intersection(_:) 1-4x

Benchmark results depend on the size of the sets and the element type. The numbers above were measured with integer keys across a range of counts from 1 to 1 million items.

@lorentey
Copy link
Member Author

@swift-ci benchmark

@lorentey
Copy link
Member Author

@swift-ci test

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - d7206d1f51ba47bdecec68d6721836d986c524c7

@swift-ci
Copy link
Contributor

Build comment file:

Performance: -O

TEST OLD NEW DELTA RATIO
Regression
SetSubtractingInt25 102 117 +14.7% 0.87x
Improvement
SetIntersectionInt100 326 75 -77.0% 4.35x
SetIntersectionBox25 272 132 -51.5% 2.06x
SetIntersectionInt50 198 106 -46.5% 1.87x
SetSubtractingInt100 151 83 -45.0% 1.82x
SetIntersectionBox0 138 85 -38.4% 1.62x
SetIntersectionInt25 135 84 -37.8% 1.61x
SetSubtractingInt50 128 104 -18.7% 1.23x
SetSubtractingBox0 149 137 -8.1% 1.09x

Code size: -O

TEST OLD NEW DELTA RATIO
Regression
SetTests.o 64415 75359 +17.0% 0.85x

Performance: -Osize

TEST OLD NEW DELTA RATIO
Regression
SetSubtractingInt25 106 117 +10.4% 0.91x (?)
ObjectiveCBridgeFromNSArrayAnyObjectForced 4641 4993 +7.6% 0.93x (?)
Improvement
SetIntersectionInt100 342 75 -78.1% 4.56x
SetSubtractingInt100 174 80 -54.0% 2.17x
SetIntersectionInt50 209 104 -50.2% 2.01x
SetIntersectionInt25 143 84 -41.3% 1.70x
SetIntersectionBox25 320 229 -28.4% 1.40x
SetSubtractingInt0 84 63 -25.0% 1.33x
SetSubtractingBox0 180 137 -23.9% 1.31x
SetSubtractingInt50 134 104 -22.4% 1.29x
SetSubtractingBox25 319 265 -16.9% 1.20x
Breadcrumbs.CopyUTF16CodeUnits.Mixed 66 55 -16.7% 1.20x
RGBHistogram 2723 2543 -6.6% 1.07x (?)

Code size: -Osize

TEST OLD NEW DELTA RATIO
Regression
SetTests.o 57759 66335 +14.8% 0.87x

Performance: -Onone

TEST OLD NEW DELTA RATIO
Regression
EqualSubstringSubstringGenericEquatable 45 51 +13.3% 0.88x
Improvement
SetIntersectionInt100 885 202 -77.2% 4.38x
SetSubtractingInt100 606 229 -62.2% 2.65x
SetIntersectionInt50 559 264 -52.8% 2.12x
SetSubtractingInt50 528 293 -44.5% 1.80x
SetSubtractingInt0 374 214 -42.8% 1.75x
SetIntersectionInt25 393 237 -39.7% 1.66x
SetIntersectionBox25 1062 711 -33.1% 1.49x
SetSubtractingInt25 476 325 -31.7% 1.46x
SetSubtractingBox0 719 573 -20.3% 1.25x
ArrayOfPOD 859 779 -9.3% 1.10x (?)
How to read the data The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview
  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB
--------------

@lorentey
Copy link
Member Author

The benchmark suit doesn't cover all these operations. Here are some highlights from local testing:

Set.isSubset<S: Sequence>(of: S):
set - 8 tasks

Set.isStrictSubset<S: Sequence>(of: S):
set - 8 tasks

Set.isSuperset<S: Sequence>(of: S):
set - 8 tasks

Set.isStrictSuperset<S: Sequence>(of: S):
set - 8 tasks

Set.isDisjoint<S: Sequence>(with: S):
set - 8 tasks

@lorentey
Copy link
Member Author

Similar charts for operations returning sets:

Set.subtracting(_: Set):
set - 10 tasks

Set.subtracting<S: Sequence>(_: S):
set - 10 tasks

Set.filter(_:):
set - 10 tasks

Set.intersection(_: Set):
set - 10 tasks

Set.intersection<S: Sequence>(_: S):
set - 10 tasks

@lorentey
Copy link
Member Author

@swift-ci please test

@swift-ci

This comment has been minimized.

@swift-ci
Copy link
Contributor

Build failed
Swift Test OS X Platform
Git Sha - d7206d1f51ba47bdecec68d6721836d986c524c7

@lorentey
Copy link
Member Author

Swift(linux-x86_64).SIL.parse_stdlib.sil (from Swift(linux-x86_64)):

sil-opt: [...]/swift/lib/ParseSIL/ParseSIL.cpp:1133: \
llvm::PointerUnion<ValueDecl *, ModuleDecl *> lookupTopDecl(\
  swift::Parser &, swift::DeclBaseName): \
  Assertion `DeclLookup.isSuccess() && DeclLookup.Results.size() == 1' failed.

Hm... 🤨

@lorentey
Copy link
Member Author

@swift-ci please test linux platform

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 76f1ac547f623661a217fc369b991564775f579b

@lorentey
Copy link
Member Author

@swift-ci please test

@lorentey
Copy link
Member Author

@swift-ci benchmark

@swift-ci

This comment has been minimized.

@swift-ci

This comment has been minimized.

@swift-ci
Copy link
Contributor

Build comment file:

Performance: -O

TEST OLD NEW DELTA RATIO
Regression
Set.Empty.IsDisjointBox0 1 2 +99.9% 0.50x
SetSubtractingInt25 104 134 +28.8% 0.78x
SetIntersectionInt0 68 84 +23.5% 0.81x
SetSubtractingInt0 73 88 +20.5% 0.83x
Breadcrumbs.CopyUTF16CodeUnits.Mixed 56 61 +8.9% 0.92x
Improvement
SetIntersectionInt100 328 79 -75.9% 4.15x
SetIntersectionBox25 271 128 -52.8% 2.12x
SetIsDisjointInt25 2 1 -50.0% 2.00x
SetIsDisjointInt50 2 1 -50.0% 2.00x
SetSubtractingInt100 152 86 -43.4% 1.77x
SetIntersectionInt50 198 116 -41.4% 1.71x
SetIsDisjointBox25 5 3 -40.0% 1.67x
SetIntersectionBox0 140 88 -37.1% 1.59x
SetIntersectionInt25 138 99 -28.3% 1.39x
SetSubtractingInt50 129 118 -8.5% 1.09x
SetSubtractingBox0 149 137 -8.1% 1.09x

Code size: -O

TEST OLD NEW DELTA RATIO
Regression
SetTests.o 74743 87639 +17.3% 0.85x

Performance: -Osize

TEST OLD NEW DELTA RATIO
Regression
Set.Empty.IsDisjointInt0 1 2 +99.9% 0.50x
Set.Empty.IsDisjointBox0 1 2 +99.9% 0.50x
Breadcrumbs.CopyUTF16CodeUnits.Mixed 57 67 +17.5% 0.85x
Improvement
SetIntersectionInt100 345 74 -78.6% 4.66x
SetSubtractingInt100 167 80 -52.1% 2.09x
SetIntersectionInt50 205 102 -50.2% 2.01x
SetIsDisjointBox25 7 4 -42.9% 1.75x
SetIntersectionInt25 140 83 -40.7% 1.69x
SetIntersectionBox25 327 234 -28.4% 1.40x
SetSubtractingInt0 86 64 -25.6% 1.34x
SetSubtractingBox0 182 137 -24.7% 1.33x
SetSubtractingInt50 138 104 -24.6% 1.33x
SetSubtractingBox25 316 259 -18.0% 1.22x

Code size: -Osize

TEST OLD NEW DELTA RATIO
Regression
SetTests.o 66551 76903 +15.6% 0.87x

Performance: -Onone

TEST OLD NEW DELTA RATIO
Regression
Set.Empty.IsDisjointBox0 6 8 +33.3% 0.75x
Set.Empty.IsDisjointInt0 6 7 +16.7% 0.86x
ArrayOfPOD 780 858 +10.0% 0.91x (?)
Improvement
SetIntersectionInt100 891 203 -77.2% 4.39x
SetSubtractingInt100 605 227 -62.5% 2.67x
SetIntersectionInt50 563 259 -54.0% 2.17x
SetSubtractingInt50 524 289 -44.8% 1.81x
SetSubtractingInt0 372 214 -42.5% 1.74x
SetIntersectionInt25 399 236 -40.9% 1.69x
SetIsDisjointBox25 27 16 -40.7% 1.69x
SetIntersectionBox25 1081 713 -34.0% 1.52x
SetIsDisjointInt50 13 9 -30.8% 1.44x
SetIsDisjointInt25 14 10 -28.6% 1.40x
SetSubtractingInt25 473 355 -24.9% 1.33x
SetSubtractingBox0 720 572 -20.6% 1.26x
How to read the data The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview
  Model Name: Mac Pro
  Model Identifier: MacPro6,1
  Processor Name: 12-Core Intel Xeon E5
  Processor Speed: 2.7 GHz
  Number of Processors: 1
  Total Number of Cores: 12
  L2 Cache (per Core): 256 KB
  L3 Cache: 30 MB
  Memory: 64 GB
--------------

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 64f62f7831bc93744ce81aaf8bf5f199ad3eee28

Use a temporary bitset to speed up the Sequence variant by roughly a factor of 3.
Use a temporary bitset to speed up the Sequence variant by roughly a factor of 3.
In the Sequence variant, don’t convert the argument to a Set — just check if self contains all elements. This makes isSuperset(of:) roughly 4 times faster.
- Use a temporary bitset to speed up the Sequence variant by roughly a factor of 4.
- Fix a logic error causing the a == b case for the set variant to be O(n) instead of O(1).
- In the sequence case, don’t convert the input to a set; simply test its elements one by one. This leads to a cool 4x speedup.
- In the set case, prefer to iterate over the smaller set. The leads to a speedup proportional to the ratio of the counts between the two sets if the caller used an unlucky ordering.
Use a temporary bitset to avoid hashing elements more than once, and to prevent rehashings during the creation of the result set.

This leads to a speedup of about 0-4x, depending on the number of elements removed.
This works the same way as Set.subtracting<S>(_:), and has similar performance benefits.
Use a temporary bitset to speed up the Sequence variant by roughly a factor of ~4-6, and the set-set variant by a factor of ~1-4, depending on the ratio of overlapping elements.
@palimondo
Copy link
Contributor

@swift-ci please benchmark

@swift-ci
Copy link
Contributor

swift-ci commented May 7, 2019

Performance: -O

TEST OLD NEW DELTA RATIO
Regression
Set.subtracting.Box.Empty 24 158 +558.3% 0.15x
Set.subtracting.Int.Empty 44 152 +245.4% 0.29x
Set.subtracting.Empty.Box 9 15 +66.7% 0.60x
Set.isDisjoint.Empty.Box 101 140 +38.6% 0.72x
SetSubtractingInt25 63 75 +19.0% 0.84x
DistinctClassFieldAccesses 178 200 +12.4% 0.89x
DataCreateEmptyArray 1650 1850 +12.1% 0.89x
Set.isDisjoint.Empty.Int 92 101 +9.8% 0.91x (?)
Improvement
Set.isStrictSubset.Int100 417 0 -100.0% 417001.00x
SetIntersectionInt100 214 50 -76.6% 4.28x
Set.isStrictSubset.Box0 295 95 -67.8% 3.11x
Set.isStrictSubset.Int0 207 87 -58.0% 2.38x
SetIntersectionBox25 186 87 -53.2% 2.14x
SetSubtractingInt100 100 53 -47.0% 1.89x
SetIntersectionInt50 132 70 -47.0% 1.89x
SetIntersectionBox0 96 58 -39.6% 1.66x
SetIntersectionInt25 91 57 -37.4% 1.60x
Set.isDisjoint.Box25 412 287 -30.3% 1.44x
Set.isDisjoint.Int25 169 127 -24.9% 1.33x
Set.isDisjoint.Int50 167 126 -24.6% 1.33x
Dictionary4 194 153 -21.1% 1.27x
SetSubtractingInt50 80 66 -17.5% 1.21x
Set.isStrictSubset.Int.Empty 52 44 -15.4% 1.18x
ArrayAppend 460 400 -13.0% 1.15x (?)
Dictionary4OfObjects 226 198 -12.4% 1.14x
CharacterLiteralsLarge 71 63 -11.3% 1.13x (?)
SetSubtractingBox0 113 102 -9.7% 1.11x (?)
SetSubtractingBox25 204 185 -9.3% 1.10x (?)
CharacterLiteralsSmall 222 204 -8.1% 1.09x (?)
Chars2 3250 3000 -7.7% 1.08x
Set.isStrictSubset.Empty.Int 138 128 -7.2% 1.08x (?)

Code size: -O

TEST OLD NEW DELTA RATIO
Regression
SetTests.o 80038 88566 +10.7% 0.90x

Performance: -Osize

TEST OLD NEW DELTA RATIO
Regression
Set.subtracting.Box.Empty 23 158 +586.9% 0.15x
Set.subtracting.Int.Empty 45 181 +302.2% 0.25x
Set.isDisjoint.Empty.Box 93 150 +61.3% 0.62x
Set.subtracting.Empty.Box 9 14 +55.5% 0.64x
DataAppendDataLargeToLarge 17400 24400 +40.2% 0.71x (?)
Set.isDisjoint.Empty.Int 98 135 +37.8% 0.73x
SetSubtractingInt25 63 77 +22.2% 0.82x
DistinctClassFieldAccesses 177 200 +13.0% 0.89x
ObjectiveCBridgeStubFromNSStringRef 93 101 +8.6% 0.92x (?)
ArrayInClass 845 915 +8.3% 0.92x (?)
ArrayLiteral2 113 122 +8.0% 0.93x (?)
Improvement
Set.isStrictSubset.Int100 393 0 -100.0% 393001.00x
SetIntersectionInt100 221 51 -76.9% 4.33x
Set.isStrictSubset.Box0 300 89 -70.3% 3.37x
Set.isStrictSubset.Int0 205 84 -59.0% 2.44x
SetSubtractingInt100 105 52 -50.5% 2.02x
SetIntersectionInt50 133 68 -48.9% 1.96x
SetIntersectionInt25 91 54 -40.7% 1.69x
Set.isDisjoint.Box25 556 381 -31.5% 1.46x (?)
SetIntersectionBox25 229 167 -27.1% 1.37x
SetSubtractingBox0 130 98 -24.6% 1.33x
Dictionary4 215 163 -24.2% 1.32x (?)
SetSubtractingBox25 228 184 -19.3% 1.24x
DropLastCountableRangeLazy 6 5 -16.7% 1.20x (?)
SetSubtractingInt50 81 68 -16.0% 1.19x
Set.isDisjoint.Int25 251 214 -14.7% 1.17x
CharacterLiteralsLarge 67 58 -13.4% 1.16x (?)
Set.isDisjoint.Int50 247 214 -13.4% 1.15x
CharacterLiteralsSmall 224 202 -9.8% 1.11x (?)
OpenClose 67 61 -9.0% 1.10x (?)
Calculator 149 136 -8.7% 1.10x
Set.isStrictSubset.Int.Empty 58 53 -8.6% 1.09x
Set.isSubset.Int.Empty 60 55 -8.3% 1.09x (?)
Dictionary4OfObjects 357 330 -7.6% 1.08x (?)
StringInterpolationSmall 1320 1230 -6.8% 1.07x (?)

Code size: -Osize

TEST OLD NEW DELTA RATIO
Regression
SetTests.o 71446 77814 +8.9% 0.92x

Performance: -Onone

TEST OLD NEW DELTA RATIO
Regression
Set.subtracting.Box.Empty 100 233 +133.0% 0.43x
Set.subtracting.Int.Empty 107 241 +125.2% 0.44x
Set.isDisjoint.Empty.Box 325 472 +45.2% 0.69x
Set.isDisjoint.Empty.Int 307 444 +44.6% 0.69x
SetIntersect 1350 1490 +10.4% 0.91x
SetIntersectionInt0 135 149 +10.4% 0.91x
SetUnionInt100 242 263 +8.7% 0.92x (?)
Improvement
Set.isStrictSubset.Int100 1028 1 -99.9% 1026.97x
SetIntersectionInt100 593 152 -74.4% 3.90x
Set.isStrictSubset.Box0 596 185 -69.0% 3.22x
SetSubtractingInt100 412 145 -64.8% 2.84x
Set.isStrictSubset.Int0 429 172 -59.9% 2.49x
SetIntersectionInt50 374 186 -50.3% 2.01x
SetSubtractingInt50 350 188 -46.3% 1.86x
SetSubtractingInt0 254 141 -44.5% 1.80x
Set.isDisjoint.Box25 1701 969 -43.0% 1.76x
SetIntersectionInt25 265 168 -36.6% 1.58x
SetIntersectionBox25 708 466 -34.2% 1.52x
SetSubtractingInt25 317 212 -33.1% 1.50x (?)
DataAppendDataLargeToLarge 24400 17600 -27.9% 1.39x (?)
Set.isDisjoint.Int25 896 648 -27.7% 1.38x
Set.isStrictSubset.Int.Empty 206 152 -26.2% 1.36x
Set.isDisjoint.Int50 869 648 -25.4% 1.34x
SetSubtractingBox0 481 376 -21.8% 1.28x
Set.isStrictSubset.Empty.Int 344 294 -14.5% 1.17x (?)
ArrayOfPOD 530 467 -11.9% 1.13x (?)
TypeFlood 151 135 -10.6% 1.12x (?)
ArrayOfGenericPOD2 617 575 -6.8% 1.07x (?)
How to read the data The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview
  Model Name: Mac mini
  Model Identifier: Macmini8,1
  Processor Name: Intel Core i7
  Processor Speed: 3.2 GHz
  Number of Processors: 1
  Total Number of Cores: 6
  L2 Cache (per Core): 256 KB
  L3 Cache: 12 MB
  Memory: 64 GB

@palimondo
Copy link
Contributor

@swift-ci please test

@swift-ci
Copy link
Contributor

Build failed
Swift Test Linux Platform
Git Sha - 010164e

@palimondo
Copy link
Contributor

palimondo commented May 12, 2019

@swift-ci please benchmark
Edit: Now with the brand new #24156! 🎉

@swift-ci
Copy link
Contributor

Performance: -O

TEST OLD NEW DELTA RATIO
Regression
Set.subtracting.Box.Empty 23 159 +591.3% 0.14x
Set.isStrictSubset.Seq.Int.Empty 116 619 +433.6% 0.19x
Set.isSubset.Seq.Int.Empty 117 619 +429.1% 0.19x
Set.subtracting.Seq.Int.Empty 187 732 +291.4% 0.26x
Set.subtracting.Seq.Box.Empty 216 826 +282.4% 0.26x
Set.isStrictSuperset.Seq.Empty.Int 162 618 +281.5% 0.26x
Set.subtracting.Int.Empty 44 152 +245.4% 0.29x
Set.subtracting.Empty.Box 8 13 +62.5% 0.62x
Set.isDisjoint.Empty.Box 101 140 +38.6% 0.72x
Set.subtracting.Seq.Int25 284 368 +29.6% 0.77x (?)
SetSubtractingInt25 63 75 +19.0% 0.84x (?)
Calculator 131 154 +17.6% 0.85x
Chars2 3050 3450 +13.1% 0.88x (?)
Array2D 3472 3920 +12.9% 0.89x (?)
RemoveWhereSwapInts 31 35 +12.9% 0.89x (?)
Set.isDisjoint.Box.Empty 96 108 +12.5% 0.89x
RemoveWhereMoveInts 17 19 +11.8% 0.89x
FlattenListLoop 2162 2407 +11.3% 0.90x (?)
MapReduce 193 214 +10.9% 0.90x (?)
MapReduceAnyCollection 193 214 +10.9% 0.90x (?)
Set.isSubset.Int.Empty 47 52 +10.6% 0.90x (?)
Set.isDisjoint.Empty.Int 92 101 +9.8% 0.91x (?)
Set.isDisjoint.Int.Empty 53 58 +9.4% 0.91x (?)
FlattenListFlatMap 3365 3667 +9.0% 0.92x (?)
OpenClose 56 61 +8.9% 0.92x (?)
LazilyFilteredArrayContains 23000 24800 +7.8% 0.93x
Improvement
Set.isStrictSubset.Int100 41838 87 -99.8% 480.89x
Set.isStrictSuperset.Seq.Int.Empty 31650 87 -99.7% 363.79x
Set.isStrictSubset.Seq.Empty.Int 31706 226 -99.3% 140.29x
Set.isStrictSuperset.Seq.Box0 8530 85 -99.0% 100.35x
Set.isStrictSuperset.Seq.Int0 3172 69 -97.8% 45.97x
Set.filter.Int100.28k 1226 85 -93.1% 14.42x
Set.filter.Int100.16k 648 48 -92.6% 13.50x
Set.filter.Int100.20k 730 59 -91.9% 12.37x
Set.filter.Int100.24k 824 70 -91.5% 11.77x
Set.isSubset.Seq.Box25 970 130 -86.6% 7.46x
Set.isStrictSubset.Seq.Box25 971 133 -86.3% 7.30x
Set.isStrictSuperset.Seq.Box25 854 132 -84.5% 6.47x
Set.isSubset.Seq.Int25 374 60 -84.0% 6.23x
Set.isStrictSubset.Seq.Int25 374 60 -84.0% 6.23x
Set.intersection.Seq.Int100 281 48 -82.9% 5.85x
Set.isStrictSuperset.Seq.Int25 321 61 -81.0% 5.26x
SetIntersectionInt100 215 50 -76.7% 4.30x
Set.isStrictSubset.Seq.Int50 427 122 -71.4% 3.50x
Set.isSubset.Seq.Int50 426 123 -71.1% 3.46x
Set.isStrictSubset.Box0 295 95 -67.8% 3.11x
Set.isStrictSubset.Seq.Int100 742 243 -67.3% 3.05x
Set.intersection.Seq.Int50 197 65 -67.0% 3.03x
Set.isStrictSuperset.Seq.Int100 742 245 -67.0% 3.03x
Set.intersection.Seq.Int25 156 52 -66.7% 3.00x
Set.intersection.Seq.Int0 107 37 -65.4% 2.89x
Set.intersection.Seq.Box25 357 125 -65.0% 2.86x
Set.intersection.Seq.Box0 268 96 -64.2% 2.79x
Set.isStrictSuperset.Seq.Int50 320 119 -62.8% 2.69x
Set.isStrictSubset.Int0 203 87 -57.1% 2.33x
Set.filter.Int50.28k 729 326 -55.3% 2.24x
Set.isSubset.Seq.Int100 534 245 -54.1% 2.18x
Set.filter.Int50.16k 394 183 -53.6% 2.15x
SetIntersectionBox25 184 88 -52.2% 2.09x
Set.subtracting.Seq.Int100 502 243 -51.6% 2.07x
Set.filter.Int50.20k 459 227 -50.5% 2.02x (?)
Set.filter.Int50.24k 527 271 -48.6% 1.94x (?)
SetIntersectionInt50 129 69 -46.5% 1.87x (?)
SetSubtractingInt100 100 54 -46.0% 1.85x
Set.isStrictSubset.Seq.Box0 856 481 -43.8% 1.78x
Set.isSubset.Seq.Box0 855 490 -42.7% 1.74x
Set.isSubset.Seq.Int0 325 188 -42.2% 1.73x
Set.isStrictSubset.Seq.Int0 324 192 -40.7% 1.69x
SetIntersectionBox0 96 57 -40.6% 1.68x
SetIntersectionInt25 90 57 -36.7% 1.58x
Set.isDisjoint.Box25 409 287 -29.8% 1.43x
Set.isDisjoint.Int25 169 123 -27.2% 1.37x
Set.isDisjoint.Int50 167 124 -25.7% 1.35x
SetSubtractingInt50 79 67 -15.2% 1.18x
ObjectiveCBridgeStubNSDateRefAccess 196 174 -11.2% 1.13x (?)
Set.subtracting.Seq.Box0 530 476 -10.2% 1.11x
SetSubtractingBox25 204 185 -9.3% 1.10x
SetSubtractingBox0 113 103 -8.8% 1.10x (?)
Set.subtracting.Seq.Int50 357 330 -7.6% 1.08x (?)
ObjectiveCBridgeStubFromNSStringRef 98 91 -7.1% 1.08x (?)

Code size: -O

TEST OLD NEW DELTA RATIO
Regression
SetTests.o 152958 166494 +8.8% 0.92x

Performance: -Osize

TEST OLD NEW DELTA RATIO
Regression
Set.subtracting.Box.Empty 23 158 +586.9% 0.15x
Set.isSubset.Seq.Int.Empty 122 636 +421.3% 0.19x
Set.isStrictSubset.Seq.Int.Empty 122 630 +416.4% 0.19x
Set.subtracting.Seq.Int.Empty 192 898 +367.7% 0.21x
Set.subtracting.Seq.Box.Empty 224 897 +300.4% 0.25x
Set.subtracting.Int.Empty 45 179 +297.8% 0.25x
Set.isStrictSuperset.Seq.Empty.Int 172 634 +268.6% 0.27x
Set.subtracting.Empty.Box 8 13 +62.5% 0.62x
Set.isDisjoint.Empty.Box 93 149 +60.2% 0.62x
Set.isDisjoint.Empty.Int 98 136 +38.8% 0.72x
Set.subtracting.Seq.Int25 284 370 +30.3% 0.77x
Array2D 3680 4592 +24.8% 0.80x (?)
RemoveWhereSwapInts 33 40 +21.2% 0.83x
SetSubtractingInt25 63 76 +20.6% 0.83x (?)
MapReduceAnyCollection 216 259 +19.9% 0.83x
FlattenListLoop 2226 2658 +19.4% 0.84x (?)
RemoveWhereMoveInts 18 21 +16.7% 0.86x (?)
ArraySetElement 262 305 +16.4% 0.86x
PrefixWhileAnySeqCntRange 188 214 +13.8% 0.88x (?)
PrefixWhileSequence 190 216 +13.7% 0.88x (?)
PrefixWhileAnySeqCRangeIter 189 214 +13.2% 0.88x (?)
RemoveWhereFilterInts 23 26 +13.0% 0.88x (?)
ObjectiveCBridgeStubFromNSStringRef 92 103 +12.0% 0.89x (?)
Set.isDisjoint.Seq.Box.Empty 90 100 +11.1% 0.90x
Set.isDisjoint.Seq.Int.Empty 54 60 +11.1% 0.90x (?)
MapReduce 236 258 +9.3% 0.91x (?)
Set.subtracting.Seq.Empty.Int 141 154 +9.2% 0.92x (?)
RangeAssignment 198 216 +9.1% 0.92x (?)
Set.subtracting.Seq.Empty.Box 143 156 +9.1% 0.92x (?)
Improvement
Set.isStrictSubset.Int100 39255 84 -99.8% 467.32x
Set.isStrictSuperset.Seq.Int.Empty 33039 92 -99.7% 359.12x
Set.isStrictSuperset.Seq.Box0 12252 86 -99.3% 142.46x
Set.isStrictSubset.Seq.Empty.Int 32654 237 -99.3% 137.78x
Set.isStrictSuperset.Seq.Int0 3314 70 -97.9% 47.34x
Set.filter.Int100.28k 1268 88 -93.1% 14.41x
Set.filter.Int100.16k 675 50 -92.6% 13.50x
Set.filter.Int100.20k 758 61 -92.0% 12.43x
Set.filter.Int100.24k 857 73 -91.5% 11.74x
Set.isSubset.Seq.Box25 1426 132 -90.7% 10.80x
Set.isStrictSubset.Seq.Box25 1425 134 -90.6% 10.63x
Set.isStrictSuperset.Seq.Box25 1234 132 -89.3% 9.35x
Set.isStrictSubset.Seq.Int25 379 60 -84.2% 6.32x
Set.isSubset.Seq.Int25 378 60 -84.1% 6.30x
Set.intersection.Seq.Int100 289 48 -83.4% 6.02x
Set.isStrictSuperset.Seq.Int25 335 60 -82.1% 5.58x
SetIntersectionInt100 221 50 -77.4% 4.42x
Set.intersection.Seq.Box0 365 95 -74.0% 3.84x
Set.intersection.Seq.Box25 477 125 -73.8% 3.82x
Set.isSubset.Seq.Int50 433 116 -73.2% 3.73x
Set.isStrictSubset.Seq.Int50 433 118 -72.7% 3.67x
Set.isStrictSubset.Box0 295 89 -69.8% 3.31x
Set.intersection.Seq.Int25 158 51 -67.7% 3.10x
Set.intersection.Seq.Int50 201 66 -67.2% 3.05x
Set.isStrictSubset.Seq.Int100 723 240 -66.8% 3.01x
Set.isStrictSuperset.Seq.Int100 727 242 -66.7% 3.00x
Set.isStrictSuperset.Seq.Int50 337 120 -64.4% 2.81x
Set.intersection.Seq.Int0 106 38 -64.2% 2.79x
Set.isStrictSubset.Seq.Box0 1229 485 -60.5% 2.53x
Set.isSubset.Seq.Box0 1230 486 -60.5% 2.53x
Set.isStrictSubset.Int0 205 84 -59.0% 2.44x
Set.filter.Int50.28k 768 332 -56.8% 2.31x
Set.filter.Int50.16k 417 188 -54.9% 2.22x
Set.isSubset.Seq.Int100 521 235 -54.9% 2.22x
Set.subtracting.Seq.Int100 520 240 -53.8% 2.17x
Set.filter.Int50.20k 483 233 -51.8% 2.07x
Set.filter.Int50.24k 552 279 -49.5% 1.98x
SetSubtractingInt100 104 53 -49.0% 1.96x
SetIntersectionInt50 132 68 -48.5% 1.94x
Set.isSubset.Seq.Int0 332 181 -45.5% 1.83x
Set.isStrictSubset.Seq.Int0 333 188 -43.5% 1.77x
SetIntersectionInt25 90 54 -40.0% 1.67x
Set.isDisjoint.Box25 559 373 -33.3% 1.50x
SetIntersectionBox25 228 168 -26.3% 1.36x
Set.subtracting.Seq.Box0 644 485 -24.7% 1.33x (?)
SetSubtractingBox0 130 98 -24.6% 1.33x
Dictionary4 212 164 -22.6% 1.29x
SetSubtractingBox25 227 184 -18.9% 1.23x
Set.subtracting.Seq.Box25 1108 905 -18.3% 1.22x (?)
SetSubtractingInt50 80 68 -15.0% 1.18x
Set.isDisjoint.Int25 251 214 -14.7% 1.17x
Set.isDisjoint.Int50 247 214 -13.4% 1.15x (?)
Set.subtracting.Seq.Int50 365 332 -9.0% 1.10x
Set.isStrictSubset.Int.Empty 55 51 -7.3% 1.08x (?)
Dictionary4OfObjects 349 326 -6.6% 1.07x

Code size: -Osize

TEST OLD NEW DELTA RATIO
Regression
SetTests.o 126038 140934 +11.8% 0.89x

Performance: -Onone

TEST OLD NEW DELTA RATIO
Regression
Set.subtracting.Box.Empty 100 232 +132.0% 0.43x
Set.subtracting.Int.Empty 108 241 +123.1% 0.45x
Set.subtracting.Seq.Int.Empty 592 1201 +102.9% 0.49x
Set.subtracting.Seq.Box.Empty 611 1225 +100.5% 0.50x
Set.isDisjoint.Empty.Int 304 444 +46.1% 0.68x
Set.isSubset.Seq.Int.Empty 804 1152 +43.3% 0.70x
Set.isDisjoint.Empty.Box 317 454 +43.2% 0.70x
Set.isStrictSubset.Seq.Int.Empty 804 1114 +38.6% 0.72x
Set.isSuperset.Seq.Empty.Int 434 574 +32.3% 0.76x
Set.isSubset.Seq.Empty.Int 183 218 +19.1% 0.84x
Set.isStrictSuperset.Seq.Empty.Int 944 1119 +18.5% 0.84x
Set.isSuperset.Seq.Int.Empty 1341 1505 +12.2% 0.89x (?)
Set.isSuperset.Seq.Int0 1429 1592 +11.4% 0.90x (?)
SetIntersect 1350 1500 +11.1% 0.90x (?)
SetIntersectionInt0 135 150 +11.1% 0.90x
Set.isSuperset.Seq.Box0 1645 1823 +10.8% 0.90x (?)
ArrayOfGenericPOD2 574 618 +7.7% 0.93x (?)
Improvement
Set.isStrictSuperset.Seq.Int.Empty 530476 184 -100.0% 2883.01x
Set.isStrictSubset.Int100 103440 172 -99.8% 601.39x
Set.isStrictSubset.Seq.Empty.Int 525401 1599 -99.7% 328.58x
Set.isStrictSuperset.Seq.Box0 70743 243 -99.7% 291.12x
Set.isStrictSuperset.Seq.Int0 53118 220 -99.6% 241.44x
Set.filter.Int100.28k 3673 607 -83.5% 6.05x
Set.filter.Int100.16k 2012 346 -82.8% 5.82x
Set.filter.Int100.20k 2391 433 -81.9% 5.52x
Set.filter.Int100.24k 2805 519 -81.5% 5.40x
Set.isSubset.Seq.Box25 7619 1497 -80.4% 5.09x
Set.isStrictSubset.Seq.Box25 7616 1520 -80.0% 5.01x
Set.isSubset.Seq.Int25 5456 1095 -79.9% 4.98x
Set.isStrictSubset.Seq.Int25 5459 1100 -79.8% 4.96x
Set.isStrictSuperset.Seq.Int25 5311 1075 -79.8% 4.94x
Set.isStrictSuperset.Seq.Box25 7159 1499 -79.1% 4.78x
SetIntersectionInt100 593 152 -74.4% 3.90x
Set.isStrictSubset.Box0 585 185 -68.4% 3.16x
SetSubtractingInt100 413 147 -64.4% 2.81x
Set.isStrictSubset.Seq.Int50 5603 2162 -61.4% 2.59x
Set.isSubset.Seq.Int50 5549 2158 -61.1% 2.57x
Set.isStrictSubset.Int0 429 172 -59.9% 2.49x
Set.isStrictSuperset.Seq.Int50 5304 2128 -59.9% 2.49x
Set.filter.Int50.28k 2178 966 -55.6% 2.25x
Set.filter.Int50.16k 1197 559 -53.3% 2.14x
SetIntersectionInt50 376 186 -50.5% 2.02x
Set.filter.Int50.20k 1430 721 -49.6% 1.98x
Set.intersection.Seq.Int100 1657 852 -48.6% 1.94x
Set.filter.Int50.24k 1688 900 -46.7% 1.88x
SetSubtractingInt50 352 189 -46.3% 1.86x
SetSubtractingInt0 253 143 -43.5% 1.77x
Set.isDisjoint.Box25 1692 973 -42.5% 1.74x
Set.intersection.Seq.Box25 2140 1257 -41.3% 1.70x
Set.intersection.Seq.Int50 1437 903 -37.2% 1.59x
SetIntersectionInt25 265 169 -36.2% 1.57x
Set.intersection.Seq.Box0 1809 1157 -36.0% 1.56x
Set.intersection.Seq.Int25 1328 884 -33.4% 1.50x
SetIntersectionBox25 706 472 -33.1% 1.50x
Set.isStrictSuperset.Seq.Int100 6345 4258 -32.9% 1.49x
SetSubtractingInt25 318 214 -32.7% 1.49x
Set.isStrictSubset.Seq.Int100 6343 4304 -32.1% 1.47x
Set.intersection.Seq.Int0 1193 859 -28.0% 1.39x
Set.isDisjoint.Int25 887 647 -27.1% 1.37x
Set.isSubset.Seq.Int100 5916 4321 -27.0% 1.37x
Set.isStrictSubset.Int.Empty 202 148 -26.7% 1.36x
Set.isDisjoint.Int50 866 647 -25.3% 1.34x
Set.subtracting.Seq.Int100 5707 4276 -25.1% 1.33x
SetSubtractingBox0 477 378 -20.8% 1.26x
Set.isSubset.Seq.Box0 7118 5760 -19.1% 1.24x (?)
Set.isStrictSubset.Seq.Int0 5322 4316 -18.9% 1.23x
Set.isStrictSubset.Seq.Box0 7100 5776 -18.6% 1.23x
Set.isSubset.Seq.Int0 5300 4347 -18.0% 1.22x
Set.subtracting.Seq.Empty.Int 347 293 -15.6% 1.18x
Set.subtracting.Seq.Int50 5290 4512 -14.7% 1.17x (?)
Set.isStrictSubset.Empty.Int 343 295 -14.0% 1.16x
Set.subtracting.Seq.Empty.Box 370 320 -13.5% 1.16x
Set.subtracting.Seq.Int0 4874 4298 -11.8% 1.13x
Set.subtracting.Seq.Int25 5127 4649 -9.3% 1.10x (?)
Set.subtracting.Seq.Box0 6208 5756 -7.3% 1.08x (?)
How to read the data The tables contain differences in performance which are larger than 8% and differences in code size which are larger than 1%.

If you see any unexpected regressions, you should consider fixing the
regressions before you merge the PR.

Noise: Sometimes the performance results (not code size!) contain false
alarms. Unexpected regressions which are marked with '(?)' are probably noise.
If you see regressions which you cannot explain you can try to run the
benchmarks again. If regressions still show up, please consult with the
performance team (@eeckstein).

Hardware Overview
  Model Name: Mac mini
  Model Identifier: Macmini8,1
  Processor Name: Intel Core i7
  Processor Speed: 3.2 GHz
  Number of Processors: 1
  Total Number of Cores: 6
  L2 Cache (per Core): 256 KB
  L3 Cache: 12 MB
  Memory: 64 GB

@palimondo
Copy link
Contributor

The runtimes of the extended Set.Sequence benchmarks from #24156 look good (all in the 20–1000 μs range). PR #24725 disables 3 benchmarks that run too long with the current implementation. Please revert it to re-enable them shortly before these optimizations land.

Similarly, there is a small issue with too low runtime (even before these changes) in a case of Set.subtracting.Empty.Box. This would be fixed by #24570, which should IMHO be also merged shortly before this one, to batch all the changes to benchmark runtimes without needing to burn good names on a small bug fix.

@shahmishal
Copy link
Member

Please update the base branch to main by Oct 5th otherwise the pull request will be closed automatically.

  • How to change the base branch: (Link)
  • More detail about the branch update: (Link)

@lorentey lorentey changed the base branch from master to main October 2, 2020 22:11
@lorentey
Copy link
Member Author

lorentey commented Nov 2, 2021

Closing in favor of #40012.

@lorentey lorentey closed this Nov 2, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants