Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[stdlib] Fix string index sharing #4896

Merged
merged 5 commits into from Oct 13, 2016

Conversation

Projects
None yet
6 participants
@natecook1000
Copy link
Member

commented Sep 21, 2016

This fixes the problems with index sharing in String.CharacterView, String.UnicodeScalarView, and String.UTF16View. This is the current behavior for those three views:

let str = "abcdefg"
let sub = str.characters.dropFirst(3)
print(str[sub.startIndex])
// Prints "a", but should print "d"

Resolves SR-1927.

@natecook1000

This comment has been minimized.

Copy link
Member Author

commented Sep 21, 2016

@swift-ci Please test

@swift-ci

This comment has been minimized.

Copy link
Contributor

commented Sep 21, 2016

Build failed
Jenkins build - Swift Test Linux Platform
Git Commit - 03279c48ce8fc4322626d899940467e588a34f44
Test requested by - @natecook1000

@swift-ci

This comment has been minimized.

Copy link
Contributor

commented Sep 21, 2016

Build failed
Jenkins build - Swift Test OS X Platform
Git Commit - 03279c48ce8fc4322626d899940467e588a34f44
Test requested by - @natecook1000

@natecook1000

This comment has been minimized.

Copy link
Member Author

commented Sep 21, 2016

@swift-ci ¯_(ツ)_/¯

@shahmishal

This comment has been minimized.

Copy link
Member

commented Sep 21, 2016

@natecook1000 This should be fixed now.

@shahmishal

This comment has been minimized.

Copy link
Member

commented Sep 21, 2016

@swift-ci Please test

@swift-ci

This comment has been minimized.

Copy link
Contributor

commented Sep 21, 2016

Build failed
Jenkins build - Swift Test Linux Platform
Git Commit - 03279c48ce8fc4322626d899940467e588a34f44
Test requested by - @shahmishal

@swift-ci

This comment has been minimized.

Copy link
Contributor

commented Sep 21, 2016

Build failed
Jenkins build - Swift Test OS X Platform
Git Commit - 03279c48ce8fc4322626d899940467e588a34f44
Test requested by - @shahmishal

@natecook1000 natecook1000 changed the title [stdlib] Fix string index sharing [WIP] [stdlib] Fix string index sharing Sep 21, 2016

@natecook1000 natecook1000 force-pushed the natecook1000:nc-stringdex branch to e81d25e Sep 21, 2016

@natecook1000

This comment has been minimized.

Copy link
Member Author

commented Sep 21, 2016

@swift-ci Please test

@swift-ci

This comment has been minimized.

Copy link
Contributor

commented Sep 21, 2016

Build failed
Jenkins build - Swift Test Linux Platform
Git Commit - 03279c48ce8fc4322626d899940467e588a34f44
Test requested by - @natecook1000

@@ -439,7 +447,8 @@ extension String.CharacterView {
let unicodeScalarRange =
bounds.lowerBound._base..<bounds.upperBound._base
return String.CharacterView(
String(_core).unicodeScalars[unicodeScalarRange]._core)
UnicodeScalarView(_core, coreOffset: _coreOffset)[unicodeScalarRange]._core,
coreOffset: bounds.lowerBound._base._position)

This comment has been minimized.

Copy link
@moiseev

moiseev Sep 21, 2016

Member

Just a minor nitpick: this expression has grown complex enough to be broken into 2 lets and a simple call. Indentation of the coreOffset: line does not help either :-)

Also, since unicodeScalarRange already contains bounds.lowerBound._base, it is probably worth expressing both arguments to String.CharacterView in terms of unicodeScalarRange, as in coreOffset: unicodeScalarRange.lowerBound._position. What do you think?

This comment has been minimized.

Copy link
@natecook1000

natecook1000 Sep 21, 2016

Author Member

Updated in the latest commit—I'd forgotten CharacterView already has a unicodeScalars property, which simplifies this a lot. Look okay?

@natecook1000 natecook1000 changed the title [WIP] [stdlib] Fix string index sharing [stdlib] Fix string index sharing Sep 21, 2016

@natecook1000

This comment has been minimized.

Copy link
Member Author

commented Sep 21, 2016

@swift-ci Please test

@natecook1000

This comment has been minimized.

Copy link
Member Author

commented Sep 22, 2016

@shahmishal It seems like this PR got disconnected from CI—should I create a new one?

@shahmishal

This comment has been minimized.

Copy link
Member

commented Sep 22, 2016

@natecook1000 This should be fixed now.

@shahmishal

This comment has been minimized.

Copy link
Member

commented Sep 22, 2016

@swift-ci Please test

@natecook1000

This comment has been minimized.

Copy link
Member Author

commented Sep 22, 2016

@swift-ci Please benchmark

@swift-ci

This comment has been minimized.

Copy link
Contributor

commented Sep 22, 2016

Build comment file:

Optimized (O)

Regression (9)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
ByteSwap 0 1 +100000.0% 0.00x
StringWithCString 152689 198939 +30.3% 0.77x
ObjectiveCBridgeFromNSDictionaryAnyObjectToString 209858 234824 +11.9% 0.89x
ClassArrayGetter 12 13 +8.3% 0.92x
ObjectiveCBridgeFromNSDictionaryAnyObjectToStringForced 134465 146593 +9.0% 0.92x
ObjectiveCBridgeStubFromArrayOfNSString 58439 63198 +8.1% 0.92x
ObjectiveCBridgeFromNSArrayAnyObject 75636 80315 +6.2% 0.94x
ObjectiveCBridgeToNSSet 15225 16194 +6.4% 0.94x(?)
ObjectiveCBridgeFromNSSetAnyObjectToStringForced 107733 114009 +5.8% 0.94x

Improvement (2)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
StringHasSuffix 767 699 -8.9% 1.10x
Calculator 37 33 -10.8% 1.12x

No Changes (125)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
ObjectiveCBridgeStubDataAppend 3071 2951 -3.9% 1.04x(?)
ArrayInClass 65 64 -1.5% 1.02x(?)
SetExclusiveOr_OfObjects 7785 7646 -1.8% 1.02x
Array2D 2068 2042 -1.3% 1.01x(?)
SortStrings 1848 1821 -1.5% 1.01x
Join 462 457 -1.1% 1.01x
ObjectiveCBridgeStubFromNSString 803 797 -0.8% 1.01x
ObjectiveCBridgeStubToNSStringRef 120 119 -0.8% 1.01x(?)
SortStringsUnicode 8235 8134 -1.2% 1.01x(?)
ObjectiveCBridgeStubToArrayOfNSString 29740 29557 -0.6% 1.01x(?)
Phonebook 7543 7470 -1.0% 1.01x(?)
SetUnion_OfObjects 6321 6267 -0.8% 1.01x
ArraySubscript 1412 1407 -0.3% 1.00x(?)
ObjectiveCBridgeToNSString 1072 1077 +0.5% 1.00x(?)
ObjectiveCBridgeFromNSDictionaryAnyObjectForced 5239 5221 -0.3% 1.00x(?)
ArrayOfGenericPOD 207 208 +0.5% 1.00x(?)
ObjectiveCBridgeFromNSStringForced 2694 2691 -0.1% 1.00x(?)
SortLettersInPlace 978 981 +0.3% 1.00x(?)
DictionarySwap 608 608 +0.0% 1.00x
RangeAssignment 279 279 +0.0% 1.00x
XorLoop 380 380 +0.0% 1.00x
ObjectiveCBridgeStubToNSString 1245 1246 +0.1% 1.00x(?)
ProtocolDispatch 3038 3037 -0.0% 1.00x(?)
TypeFlood 0 0 +0.0% 1.00x
ObjectiveCBridgeFromNSSetAnyObject 262056 260771 -0.5% 1.00x(?)
Dictionary3 509 507 -0.4% 1.00x(?)
Dictionary2 2030 2025 -0.2% 1.00x(?)
ObjectiveCBridgeStubToNSDateRef 3381 3376 -0.1% 1.00x(?)
PopFrontUnsafePointer 9044 9040 -0.0% 1.00x(?)
RC4 165 165 +0.0% 1.00x
DictionaryLiteral 2731 2736 +0.2% 1.00x(?)
DictionaryRemoveOfObjects 19070 19065 -0.0% 1.00x(?)
SetIsSubsetOf 476 475 -0.2% 1.00x
SetIntersect 1042 1040 -0.2% 1.00x(?)
SetExclusiveOr 2535 2536 +0.0% 1.00x(?)
ObjectiveCBridgeStubNSDateRefAccess 338 338 +0.0% 1.00x
ArrayOfGenericRef 3587 3594 +0.2% 1.00x(?)
ObjectiveCBridgeStubDateAccess 182 182 +0.0% 1.00x
ObjectiveCBridgeToNSDictionary 16362 16394 +0.2% 1.00x(?)
MonteCarloPi 42506 42530 +0.1% 1.00x(?)
MonteCarloE 9959 9988 +0.3% 1.00x(?)
HashTest 1701 1699 -0.1% 1.00x(?)
SetIsSubsetOf_OfObjects 485 486 +0.2% 1.00x
DictionaryRemove 4554 4570 +0.3% 1.00x(?)
LinkedList 6849 6852 +0.0% 1.00x(?)
ArrayLiteral 1158 1155 -0.3% 1.00x(?)
NSStringConversion 808 805 -0.4% 1.00x(?)
RGBHistogram 2830 2828 -0.1% 1.00x(?)
ArrayValueProp4 5 5 +0.0% 1.00x
ArrayOfPOD 172 172 +0.0% 1.00x
SetUnion 2861 2873 +0.4% 1.00x(?)
StringBuilder 1318 1318 +0.0% 1.00x
ArrayValueProp2 5 5 +0.0% 1.00x
BitCount 1 1 +0.0% 1.00x
ArrayValueProp3 5 5 +0.0% 1.00x
ArrayValueProp 5 5 +0.0% 1.00x
GlobalClass 0 0 +0.0% 1.00x
Memset 222 222 +0.0% 1.00x
ObjectiveCBridgeStubNSDataAppend 2457 2469 +0.5% 1.00x(?)
DictionarySwapOfObjects 6625 6690 +1.0% 0.99x(?)
Integrate 237 239 +0.8% 0.99x
PopFrontArray 1091 1102 +1.0% 0.99x(?)
Dictionary3OfObjects 833 840 +0.8% 0.99x
SuperChars 202128 204076 +1.0% 0.99x
StringInterpolation 10916 11008 +0.8% 0.99x(?)
AnyHashableWithAClass 62911 63578 +1.1% 0.99x(?)
ProtocolDispatch2 162 164 +1.2% 0.99x
ErrorHandling 2947 2989 +1.4% 0.99x(?)
ArrayOfRef 3474 3517 +1.2% 0.99x(?)
ObjectiveCBridgeStubURLAppendPathRef 241462 244851 +1.4% 0.99x(?)
ObjectiveCBridgeStubDateMutation 270 273 +1.1% 0.99x
ObjectiveCBridgeToNSArray 30840 31180 +1.1% 0.99x(?)
NSError 331 334 +0.9% 0.99x(?)
PopFrontArrayGeneric 1093 1099 +0.6% 0.99x(?)
CaptureProp 4463 4521 +1.3% 0.99x
MapReduce 340 342 +0.6% 0.99x(?)
Hanoi 3492 3534 +1.2% 0.99x(?)
ObjectiveCBridgeStubURLAppendPath 231134 234547 +1.5% 0.99x(?)
Sim2DArray 277 279 +0.7% 0.99x(?)
StringHasSuffixUnicode 62927 63438 +0.8% 0.99x
ArrayAppend 730 736 +0.8% 0.99x
DeadArray 185 187 +1.1% 0.99x
AngryPhonebook 2887 2910 +0.8% 0.99x
SevenBoom 1420 1431 +0.8% 0.99x(?)
RecursiveOwnedParameter 1842 1877 +1.9% 0.98x
ObjectiveCBridgeFromNSArrayAnyObjectToString 122368 125367 +2.5% 0.98x(?)
Histogram 636 646 +1.6% 0.98x(?)
ObjectiveCBridgeStubFromNSDateRef 3683 3744 +1.7% 0.98x
StrComplexWalk 2845 2901 +2.0% 0.98x(?)
ObjectiveCBridgeStubToNSDate 13381 13666 +2.1% 0.98x(?)
StaticArray 2747 2793 +1.7% 0.98x(?)
SetIntersect_OfObjects 2030 2065 +1.7% 0.98x
DictionaryOfObjects 2228 2277 +2.2% 0.98x(?)
StringEqualPointerComparison 7296 7434 +1.9% 0.98x(?)
IterateData 2471 2524 +2.1% 0.98x
OpenClose 53 54 +1.9% 0.98x
Dictionary 740 752 +1.6% 0.98x
RGBHistogramOfObjects 21168 21590 +2.0% 0.98x
Chars 619 630 +1.8% 0.98x
StringWalk 5618 5707 +1.6% 0.98x(?)
ObjectiveCBridgeStubNSDateMutationRef 12370 12613 +2.0% 0.98x(?)
ArrayAppendReserved 514 522 +1.6% 0.98x
TwoSum 1556 1594 +2.4% 0.98x
ObjectiveCBridgeStubFromNSDate 3588 3644 +1.6% 0.98x(?)
StackPromo 20888 21499 +2.9% 0.97x(?)
ObjectiveCBridgeFromNSArrayAnyObjectForced 6171 6384 +3.5% 0.97x(?)
Prims 701 723 +3.1% 0.97x
ObjectiveCBridgeStubFromNSStringRef 134 138 +3.0% 0.97x
ObjectAllocation 149 153 +2.7% 0.97x(?)
Walsh 309 319 +3.2% 0.97x
StringHasPrefix 591 608 +2.9% 0.97x
UTF8Decode 281 289 +2.9% 0.97x
ObjectiveCBridgeFromNSString 1818 1876 +3.2% 0.97x
ObjectiveCBridgeFromNSSetAnyObjectForced 4643 4767 +2.7% 0.97x
Dictionary2OfObjects 3384 3489 +3.1% 0.97x
ObjectiveCBridgeFromNSDictionaryAnyObject 203738 213176 +4.6% 0.96x
135 3021093 3156778 +4.5% 0.96x
NopDeinit 36530 38056 +4.2% 0.96x
NSDictionaryCastToSwift 6720 6978 +3.8% 0.96x
StrToInt 5226 5441 +4.1% 0.96x
ObjectiveCBridgeFromNSSetAnyObjectToString 147902 153976 +4.1% 0.96x
DictionaryBridge 3827 4026 +5.2% 0.95x(?)
PolymorphicCalls 20 21 +5.0% 0.95x
StringHasPrefixUnicode 13558 14345 +5.8% 0.95x
ObjectiveCBridgeFromNSArrayAnyObjectToStringForced 90728 95756 +5.5% 0.95x(?)
**Unoptimized (Onone)**

Regression (7)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
StrToInt 5293 5953 +12.5% 0.89x
StringWalk 20731 23342 +12.6% 0.89x
ObjectiveCBridgeFromNSDictionaryAnyObjectToString 216731 240873 +11.1% 0.90x(?)
ByteSwap 9 10 +11.1% 0.90x
ObjectiveCBridgeFromNSSetAnyObjectToStringForced 116682 127088 +8.9% 0.92x
ArrayAppend 3313 3559 +7.4% 0.93x
StrComplexWalk 7375 7848 +6.4% 0.94x

Improvement (1)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
ObjectiveCBridgeStubDateMutation 519 447 -13.9% 1.16x

No Changes (128)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
ObjectiveCBridgeStubFromNSDateRef 4262 4073 -4.4% 1.05x
ArrayOfGenericPOD 3524 3362 -4.6% 1.05x
NSStringConversion 3119 3000 -3.8% 1.04x
Chars 5365 5149 -4.0% 1.04x
StringWithCString 154407 150053 -2.8% 1.03x(?)
ObjectiveCBridgeStubFromNSStringRef 172 167 -2.9% 1.03x(?)
ObjectiveCBridgeStubFromNSString 821 799 -2.7% 1.03x
NSError 698 680 -2.6% 1.03x(?)
ObjectiveCBridgeFromNSArrayAnyObjectToStringForced 96264 93609 -2.8% 1.03x
SortStrings 2747 2694 -1.9% 1.02x(?)
ObjectiveCBridgeStubToNSString 1323 1298 -1.9% 1.02x
TypeFlood 197 193 -2.0% 1.02x(?)
ObjectiveCBridgeStubToArrayOfNSString 29769 29138 -2.1% 1.02x
ObjectiveCBridgeStubDataAppend 3432 3359 -2.1% 1.02x
SetExclusiveOr_OfObjects 45350 44626 -1.6% 1.02x(?)
SetUnion_OfObjects 33064 32571 -1.5% 1.02x(?)
RecursiveOwnedParameter 10376 10281 -0.9% 1.01x(?)
ObjectiveCBridgeToNSArray 30905 30747 -0.5% 1.01x(?)
PopFrontArrayGeneric 9755 9702 -0.5% 1.01x
AngryPhonebook 2982 2967 -0.5% 1.01x
OpenClose 450 447 -0.7% 1.01x(?)
ObjectiveCBridgeStubNSDateRefAccess 1296 1279 -1.3% 1.01x
StringBuilder 2823 2782 -1.4% 1.01x
SevenBoom 1546 1536 -0.7% 1.01x(?)
ArraySubscript 5508 5496 -0.2% 1.00x(?)
ObjectiveCBridgeToNSString 1100 1100 +0.0% 1.00x
Integrate 367 366 -0.3% 1.00x
Histogram 11536 11559 +0.2% 1.00x(?)
MonteCarloPi 50493 50546 +0.1% 1.00x(?)
Prims 13148 13121 -0.2% 1.00x(?)
ObjectiveCBridgeStubToNSDate 13709 13747 +0.3% 1.00x(?)
XorLoop 18893 18894 +0.0% 1.00x(?)
StringInterpolation 15788 15808 +0.1% 1.00x(?)
StaticArray 35891 35991 +0.3% 1.00x(?)
ProtocolDispatch2 440 441 +0.2% 1.00x(?)
Walsh 12590 12584 -0.1% 1.00x(?)
Dictionary2 4363 4377 +0.3% 1.00x(?)
ErrorHandling 3789 3800 +0.3% 1.00x(?)
ObjectiveCBridgeStubToNSDateRef 3352 3341 -0.3% 1.00x(?)
PopFrontUnsafePointer 258686 258911 +0.1% 1.00x(?)
CaptureProp 118483 117901 -0.5% 1.00x
ObjectiveCBridgeStubToNSStringRef 159 159 +0.0% 1.00x
MapReduce 42890 42847 -0.1% 1.00x(?)
DictionaryRemoveOfObjects 45859 45877 +0.0% 1.00x(?)
UTF8Decode 41062 41140 +0.2% 1.00x(?)
Dictionary 2140 2150 +0.5% 1.00x(?)
NopDeinit 59826 59830 +0.0% 1.00x(?)
SetIntersect 14766 14771 +0.0% 1.00x(?)
GlobalClass 0 0 +0.0% 1.00x
ArrayInClass 3952 3958 +0.1% 1.00x
Sim2DArray 14333 14343 +0.1% 1.00x
ObjectiveCBridgeToNSDictionary 16442 16380 -0.4% 1.00x(?)
HashTest 5228 5243 +0.3% 1.00x(?)
LinkedList 26444 26349 -0.4% 1.00x(?)
RGBHistogramOfObjects 91201 91249 +0.1% 1.00x(?)
RGBHistogram 43082 43275 +0.5% 1.00x(?)
ObjectiveCBridgeFromNSSetAnyObjectForced 7593 7627 +0.5% 1.00x(?)
ArrayValueProp4 3004 3018 +0.5% 1.00x(?)
SetUnion 17946 17916 -0.2% 1.00x(?)
DeadArray 121850 121968 +0.1% 1.00x(?)
ArrayValueProp2 3135 3145 +0.3% 1.00x(?)
AnyHashableWithAClass 76836 76819 -0.0% 1.00x(?)
ArrayValueProp 2539 2534 -0.2% 1.00x(?)
Memset 19793 19814 +0.1% 1.00x
Dictionary2OfObjects 6241 6253 +0.2% 1.00x(?)
ArrayValueProp3 3058 3066 +0.3% 1.00x(?)
DictionarySwapOfObjects 21141 21249 +0.5% 0.99x(?)
StackPromo 126228 127392 +0.9% 0.99x(?)
135 5597808 5648062 +0.9% 0.99x
DictionaryBridge 3989 4023 +0.8% 0.99x(?)
ObjectiveCBridgeFromNSDictionaryAnyObjectForced 7950 8000 +0.6% 0.99x(?)
ObjectiveCBridgeFromNSArrayAnyObjectForced 10234 10351 +1.1% 0.99x(?)
ObjectiveCBridgeFromNSStringForced 3107 3134 +0.9% 0.99x(?)
RangeAssignment 27103 27300 +0.7% 0.99x(?)
SuperChars 253366 254860 +0.6% 0.99x(?)
ObjectAllocation 580 584 +0.7% 0.99x(?)
ObjectiveCBridgeFromNSSetAnyObject 271069 274516 +1.3% 0.99x(?)
SetIntersect_OfObjects 13820 14010 +1.4% 0.99x
ArrayOfRef 9576 9664 +0.9% 0.99x(?)
ObjectiveCBridgeStubURLAppendPathRef 242080 244984 +1.2% 0.99x
DictionaryOfObjects 4733 4798 +1.4% 0.99x(?)
IterateData 10618 10679 +0.6% 0.99x(?)
DictionaryLiteral 16714 16943 +1.4% 0.99x
Hanoi 20153 20374 +1.1% 0.99x(?)
SortStringsUnicode 9277 9330 +0.6% 0.99x(?)
SetExclusiveOr 29023 29348 +1.1% 0.99x(?)
StringHasSuffix 1539 1560 +1.4% 0.99x(?)
Phonebook 63163 63845 +1.1% 0.99x(?)
ObjectiveCBridgeStubNSDateMutationRef 14466 14620 +1.1% 0.99x(?)
StringHasSuffixUnicode 64633 65058 +0.7% 0.99x(?)
SetIsSubsetOf_OfObjects 2162 2176 +0.7% 0.99x(?)
ObjectiveCBridgeToNSSet 16540 16641 +0.6% 0.99x(?)
RC4 9514 9563 +0.5% 0.99x(?)
TwoSum 6353 6405 +0.8% 0.99x(?)
ObjectiveCBridgeStubNSDataAppend 2892 2919 +0.9% 0.99x(?)
ObjectiveCBridgeStubFromNSDate 3884 3920 +0.9% 0.99x(?)
Array2D 793665 811658 +2.3% 0.98x(?)
DictionarySwap 7432 7593 +2.2% 0.98x(?)
ObjectiveCBridgeFromNSDictionaryAnyObject 213166 218068 +2.3% 0.98x(?)
ProtocolDispatch 6160 6255 +1.5% 0.98x
StringHasPrefix 1714 1755 +2.4% 0.98x
StringEqualPointerComparison 9506 9672 +1.8% 0.98x
PolymorphicCalls 1137 1155 +1.6% 0.98x(?)
ObjectiveCBridgeStubURLAppendPath 236963 242164 +2.2% 0.98x(?)
ObjectiveCBridgeStubDateAccess 1101 1124 +2.1% 0.98x
MonteCarloE 102938 105577 +2.6% 0.98x
DictionaryRemove 17201 17474 +1.6% 0.98x(?)
StringHasPrefixUnicode 15254 15490 +1.6% 0.98x(?)
PopFrontArray 24069 24824 +3.1% 0.97x(?)
ObjectiveCBridgeFromNSArrayAnyObjectToString 124128 128493 +3.5% 0.97x
ClassArrayGetter 1269 1303 +2.7% 0.97x(?)
SortLettersInPlace 2506 2581 +3.0% 0.97x
Dictionary3OfObjects 2196 2260 +2.9% 0.97x
Join 1444 1484 +2.8% 0.97x(?)
ArrayLiteral 1215 1248 +2.7% 0.97x
Dictionary3 1556 1604 +3.1% 0.97x
Calculator 920 948 +3.0% 0.97x
SetIsSubsetOf 2380 2450 +2.9% 0.97x
ObjectiveCBridgeFromNSArrayAnyObject 79473 81817 +3.0% 0.97x
ArrayOfGenericRef 9931 10202 +2.7% 0.97x
BitCount 96 99 +3.1% 0.97x
ObjectiveCBridgeStubFromArrayOfNSString 60983 62724 +2.9% 0.97x(?)
NSDictionaryCastToSwift 8079 8439 +4.5% 0.96x
ObjectiveCBridgeFromNSSetAnyObjectToString 166119 172445 +3.8% 0.96x
ObjectiveCBridgeFromNSString 5329 5548 +4.1% 0.96x
ArrayAppendReserved 3148 3263 +3.6% 0.96x
ObjectiveCBridgeFromNSDictionaryAnyObjectToStringForced 139826 147179 +5.3% 0.95x
ArrayOfPOD 2272 2386 +5.0% 0.95x
**Hardware Overview** Model Name: Mac mini Model Identifier: Macmini7,1 Processor Name: Intel Core i7 Processor Speed: 3 GHz Number of Processors: 1 Total Number of Cores: 2 L2 Cache (per Core): 256 KB L3 Cache: 4 MB Memory: 16 GB
@natecook1000

This comment has been minimized.

Copy link
Member Author

commented Sep 23, 2016

@moiseev Tests are passing, can you review?

@moiseev

This comment has been minimized.

Copy link
Member

commented Sep 23, 2016

The change LGTM. Regressions worry me a bit. Let's run benchmarks again and see.
@swift-ci Please benchmark.

@swift-ci

This comment has been minimized.

Copy link
Contributor

commented Sep 23, 2016

Build comment file:

Optimized (O)

Regression (4)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
ByteSwap 0 1 +100000.0% 0.00x
ClassArrayGetter 12 13 +8.3% 0.92x
Calculator 34 37 +8.8% 0.92x
StrToInt 5172 5640 +9.1% 0.92x

Improvement (1)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
ObjectiveCBridgeToNSSet 16694 15750 -5.7% 1.06x(?)

No Changes (131)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
StringHasSuffix 756 721 -4.6% 1.05x
PolymorphicCalls 21 20 -4.8% 1.05x
ObjectiveCBridgeToNSDictionary 16803 16047 -4.5% 1.05x
ArrayOfGenericPOD 214 208 -2.8% 1.03x
StringHasPrefix 608 591 -2.8% 1.03x
StaticArray 2858 2771 -3.0% 1.03x(?)
ObjectiveCBridgeStubDateMutation 273 265 -2.9% 1.03x
StringHasPrefixUnicode 13873 13505 -2.6% 1.03x
ArrayOfPOD 182 176 -3.3% 1.03x
ObjectiveCBridgeStubFromNSDate 3696 3598 -2.6% 1.03x
ObjectiveCBridgeFromNSArrayAnyObjectToString 124992 123099 -1.5% 1.02x(?)
DictionarySwapOfObjects 6781 6668 -1.7% 1.02x(?)
ObjectiveCBridgeFromNSStringForced 2739 2685 -2.0% 1.02x(?)
ArrayOfGenericRef 3636 3561 -2.1% 1.02x(?)
ArrayLiteral 1183 1165 -1.5% 1.02x
DeadArray 185 182 -1.6% 1.02x
ObjectiveCBridgeFromNSDictionaryAnyObjectToString 211080 209314 -0.8% 1.01x(?)
DictionarySwap 614 609 -0.8% 1.01x
ErrorHandling 2961 2929 -1.1% 1.01x(?)
StringHasSuffixUnicode 63416 62749 -1.1% 1.01x(?)
AnyHashableWithAClass 63548 62741 -1.3% 1.01x
HashTest 1718 1697 -1.2% 1.01x(?)
ArrayOfRef 3577 3548 -0.8% 1.01x
ObjectiveCBridgeStubToNSStringRef 120 119 -0.8% 1.01x(?)
Hanoi 3597 3576 -0.6% 1.01x
ObjectiveCBridgeStubURLAppendPath 234747 231890 -1.2% 1.01x(?)
SortStringsUnicode 8297 8250 -0.6% 1.01x(?)
Dictionary 770 759 -1.4% 1.01x(?)
SetExclusiveOr 2550 2533 -0.7% 1.01x(?)
Phonebook 7516 7477 -0.5% 1.01x(?)
DictionaryRemove 4623 4598 -0.5% 1.01x(?)
SetUnion 2882 2840 -1.5% 1.01x(?)
ObjectiveCBridgeStubFromArrayOfNSString 58263 57592 -1.1% 1.01x(?)
Dictionary2OfObjects 3425 3406 -0.6% 1.01x
ArraySubscript 1425 1424 -0.1% 1.00x(?)
ObjectiveCBridgeToNSString 1073 1068 -0.5% 1.00x(?)
MonteCarloPi 42526 42526 +0.0% 1.00x
StackPromo 21301 21315 +0.1% 1.00x(?)
135 3002024 2994561 -0.2% 1.00x
Integrate 239 238 -0.4% 1.00x
Array2D 2071 2078 +0.3% 1.00x(?)
Histogram 636 636 +0.0% 1.00x
DictionaryBridge 3892 3889 -0.1% 1.00x(?)
StringWithCString 145936 146461 +0.4% 1.00x(?)
PopFrontArray 1093 1098 +0.5% 1.00x(?)
Dictionary3OfObjects 857 856 -0.1% 1.00x(?)
XorLoop 379 379 +0.0% 1.00x
ObjectiveCBridgeStubToNSString 1244 1248 +0.3% 1.00x(?)
ProtocolDispatch 3039 3038 -0.0% 1.00x
ObjectAllocation 157 157 +0.0% 1.00x
TypeFlood 0 0 +0.0% 1.00x
ObjectiveCBridgeFromNSSetAnyObject 266832 266047 -0.3% 1.00x(?)
Dictionary3 508 507 -0.2% 1.00x(?)
Dictionary2 2035 2030 -0.2% 1.00x(?)
SetIntersect_OfObjects 2037 2037 +0.0% 1.00x
Join 459 460 +0.2% 1.00x(?)
DictionaryOfObjects 2238 2234 -0.2% 1.00x(?)
PopFrontArrayGeneric 1099 1096 -0.3% 1.00x
StringEqualPointerComparison 7331 7348 +0.2% 1.00x
CaptureProp 4513 4501 -0.3% 1.00x(?)
DictionaryRemoveOfObjects 19226 19254 +0.1% 1.00x(?)
UTF8Decode 281 281 +0.0% 1.00x
ObjectiveCBridgeFromNSDictionaryAnyObjectToStringForced 137411 137269 -0.1% 1.00x(?)
SetIsSubsetOf 476 476 +0.0% 1.00x
ObjectiveCBridgeFromNSArrayAnyObject 76172 75883 -0.4% 1.00x(?)
SetIntersect 1041 1039 -0.2% 1.00x(?)
NSDictionaryCastToSwift 6840 6845 +0.1% 1.00x(?)
RGBHistogramOfObjects 21418 21363 -0.3% 1.00x(?)
ObjectiveCBridgeStubNSDateRefAccess 338 338 +0.0% 1.00x
StringBuilder 1339 1334 -0.4% 1.00x(?)
ObjectiveCBridgeStubDateAccess 182 182 +0.0% 1.00x
MonteCarloE 9993 10001 +0.1% 1.00x(?)
RC4 165 165 +0.0% 1.00x
SetIsSubsetOf_OfObjects 485 486 +0.2% 1.00x
ArrayAppend 740 739 -0.1% 1.00x(?)
ObjectiveCBridgeFromNSSetAnyObjectToStringForced 111541 111038 -0.5% 1.00x(?)
RGBHistogram 2813 2818 +0.2% 1.00x(?)
ArrayValueProp4 5 5 +0.0% 1.00x
ArrayValueProp2 5 5 +0.0% 1.00x
BitCount 1 1 +0.0% 1.00x
AngryPhonebook 2841 2847 +0.2% 1.00x(?)
ArrayValueProp3 5 5 +0.0% 1.00x
StringWalk 5677 5657 -0.3% 1.00x(?)
ArrayValueProp 5 5 +0.0% 1.00x
GlobalClass 0 0 +0.0% 1.00x
Memset 222 222 +0.0% 1.00x
ArrayAppendReserved 519 519 +0.0% 1.00x
ObjectiveCBridgeStubNSDataAppend 2443 2452 +0.4% 1.00x(?)
ObjectiveCBridgeFromNSDictionaryAnyObject 204077 205905 +0.9% 0.99x(?)
RecursiveOwnedParameter 1864 1875 +0.6% 0.99x(?)
ObjectiveCBridgeFromNSDictionaryAnyObjectForced 5154 5203 +0.9% 0.99x(?)
RangeAssignment 277 281 +1.4% 0.99x(?)
StrComplexWalk 2889 2927 +1.3% 0.99x(?)
ObjectiveCBridgeToNSArray 30669 31025 +1.2% 0.99x(?)
NSError 317 319 +0.6% 0.99x(?)
ObjectiveCBridgeStubToNSDateRef 3286 3303 +0.5% 0.99x(?)
LinkedList 6807 6904 +1.4% 0.99x(?)
NSStringConversion 811 816 +0.6% 0.99x
ObjectiveCBridgeFromNSSetAnyObjectForced 4535 4577 +0.9% 0.99x(?)
SortStrings 1756 1785 +1.6% 0.98x
ObjectiveCBridgeFromNSArrayAnyObjectForced 5999 6092 +1.6% 0.98x(?)
SuperChars 201894 206045 +2.1% 0.98x
ObjectiveCBridgeStubToNSDate 13454 13663 +1.6% 0.98x(?)
StringInterpolation 10971 11176 +1.9% 0.98x(?)
ProtocolDispatch2 160 163 +1.9% 0.98x
MapReduce 361 369 +2.2% 0.98x(?)
DictionaryLiteral 2752 2796 +1.6% 0.98x(?)
OpenClose 47 48 +2.1% 0.98x
NopDeinit 37654 38517 +2.3% 0.98x
ObjectiveCBridgeStubToArrayOfNSString 28911 29442 +1.8% 0.98x(?)
ObjectiveCBridgeStubDataAppend 2972 3039 +2.2% 0.98x(?)
ArrayInClass 65 66 +1.5% 0.98x(?)
ObjectiveCBridgeFromNSString 1793 1834 +2.3% 0.98x
Sim2DArray 272 277 +1.8% 0.98x(?)
SetExclusiveOr_OfObjects 7516 7682 +2.2% 0.98x
SetUnion_OfObjects 6138 6292 +2.5% 0.98x
Chars 616 629 +2.1% 0.98x
TwoSum 1556 1585 +1.9% 0.98x(?)
Prims 695 717 +3.2% 0.97x
ObjectiveCBridgeStubFromNSStringRef 134 138 +3.0% 0.97x
ObjectiveCBridgeStubFromNSString 774 795 +2.7% 0.97x
ObjectiveCBridgeStubURLAppendPathRef 237399 244970 +3.2% 0.97x(?)
PopFrontUnsafePointer 8988 9241 +2.8% 0.97x(?)
ObjectiveCBridgeFromNSSetAnyObjectToString 143067 147919 +3.4% 0.97x(?)
ObjectiveCBridgeFromNSArrayAnyObjectToStringForced 91068 93768 +3.0% 0.97x(?)
ObjectiveCBridgeStubNSDateMutationRef 12213 12654 +3.6% 0.97x
SevenBoom 1331 1367 +2.7% 0.97x(?)
ObjectiveCBridgeStubFromNSDateRef 3682 3831 +4.0% 0.96x
IterateData 2481 2587 +4.3% 0.96x
SortLettersInPlace 929 973 +4.7% 0.95x
Walsh 305 320 +4.9% 0.95x
**Unoptimized (Onone)**

Regression (5)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
TypeFlood 184 216 +17.4% 0.85x(?)
StrToInt 5306 6142 +15.8% 0.86x
ArrayOfGenericPOD 3431 3692 +7.6% 0.93x
ObjectiveCBridgeStubDateMutation 469 499 +6.4% 0.94x
ObjectiveCBridgeStubToArrayOfNSString 28746 30661 +6.7% 0.94x

Improvement (1)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
StringHasPrefixUnicode 16394 15297 -6.7% 1.07x

No Changes (130)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
ObjectiveCBridgeFromNSDictionaryAnyObjectToString 230462 219234 -4.9% 1.05x
StrComplexWalk 7997 7584 -5.2% 1.05x
Calculator 969 922 -4.8% 1.05x
StringBuilder 2862 2737 -4.4% 1.05x(?)
ObjectiveCBridgeStubDateAccess 1128 1073 -4.9% 1.05x
ObjectiveCBridgeStubNSDataAppend 2855 2731 -4.3% 1.05x
DictionaryBridge 3902 3761 -3.6% 1.04x(?)
BitCount 100 96 -4.0% 1.04x
Prims 13522 13119 -3.0% 1.03x
ObjectiveCBridgeStubToNSDateRef 3376 3271 -3.1% 1.03x
ObjectiveCBridgeFromNSSetAnyObjectForced 7620 7382 -3.1% 1.03x(?)
ObjectiveCBridgeStubFromArrayOfNSString 62584 60849 -2.8% 1.03x(?)
Histogram 11770 11504 -2.3% 1.02x
Dictionary3OfObjects 2224 2184 -1.8% 1.02x(?)
StringInterpolation 16597 16316 -1.7% 1.02x
StringHasSuffix 1628 1589 -2.4% 1.02x
ObjectiveCBridgeStubToNSString 1318 1293 -1.9% 1.02x
ObjectiveCBridgeStubURLAppendPathRef 247259 243424 -1.6% 1.02x(?)
OpenClose 466 456 -2.1% 1.02x
SortStringsUnicode 9418 9251 -1.8% 1.02x
ObjectiveCBridgeFromNSDictionaryAnyObjectToStringForced 143799 141564 -1.6% 1.02x(?)
ObjectiveCBridgeFromNSArrayAnyObject 80317 78511 -2.2% 1.02x(?)
ObjectiveCBridgeStubNSDateRefAccess 1259 1232 -2.1% 1.02x
Sim2DArray 14543 14325 -1.5% 1.02x
StackPromo 131611 130133 -1.1% 1.01x(?)
RecursiveOwnedParameter 10350 10297 -0.5% 1.01x(?)
Integrate 371 369 -0.5% 1.01x(?)
Array2D 938113 927223 -1.2% 1.01x(?)
SortStrings 2704 2688 -0.6% 1.01x(?)
ObjectiveCBridgeFromNSStringForced 3074 3051 -0.8% 1.01x(?)
DictionarySwap 7401 7327 -1.0% 1.01x(?)
PopFrontArray 24474 24182 -1.2% 1.01x(?)
ObjectiveCBridgeStubToNSDate 14483 14313 -1.2% 1.01x(?)
ArrayLiteral 1214 1204 -0.8% 1.01x(?)
ObjectiveCBridgeToNSSet 16757 16532 -1.3% 1.01x(?)
CaptureProp 115993 114656 -1.1% 1.01x(?)
ObjectiveCBridgeStubToNSStringRef 159 158 -0.6% 1.01x(?)
AngryPhonebook 3001 2959 -1.4% 1.01x
IterateData 10751 10662 -0.8% 1.01x(?)
ObjectiveCBridgeStubURLAppendPath 238763 236380 -1.0% 1.01x
NSDictionaryCastToSwift 8229 8154 -0.9% 1.01x(?)
ArrayAppend 3440 3417 -0.7% 1.01x(?)
ObjectiveCBridgeFromNSSetAnyObjectToStringForced 126462 124676 -1.4% 1.01x(?)
DeadArray 122105 121056 -0.9% 1.01x(?)
AnyHashableWithAClass 77378 76705 -0.9% 1.01x(?)
TwoSum 6595 6545 -0.8% 1.01x(?)
ObjectiveCBridgeToNSString 1102 1098 -0.4% 1.00x(?)
DictionarySwapOfObjects 21058 21029 -0.1% 1.00x(?)
ObjectiveCBridgeFromNSDictionaryAnyObject 210450 210743 +0.1% 1.00x(?)
135 5742047 5766246 +0.4% 1.00x
ClassArrayGetter 1266 1262 -0.3% 1.00x(?)
MonteCarloPi 50631 50632 +0.0% 1.00x(?)
StringWithCString 154760 154594 -0.1% 1.00x(?)
ObjectiveCBridgeFromNSArrayAnyObjectForced 10598 10627 +0.3% 1.00x(?)
RangeAssignment 27045 27171 +0.5% 1.00x(?)
ByteSwap 9 9 +0.0% 1.00x
XorLoop 18925 18887 -0.2% 1.00x(?)
ProtocolDispatch 5484 5496 +0.2% 1.00x(?)
ObjectiveCBridgeFromNSSetAnyObject 274713 274563 -0.1% 1.00x(?)
ProtocolDispatch2 445 445 +0.0% 1.00x
Dictionary3 1601 1600 -0.1% 1.00x(?)
Dictionary2 4408 4411 +0.1% 1.00x(?)
SetIntersect_OfObjects 13972 13966 -0.0% 1.00x(?)
PopFrontUnsafePointer 258674 258267 -0.2% 1.00x(?)
StringEqualPointerComparison 9711 9672 -0.4% 1.00x(?)
DictionaryRemoveOfObjects 45502 45633 +0.3% 1.00x(?)
SetIsSubsetOf 2497 2493 -0.2% 1.00x(?)
Dictionary 2145 2149 +0.2% 1.00x
ObjectiveCBridgeStubDataAppend 3398 3390 -0.2% 1.00x(?)
SetExclusiveOr 29171 29201 +0.1% 1.00x(?)
GlobalClass 0 0 +0.0% 1.00x
ArrayInClass 3952 3950 -0.1% 1.00x(?)
ArrayOfGenericRef 10053 10043 -0.1% 1.00x(?)
Phonebook 64073 64201 +0.2% 1.00x(?)
SetExclusiveOr_OfObjects 45287 45347 +0.1% 1.00x(?)
ObjectiveCBridgeToNSDictionary 16568 16570 +0.0% 1.00x(?)
MonteCarloE 106896 106451 -0.4% 1.00x(?)
SetUnion_OfObjects 33255 33342 +0.3% 1.00x
StringHasSuffixUnicode 64759 64827 +0.1% 1.00x(?)
HashTest 5243 5235 -0.1% 1.00x(?)
DictionaryRemove 17313 17296 -0.1% 1.00x(?)
LinkedList 26441 26382 -0.2% 1.00x(?)
NSStringConversion 2971 2979 +0.3% 1.00x(?)
ArrayAppendReserved 3255 3248 -0.2% 1.00x(?)
SetUnion 17845 17857 +0.1% 1.00x(?)
ObjectiveCBridgeStubNSDateMutationRef 14663 14592 -0.5% 1.00x(?)
RC4 9373 9375 +0.0% 1.00x(?)
Memset 19794 19803 +0.1% 1.00x(?)
Dictionary2OfObjects 6216 6221 +0.1% 1.00x(?)
ArrayValueProp4 3004 3007 +0.1% 1.00x(?)
ObjectiveCBridgeStubFromNSDate 3986 3974 -0.3% 1.00x
ArraySubscript 6261 6301 +0.6% 0.99x(?)
ObjectiveCBridgeFromNSDictionaryAnyObjectForced 7467 7560 +1.2% 0.99x(?)
ObjectiveCBridgeStubFromNSDateRef 4200 4226 +0.6% 0.99x
SortLettersInPlace 2514 2531 +0.7% 0.99x(?)
SuperChars 260454 262543 +0.8% 0.99x(?)
ObjectAllocation 568 575 +1.2% 0.99x(?)
StringHasPrefix 1594 1603 +0.6% 0.99x
Join 1462 1480 +1.2% 0.99x(?)
ArrayOfRef 9364 9416 +0.6% 0.99x
ObjectiveCBridgeToNSArray 30352 30620 +0.9% 0.99x(?)
PolymorphicCalls 1147 1162 +1.3% 0.99x
MapReduce 43081 43437 +0.8% 0.99x
Hanoi 20072 20199 +0.6% 0.99x
UTF8Decode 41243 41503 +0.6% 0.99x(?)
ObjectiveCBridgeFromNSString 5235 5267 +0.6% 0.99x(?)
SetIsSubsetOf_OfObjects 2153 2172 +0.9% 0.99x(?)
ArrayValueProp3 3033 3065 +1.1% 0.99x(?)
ArrayValueProp 2506 2534 +1.1% 0.99x(?)
SevenBoom 1484 1492 +0.5% 0.99x
ObjectiveCBridgeFromNSArrayAnyObjectToString 127787 130894 +2.4% 0.98x
Walsh 12726 12921 +1.5% 0.98x
DictionaryOfObjects 4761 4845 +1.8% 0.98x(?)
PopFrontArrayGeneric 9623 9815 +2.0% 0.98x
NopDeinit 61582 63008 +2.3% 0.98x
SetIntersect 14744 15061 +2.1% 0.98x
ArrayOfPOD 2387 2447 +2.5% 0.98x
Chars 5147 5254 +2.1% 0.98x
ErrorHandling 3727 3841 +3.1% 0.97x(?)
ObjectiveCBridgeStubFromNSStringRef 167 173 +3.6% 0.97x
ObjectiveCBridgeStubFromNSString 804 828 +3.0% 0.97x
StaticArray 34893 36004 +3.2% 0.97x
DictionaryLiteral 16461 16905 +2.7% 0.97x(?)
RGBHistogramOfObjects 91061 93426 +2.6% 0.97x(?)
RGBHistogram 43052 44221 +2.7% 0.97x
ObjectiveCBridgeFromNSArrayAnyObjectToStringForced 94106 96942 +3.0% 0.97x(?)
StringWalk 21402 22081 +3.2% 0.97x
ArrayValueProp2 3028 3113 +2.8% 0.97x
NSError 673 698 +3.7% 0.96x(?)
ObjectiveCBridgeFromNSSetAnyObjectToString 157909 164398 +4.1% 0.96x(?)
**Hardware Overview** Model Name: Mac mini Model Identifier: Macmini7,1 Processor Name: Intel Core i7 Processor Speed: 3 GHz Number of Processors: 1 Total Number of Cores: 2 L2 Cache (per Core): 256 KB L3 Cache: 4 MB Memory: 16 GB
@gottesmm

This comment has been minimized.

Copy link
Member

commented Sep 24, 2016

@moiseev Just an FYI, I wouldn't trust the benchmarking results you have. It is only doing 3 samples which is not enough. I am going to change the bot real quick to use 20 samples like we do on our main benchmarking bot.

In the future, we should have a system where we do 3-5 samples and then anything with > 5% change is rerun with 20 samples or until a statistical model is satisfied.

@gottesmm

This comment has been minimized.

Copy link
Member

commented Sep 24, 2016

I have a patch that will allow me to make such a change in #4994

@swift-ci

This comment has been minimized.

Copy link
Contributor

commented Sep 27, 2016

Build comment file:

Build failed before running benchmark.


@natecook1000

This comment has been minimized.

Copy link
Member Author

commented Sep 27, 2016

I can't get the full benchmarks to run on their own (i.e., using the --benchmark option on build-script) to get a proper comparison, but running just the regressing StrToInt benchmark locally gives me better performance on the new branch. Here's the output of three Benchmark_O runs with --num-iters=20 --num-samples=100 for each branch:

master branch:

#,TEST,SAMPLES,MIN(us),MAX(us),MEAN(us),SD(us),MEDIAN(us)
120,StrToInt,100,4747,5447,4954,162,4940
120,StrToInt,100,4724,5366,4899,150,4821
120,StrToInt,100,4757,6156,5182,402,5065

nc-stringdex branch:

#,TEST,SAMPLES,MIN(us),MAX(us),MEAN(us),SD(us),MEDIAN(us)
120,StrToInt,100,3829,4347,3972,132,3901
120,StrToInt,100,3838,4364,3962,121,3904
120,StrToInt,100,3858,4477,3993,139,3926
@moiseev

This comment has been minimized.

Copy link
Member

commented Sep 28, 2016

@swift-ci Please benchmark

@swift-ci

This comment has been minimized.

Copy link
Contributor

commented Sep 28, 2016

Build comment file:

Build failed before running benchmark.


@shahmishal

This comment has been minimized.

Copy link
Member

commented Sep 28, 2016

its failing because of --no-assertions flag. https://ci.swift.org/job/oss-swift_tools-R_stdlib-RD_test-simulator/

@gottesmm

This comment has been minimized.

Copy link
Member

commented Sep 29, 2016

@natecook1000 Don't use the iters flag. The way to think about the benchmarking script is that iters decides the number of iterations per sample. You should just be modifying the number of samples. The only time you want to modify the number of iterations is if you want to get one sample to run really long in instruments.

That being said, there are a couple of things that could be going on here your machine (or the benchmarking machine for that matter) could be down clocking or could be noisy in certain manners due to other processes. I would say lets run the benchmarks one more time just out of paranoia. If we don't get the same thing, lets just commit and if we get a regression on our internal system we can look into it.

At some point I imagine we will start tracking performance externally which will make this sort of thing easier.

@natecook1000

This comment has been minimized.

Copy link
Member Author

commented Sep 29, 2016

Thanks @gottesmm—same results again for me, let's see if CI can benchmark again…

@swift-ci Please benchmark

@swift-ci

This comment has been minimized.

Copy link
Contributor

commented Sep 29, 2016

Build comment file:

Build failed before running benchmark.


@shahmishal

This comment has been minimized.

Copy link
Member

commented Sep 29, 2016

its still broken due to rdar://28485525, --no-assertions flag

@natecook1000

This comment has been minimized.

Copy link
Member Author

commented Oct 5, 2016

@swift-ci Please benchmark

@swift-ci

This comment has been minimized.

Copy link
Contributor

commented Oct 5, 2016

Build comment file:

Build failed before running benchmark.


@shahmishal

This comment has been minimized.

Copy link
Member

commented Oct 5, 2016

@natecook1000 --no-assertions flag is still broken

@shahmishal

This comment has been minimized.

Copy link
Member

commented Oct 6, 2016

This PR should fix the no-assertions flag issue,
#5137

@shahmishal

This comment has been minimized.

Copy link
Member

commented Oct 6, 2016

@swift-ci Please benchmark

@swift-ci

This comment has been minimized.

Copy link
Contributor

commented Oct 6, 2016

Build comment file:

Optimized (O)

Regression (4)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
StringWithCString 149561 202012 +35.1% 0.74x
StringHasSuffix 716 839 +17.2% 0.85x
StringHasPrefix 608 701 +15.3% 0.87x
ClassArrayGetter 12 13 +8.3% 0.92x(?)

Improvement (0)

No Changes (132)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
SortStringsUnicode 8460 8177 -3.4% 1.03x
StringHasPrefixUnicode 14034 13574 -3.3% 1.03x
ObjectiveCBridgeStubFromNSDate 3668 3563 -2.9% 1.03x(?)
PopFrontArrayGeneric 1116 1089 -2.4% 1.02x(?)
IterateData 2597 2536 -2.4% 1.02x
SetIsSubsetOf 508 499 -1.8% 1.02x
Dictionary 748 736 -1.6% 1.02x
ObjectiveCBridgeFromNSString 1813 1770 -2.4% 1.02x
ObjectiveCBridgeStubNSDataAppend 2431 2394 -1.5% 1.02x(?)
RecursiveOwnedParameter 1942 1931 -0.6% 1.01x
ObjectiveCBridgeFromNSDictionaryAnyObjectToString 211759 210435 -0.6% 1.01x(?)
ObjectiveCBridgeFromNSStringForced 2690 2661 -1.1% 1.01x(?)
ObjectiveCBridgeFromNSDictionaryAnyObject 202577 200683 -0.9% 1.01x(?)
RangeAssignment 285 283 -0.7% 1.01x(?)
ErrorHandling 2950 2929 -0.7% 1.01x(?)
StaticArray 2751 2717 -1.2% 1.01x(?)
StringHasSuffixUnicode 63501 62830 -1.1% 1.01x
ProtocolDispatch2 159 157 -1.3% 1.01x
ObjectiveCBridgeStubURLAppendPathRef 232962 230908 -0.9% 1.01x(?)
ObjectiveCBridgeToNSSet 15714 15635 -0.5% 1.01x(?)
NSError 327 325 -0.6% 1.01x(?)
CaptureProp 4124 4100 -0.6% 1.01x(?)
Hanoi 3447 3428 -0.6% 1.01x(?)
ObjectiveCBridgeFromNSDictionaryAnyObjectToStringForced 134955 133430 -1.1% 1.01x(?)
SetExclusiveOr 2629 2603 -1.0% 1.01x(?)
NSDictionaryCastToSwift 6988 6950 -0.5% 1.01x(?)
ObjectiveCBridgeToNSDictionary 16227 16130 -0.6% 1.01x(?)
ObjectiveCBridgeFromNSSetAnyObjectToString 143863 143089 -0.5% 1.01x(?)
RGBHistogram 2838 2822 -0.6% 1.01x(?)
SetUnion 3003 2978 -0.8% 1.01x(?)
Chars 639 630 -1.4% 1.01x
ObjectiveCBridgeFromNSArrayAnyObjectToStringForced 90398 89670 -0.8% 1.01x(?)
ObjectiveCBridgeStubFromArrayOfNSString 58737 58189 -0.9% 1.01x(?)
ArraySubscript 1415 1417 +0.1% 1.00x(?)
DictionarySwapOfObjects 6779 6764 -0.2% 1.00x(?)
StackPromo 21578 21648 +0.3% 1.00x(?)
Integrate 238 238 +0.0% 1.00x
Array2D 2038 2032 -0.3% 1.00x(?)
Histogram 656 656 +0.0% 1.00x
SortStrings 1821 1818 -0.2% 1.00x(?)
DictionaryBridge 3825 3809 -0.4% 1.00x(?)
ObjectiveCBridgeFromNSDictionaryAnyObjectForced 5319 5309 -0.2% 1.00x(?)
ObjectiveCBridgeStubFromNSDateRef 3712 3712 +0.0% 1.00x
MonteCarloPi 45007 44988 -0.0% 1.00x(?)
ObjectiveCBridgeFromNSArrayAnyObjectForced 6049 6041 -0.1% 1.00x(?)
SortLettersInPlace 972 972 +0.0% 1.00x
DictionarySwap 631 631 +0.0% 1.00x
ArrayOfGenericPOD 220 220 +0.0% 1.00x
Dictionary3OfObjects 857 857 +0.0% 1.00x
ByteSwap 0 0 +0.0% 1.00x
SuperChars 210305 211208 +0.4% 1.00x(?)
ObjectiveCBridgeStubToNSDate 12922 12979 +0.4% 1.00x(?)
XorLoop 380 380 +0.0% 1.00x
StringInterpolation 10882 10917 +0.3% 1.00x(?)
ObjectiveCBridgeStubToNSString 1285 1281 -0.3% 1.00x(?)
ObjectiveCBridgeStubFromNSString 781 778 -0.4% 1.00x(?)
ProtocolDispatch 3040 3040 +0.0% 1.00x
TypeFlood 0 0 +0.0% 1.00x
ObjectiveCBridgeFromNSSetAnyObject 260751 259597 -0.4% 1.00x(?)
AnyHashableWithAClass 62827 62663 -0.3% 1.00x(?)
HashTest 1743 1745 +0.1% 1.00x(?)
StrComplexWalk 2907 2893 -0.5% 1.00x(?)
SetIntersect_OfObjects 2272 2271 -0.0% 1.00x(?)
ArrayOfRef 3548 3561 +0.4% 1.00x(?)
ObjectiveCBridgeStubDateMutation 273 273 +0.0% 1.00x
ObjectiveCBridgeToNSArray 30241 30390 +0.5% 1.00x(?)
DictionaryOfObjects 2282 2278 -0.2% 1.00x(?)
PopFrontUnsafePointer 9112 9136 +0.3% 1.00x(?)
ObjectiveCBridgeStubToNSStringRef 123 123 +0.0% 1.00x
MapReduce 342 342 +0.0% 1.00x
AngryPhonebook 2821 2815 -0.2% 1.00x(?)
DictionaryLiteral 2753 2740 -0.5% 1.00x(?)
OpenClose 48 48 +0.0% 1.00x
DictionaryRemoveOfObjects 19811 19837 +0.1% 1.00x(?)
UTF8Decode 288 288 +0.0% 1.00x
ObjectiveCBridgeStubURLAppendPath 233723 232646 -0.5% 1.00x(?)
NopDeinit 38154 38127 -0.1% 1.00x(?)
ObjectiveCBridgeStubToArrayOfNSString 29363 29432 +0.2% 1.00x(?)
ObjectiveCBridgeFromNSArrayAnyObject 75839 75729 -0.1% 1.00x(?)
SetIntersect 1084 1084 +0.0% 1.00x
ObjectiveCBridgeStubNSDateRefAccess 308 308 +0.0% 1.00x
ArrayInClass 63 63 +0.0% 1.00x
ObjectiveCBridgeStubDateAccess 182 182 +0.0% 1.00x
Sim2DArray 277 277 +0.0% 1.00x
MonteCarloE 10563 10554 -0.1% 1.00x(?)
RC4 165 165 +0.0% 1.00x
Walsh 309 309 +0.0% 1.00x
SetIsSubsetOf_OfObjects 563 563 +0.0% 1.00x
ArrayAppend 772 774 +0.3% 1.00x(?)
DictionaryRemove 4579 4565 -0.3% 1.00x(?)
LinkedList 7209 7209 +0.0% 1.00x
ObjectiveCBridgeFromNSSetAnyObjectForced 4587 4605 +0.4% 1.00x(?)
ArrayValueProp4 6 6 +0.0% 1.00x
ArrayOfPOD 182 182 +0.0% 1.00x
StringBuilder 1318 1320 +0.1% 1.00x(?)
BitCount 1 1 +0.0% 1.00x
ArrayLiteral 1176 1179 +0.3% 1.00x(?)
ArrayValueProp3 6 6 +0.0% 1.00x
StringWalk 5880 5879 -0.0% 1.00x(?)
ArrayValueProp 6 6 +0.0% 1.00x
ObjectiveCBridgeStubNSDateMutationRef 11985 11964 -0.2% 1.00x(?)
GlobalClass 0 0 +0.0% 1.00x
Memset 235 235 +0.0% 1.00x
Dictionary2OfObjects 3419 3405 -0.4% 1.00x(?)
ArrayAppendReserved 536 536 +0.0% 1.00x
TwoSum 1567 1563 -0.3% 1.00x(?)
ArrayValueProp2 6 6 +0.0% 1.00x
ObjectiveCBridgeToNSString 1065 1075 +0.9% 0.99x(?)
ObjectiveCBridgeFromNSArrayAnyObjectToString 116049 117813 +1.5% 0.99x(?)
Prims 714 721 +1.0% 0.99x(?)
ObjectAllocation 156 157 +0.6% 0.99x(?)
Dictionary3 509 513 +0.8% 0.99x(?)
Dictionary2 2023 2046 +1.1% 0.99x
Join 465 470 +1.1% 0.99x
ObjectiveCBridgeStubToNSDateRef 3373 3396 +0.7% 0.99x(?)
StringEqualPointerComparison 7297 7340 +0.6% 0.99x
ObjectiveCBridgeStubDataAppend 2954 2975 +0.7% 0.99x(?)
RGBHistogramOfObjects 21985 22128 +0.7% 0.99x(?)
ArrayOfGenericRef 3592 3613 +0.6% 0.99x(?)
Phonebook 7435 7499 +0.9% 0.99x
SetExclusiveOr_OfObjects 7683 7758 +1.0% 0.99x(?)
SetUnion_OfObjects 6468 6513 +0.7% 0.99x(?)
NSStringConversion 813 819 +0.7% 0.99x(?)
SevenBoom 1372 1380 +0.6% 0.99x(?)
PopFrontArray 1096 1119 +2.1% 0.98x(?)
135 3122324 3174467 +1.7% 0.98x
StrToInt 5081 5173 +1.8% 0.98x
ObjectiveCBridgeFromNSSetAnyObjectToStringForced 105613 107821 +2.1% 0.98x(?)
DeadArray 182 185 +1.6% 0.98x
ObjectiveCBridgeStubFromNSStringRef 135 139 +3.0% 0.97x
Calculator 35 36 +2.9% 0.97x(?)
PolymorphicCalls 21 22 +4.8% 0.95x
**Unoptimized (Onone)**

Regression (3)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
StrToInt 5246 5907 +12.6% 0.89x
ObjectiveCBridgeStubDateAccess 1065 1128 +5.9% 0.94x
StringWalk 21470 22939 +6.8% 0.94x(?)

Improvement (5)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
ArrayAppend 3671 3446 -6.1% 1.07x
PopFrontUnsafePointer 258728 242538 -6.3% 1.07x
ArrayAppendReserved 3422 3190 -6.8% 1.07x
ObjectiveCBridgeStubFromNSStringRef 181 168 -7.2% 1.08x
ObjectiveCBridgeStubDateMutation 547 486 -11.2% 1.13x

No Changes (128)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
ArrayOfGenericPOD 3609 3436 -4.8% 1.05x
StringHasSuffix 1792 1711 -4.5% 1.05x
SortStrings 2786 2707 -2.8% 1.03x
DictionaryBridge 3986 3855 -3.3% 1.03x(?)
SortStringsUnicode 9477 9227 -2.6% 1.03x
ObjectiveCBridgeFromNSString 5302 5153 -2.8% 1.03x(?)
StringHasPrefixUnicode 15767 15291 -3.0% 1.03x
ObjectiveCBridgeFromNSDictionaryAnyObjectToString 217348 212565 -2.2% 1.02x(?)
TypeFlood 191 187 -2.1% 1.02x(?)
CaptureProp 121336 118607 -2.2% 1.02x
ObjectiveCBridgeStubNSDataAppend 2856 2813 -1.5% 1.02x(?)
ObjectiveCBridgeStubFromNSDate 3932 3870 -1.6% 1.02x
135 5592341 5538894 -1.0% 1.01x
RecursiveOwnedParameter 10920 10841 -0.7% 1.01x(?)
Histogram 11809 11742 -0.6% 1.01x(?)
ObjectiveCBridgeFromNSDictionaryAnyObjectForced 7707 7658 -0.6% 1.01x(?)
ObjectiveCBridgeStubFromNSDateRef 4179 4143 -0.9% 1.01x
ObjectiveCBridgeFromNSArrayAnyObjectForced 10107 10049 -0.6% 1.01x(?)
ObjectiveCBridgeStubFromNSString 834 823 -1.3% 1.01x
PopFrontArrayGeneric 9638 9550 -0.9% 1.01x(?)
IterateData 10662 10558 -1.0% 1.01x(?)
DictionaryLiteral 16807 16589 -1.3% 1.01x(?)
ObjectiveCBridgeStubURLAppendPath 234986 231934 -1.3% 1.01x(?)
SetExclusiveOr 30504 30321 -0.6% 1.01x
NSDictionaryCastToSwift 8416 8322 -1.1% 1.01x(?)
ObjectiveCBridgeFromNSSetAnyObjectToString 158041 157171 -0.6% 1.01x(?)
NSStringConversion 3045 3003 -1.4% 1.01x(?)
ObjectiveCBridgeFromNSSetAnyObjectForced 7479 7409 -0.9% 1.01x(?)
ObjectiveCBridgeStubFromArrayOfNSString 58853 58404 -0.8% 1.01x(?)
TwoSum 6568 6515 -0.8% 1.01x(?)
ArraySubscript 5644 5662 +0.3% 1.00x(?)
DictionarySwapOfObjects 21058 21116 +0.3% 1.00x(?)
Integrate 365 365 +0.0% 1.00x
ClassArrayGetter 1266 1266 +0.0% 1.00x
Array2D 813998 815392 +0.2% 1.00x(?)
MonteCarloPi 53666 53681 +0.0% 1.00x(?)
StringWithCString 154754 154666 -0.1% 1.00x(?)
ObjectiveCBridgeFromNSStringForced 3103 3110 +0.2% 1.00x(?)
Prims 13558 13572 +0.1% 1.00x(?)
SortLettersInPlace 2601 2596 -0.2% 1.00x(?)
DictionarySwap 7524 7537 +0.2% 1.00x(?)
PopFrontArray 23830 23925 +0.4% 1.00x(?)
Dictionary3OfObjects 2259 2252 -0.3% 1.00x(?)
RangeAssignment 27521 27571 +0.2% 1.00x(?)
StrComplexWalk 7120 7129 +0.1% 1.00x(?)
ByteSwap 9 9 +0.0% 1.00x
SuperChars 261713 262623 +0.3% 1.00x(?)
XorLoop 19992 20001 +0.1% 1.00x(?)
StringInterpolation 15633 15671 +0.2% 1.00x(?)
Join 1463 1459 -0.3% 1.00x(?)
StaticArray 35175 35244 +0.2% 1.00x(?)
ObjectAllocation 566 566 +0.0% 1.00x
ObjectiveCBridgeFromNSSetAnyObject 270843 271068 +0.1% 1.00x(?)
ProtocolDispatch2 446 445 -0.2% 1.00x
HashTest 5366 5371 +0.1% 1.00x(?)
Dictionary3 1609 1607 -0.1% 1.00x(?)
SetIntersect_OfObjects 13929 13868 -0.4% 1.00x
ErrorHandling 3837 3840 +0.1% 1.00x(?)
NSError 684 686 +0.3% 1.00x(?)
DictionaryOfObjects 4888 4873 -0.3% 1.00x(?)
ObjectiveCBridgeStubToNSDateRef 3365 3352 -0.4% 1.00x(?)
StringEqualPointerComparison 9786 9824 +0.4% 1.00x(?)
RC4 9364 9371 +0.1% 1.00x(?)
ObjectiveCBridgeStubToNSStringRef 163 163 +0.0% 1.00x
MapReduce 45117 45083 -0.1% 1.00x(?)
Hanoi 20409 20354 -0.3% 1.00x(?)
DictionaryRemoveOfObjects 47001 46922 -0.2% 1.00x(?)
UTF8Decode 43463 43501 +0.1% 1.00x(?)
SetIsSubsetOf 2457 2445 -0.5% 1.00x(?)
Dictionary 2137 2141 +0.2% 1.00x(?)
NopDeinit 58123 58131 +0.0% 1.00x(?)
ObjectiveCBridgeStubToArrayOfNSString 29633 29726 +0.3% 1.00x(?)
ObjectiveCBridgeFromNSArrayAnyObject 77910 77650 -0.3% 1.00x(?)
SetIntersect 15204 15166 -0.2% 1.00x
RGBHistogramOfObjects 93578 93240 -0.4% 1.00x(?)
ObjectiveCBridgeStubNSDateRefAccess 1229 1235 +0.5% 1.00x
ArrayInClass 3955 3958 +0.1% 1.00x(?)
ArrayOfGenericRef 10057 10067 +0.1% 1.00x(?)
Phonebook 62370 62521 +0.2% 1.00x(?)
Sim2DArray 14586 14576 -0.1% 1.00x
SetExclusiveOr_OfObjects 46812 46755 -0.1% 1.00x(?)
MonteCarloE 108674 108580 -0.1% 1.00x(?)
SetUnion_OfObjects 34444 34542 +0.3% 1.00x(?)
StringHasSuffixUnicode 65106 64786 -0.5% 1.00x
Walsh 13223 13229 +0.1% 1.00x(?)
SetIsSubsetOf_OfObjects 2206 2216 +0.5% 1.00x(?)
DictionaryRemove 17427 17383 -0.2% 1.00x(?)
LinkedList 27545 27565 +0.1% 1.00x(?)
ObjectiveCBridgeFromNSSetAnyObjectToStringForced 119224 119614 +0.3% 1.00x(?)
RGBHistogram 43975 43933 -0.1% 1.00x(?)
SetUnion 18428 18364 -0.3% 1.00x(?)
Chars 4999 5006 +0.1% 1.00x
DeadArray 120723 120331 -0.3% 1.00x(?)
AnyHashableWithAClass 77926 77795 -0.2% 1.00x(?)
ArrayValueProp3 3074 3075 +0.0% 1.00x(?)
ObjectiveCBridgeFromNSArrayAnyObjectToStringForced 91219 90838 -0.4% 1.00x(?)
GlobalClass 0 0 +0.0% 1.00x
Memset 20650 20652 +0.0% 1.00x(?)
ArrayValueProp4 3002 3011 +0.3% 1.00x
ArrayValueProp2 3143 3144 +0.0% 1.00x(?)
ObjectiveCBridgeToNSString 1100 1109 +0.8% 0.99x(?)
StackPromo 128057 129019 +0.8% 0.99x(?)
ObjectiveCBridgeStubToNSDate 13638 13804 +1.2% 0.99x(?)
ObjectiveCBridgeStubToNSString 1334 1341 +0.5% 0.99x(?)
ProtocolDispatch 5700 5754 +0.9% 0.99x(?)
AngryPhonebook 2977 2998 +0.7% 0.99x(?)
Dictionary2 4365 4387 +0.5% 0.99x(?)
ArrayOfRef 9362 9414 +0.6% 0.99x(?)
ObjectiveCBridgeStubURLAppendPathRef 238041 239453 +0.6% 0.99x(?)
ObjectiveCBridgeToNSArray 30571 30967 +1.3% 0.99x(?)
PolymorphicCalls 1136 1143 +0.6% 0.99x
ObjectiveCBridgeFromNSDictionaryAnyObjectToStringForced 140348 142295 +1.4% 0.99x(?)
BitCount 96 97 +1.0% 0.99x
ArrayValueProp 2605 2620 +0.6% 0.99x(?)
ObjectiveCBridgeStubNSDateMutationRef 14283 14401 +0.8% 0.99x(?)
Dictionary2OfObjects 6214 6284 +1.1% 0.99x(?)
SevenBoom 1520 1531 +0.7% 0.99x(?)
ObjectiveCBridgeFromNSDictionaryAnyObject 208237 212452 +2.0% 0.98x(?)
StringHasPrefix 1672 1713 +2.5% 0.98x
ObjectiveCBridgeStubDataAppend 3328 3385 +1.7% 0.98x(?)
StringBuilder 2731 2796 +2.4% 0.98x(?)
ArrayLiteral 1251 1275 +1.9% 0.98x(?)
Calculator 979 1011 +3.3% 0.97x
ObjectiveCBridgeToNSDictionary 16470 17027 +3.4% 0.97x(?)
ArrayOfPOD 2328 2389 +2.6% 0.97x
ObjectiveCBridgeFromNSArrayAnyObjectToString 119192 123850 +3.9% 0.96x(?)
ObjectiveCBridgeToNSSet 15860 16504 +4.1% 0.96x(?)
OpenClose 424 445 +5.0% 0.95x
**Hardware Overview** Model Name: Mac mini Model Identifier: Macmini7,1 Processor Name: Intel Core i5 Processor Speed: 2.8 GHz Number of Processors: 1 Total Number of Cores: 2 L2 Cache (per Core): 256 KB L3 Cache: 3 MB Memory: 16 GB
@natecook1000

This comment has been minimized.

Copy link
Member Author

commented Oct 7, 2016

@swift-ci Please benchmark

@swift-ci

This comment has been minimized.

Copy link
Contributor

commented Oct 7, 2016

Build comment file:

Optimized (O)

Regression (1)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
StringWithCString 149600 201533 +34.7% 0.74x

Improvement (0)

No Changes (135)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
ObjectiveCBridgeStubFromNSString 783 760 -2.9% 1.03x
ProtocolDispatch2 160 157 -1.9% 1.02x
ObjectiveCBridgeStubURLAppendPathRef 233110 228955 -1.8% 1.02x(?)
ObjectiveCBridgeToNSSet 15903 15661 -1.5% 1.02x(?)
SortStringsUnicode 8332 8190 -1.7% 1.02x(?)
SetIsSubsetOf 508 499 -1.8% 1.02x
Dictionary 750 738 -1.6% 1.02x
SetExclusiveOr 2639 2592 -1.8% 1.02x
NSStringConversion 827 808 -2.3% 1.02x(?)
Chars 640 630 -1.6% 1.02x
SortStrings 1826 1807 -1.0% 1.01x
ObjectiveCBridgeFromNSStringForced 2688 2673 -0.6% 1.01x(?)
RangeAssignment 286 284 -0.7% 1.01x(?)
ErrorHandling 2945 2924 -0.7% 1.01x(?)
ObjectAllocation 155 154 -0.7% 1.01x(?)
HashTest 1747 1737 -0.6% 1.01x(?)
Join 470 466 -0.8% 1.01x(?)
ObjectiveCBridgeToNSArray 30147 29935 -0.7% 1.01x(?)
DictionaryLiteral 2763 2735 -1.0% 1.01x(?)
ObjectiveCBridgeStubURLAppendPath 231570 230265 -0.6% 1.01x(?)
ObjectiveCBridgeStubToArrayOfNSString 29278 28847 -1.5% 1.01x(?)
ObjectiveCBridgeStubDataAppend 2991 2975 -0.5% 1.01x(?)
StringHasPrefixUnicode 13924 13818 -0.8% 1.01x
SetUnion 3018 2982 -1.2% 1.01x
StringBuilder 1321 1314 -0.5% 1.01x(?)
ObjectiveCBridgeStubNSDataAppend 2433 2402 -1.3% 1.01x(?)
SevenBoom 1371 1361 -0.7% 1.01x(?)
Dictionary2OfObjects 3442 3411 -0.9% 1.01x(?)
ArraySubscript 1409 1416 +0.5% 1.00x(?)
ObjectiveCBridgeToNSString 1070 1066 -0.4% 1.00x(?)
DictionarySwapOfObjects 6758 6753 -0.1% 1.00x(?)
StackPromo 21556 21610 +0.2% 1.00x(?)
ObjectiveCBridgeFromNSDictionaryAnyObject 201546 200659 -0.4% 1.00x(?)
RecursiveOwnedParameter 1932 1931 -0.1% 1.00x(?)
ObjectiveCBridgeStubToNSString 1286 1281 -0.4% 1.00x(?)
ClassArrayGetter 12 12 +0.0% 1.00x
Array2D 2029 2027 -0.1% 1.00x(?)
Histogram 656 657 +0.1% 1.00x(?)
DictionaryBridge 3820 3832 +0.3% 1.00x(?)
ObjectiveCBridgeFromNSDictionaryAnyObjectForced 5270 5279 +0.2% 1.00x(?)
ObjectiveCBridgeStubFromNSDateRef 3708 3711 +0.1% 1.00x(?)
MonteCarloPi 45010 44984 -0.1% 1.00x(?)
ObjectiveCBridgeFromNSArrayAnyObjectForced 6025 6055 +0.5% 1.00x(?)
Prims 717 717 +0.0% 1.00x
SortLettersInPlace 971 970 -0.1% 1.00x(?)
DictionarySwap 632 632 +0.0% 1.00x
ObjectiveCBridgeFromNSString 1795 1794 -0.1% 1.00x(?)
PopFrontArray 1119 1121 +0.2% 1.00x(?)
ArrayOfGenericPOD 220 220 +0.0% 1.00x
Dictionary3OfObjects 858 856 -0.2% 1.00x(?)
StringHasPrefix 609 609 +0.0% 1.00x
ByteSwap 0 0 +0.0% 1.00x
XorLoop 380 380 +0.0% 1.00x
Integrate 238 238 +0.0% 1.00x
ProtocolDispatch 3040 3041 +0.0% 1.00x(?)
TypeFlood 0 0 +0.0% 1.00x
ObjectiveCBridgeFromNSSetAnyObject 260757 261185 +0.2% 1.00x(?)
ArrayLiteral 1180 1176 -0.3% 1.00x(?)
Dictionary3 509 510 +0.2% 1.00x(?)
StrComplexWalk 2901 2904 +0.1% 1.00x(?)
SetIntersect_OfObjects 2270 2273 +0.1% 1.00x(?)
ArrayOfRef 3539 3544 +0.1% 1.00x(?)
ObjectiveCBridgeStubDateMutation 273 273 +0.0% 1.00x
DictionaryOfObjects 2284 2286 +0.1% 1.00x(?)
ObjectiveCBridgeStubToNSDateRef 3361 3349 -0.4% 1.00x(?)
PopFrontArrayGeneric 1117 1116 -0.1% 1.00x(?)
PopFrontUnsafePointer 9122 9124 +0.0% 1.00x(?)
CaptureProp 4111 4113 +0.1% 1.00x(?)
RC4 165 165 +0.0% 1.00x
ObjectiveCBridgeStubToNSStringRef 123 123 +0.0% 1.00x
MapReduce 342 342 +0.0% 1.00x
AngryPhonebook 2825 2814 -0.4% 1.00x(?)
Hanoi 3447 3430 -0.5% 1.00x(?)
OpenClose 48 48 +0.0% 1.00x
DictionaryRemoveOfObjects 19857 19782 -0.4% 1.00x(?)
UTF8Decode 288 288 +0.0% 1.00x
ObjectiveCBridgeFromNSDictionaryAnyObjectToStringForced 134475 134423 -0.0% 1.00x(?)
SetIntersect 1084 1085 +0.1% 1.00x(?)
ObjectiveCBridgeStubNSDateRefAccess 308 308 +0.0% 1.00x
ArrayInClass 63 63 +0.0% 1.00x
ArrayOfGenericRef 3604 3616 +0.3% 1.00x(?)
StringHasSuffix 710 709 -0.1% 1.00x(?)
Phonebook 7449 7424 -0.3% 1.00x(?)
ObjectiveCBridgeStubDateAccess 182 182 +0.0% 1.00x
Sim2DArray 277 277 +0.0% 1.00x
MonteCarloE 10554 10549 -0.1% 1.00x(?)
StringHasSuffixUnicode 63014 63132 +0.2% 1.00x(?)
Walsh 308 309 +0.3% 1.00x(?)
SetIsSubsetOf_OfObjects 563 563 +0.0% 1.00x
ArrayAppend 775 774 -0.1% 1.00x(?)
DictionaryRemove 4581 4575 -0.1% 1.00x(?)
LinkedList 7213 7210 -0.0% 1.00x(?)
ObjectiveCBridgeFromNSSetAnyObjectToStringForced 106058 106481 +0.4% 1.00x(?)
RGBHistogramOfObjects 21999 22068 +0.3% 1.00x(?)
RGBHistogram 2832 2827 -0.2% 1.00x(?)
ObjectiveCBridgeFromNSSetAnyObjectForced 4576 4554 -0.5% 1.00x(?)
ArrayValueProp4 6 6 +0.0% 1.00x
ArrayOfPOD 182 182 +0.0% 1.00x
BitCount 1 1 +0.0% 1.00x
ObjectiveCBridgeFromNSArrayAnyObjectToStringForced 90788 90817 +0.0% 1.00x(?)
StringWalk 5886 5883 -0.1% 1.00x(?)
ArrayValueProp 6 6 +0.0% 1.00x
ObjectiveCBridgeStubFromArrayOfNSString 58606 58442 -0.3% 1.00x(?)
GlobalClass 0 0 +0.0% 1.00x
Memset 235 235 +0.0% 1.00x
ArrayAppendReserved 536 537 +0.2% 1.00x(?)
ArrayValueProp2 6 6 +0.0% 1.00x
ArrayValueProp3 6 6 +0.0% 1.00x
ObjectiveCBridgeFromNSDictionaryAnyObjectToString 209098 210663 +0.8% 0.99x(?)
ObjectiveCBridgeFromNSArrayAnyObjectToString 116500 117139 +0.6% 0.99x(?)
SuperChars 210034 211159 +0.5% 0.99x(?)
StringInterpolation 10769 10911 +1.3% 0.99x(?)
StaticArray 2747 2771 +0.9% 0.99x(?)
Dictionary2 2035 2047 +0.6% 0.99x(?)
NSError 324 327 +0.9% 0.99x(?)
StringEqualPointerComparison 7299 7342 +0.6% 0.99x(?)
ObjectiveCBridgeFromNSArrayAnyObject 75667 76126 +0.6% 0.99x(?)
NSDictionaryCastToSwift 6895 6964 +1.0% 0.99x(?)
ObjectiveCBridgeFromNSSetAnyObjectToString 142774 143656 +0.6% 0.99x(?)
SetExclusiveOr_OfObjects 7718 7795 +1.0% 0.99x(?)
ObjectiveCBridgeToNSDictionary 15962 16142 +1.1% 0.99x(?)
SetUnion_OfObjects 6447 6503 +0.9% 0.99x(?)
TwoSum 1576 1584 +0.5% 0.99x(?)
135 3091506 3141456 +1.6% 0.98x
ObjectiveCBridgeStubToNSDate 13169 13463 +2.2% 0.98x(?)
IterateData 2545 2594 +1.9% 0.98x
StrToInt 5081 5210 +2.5% 0.98x
DeadArray 182 185 +1.6% 0.98x
AnyHashableWithAClass 62293 63384 +1.8% 0.98x(?)
ObjectiveCBridgeStubNSDateMutationRef 11750 11983 +2.0% 0.98x(?)
Calculator 35 36 +2.9% 0.97x
NopDeinit 36981 37996 +2.7% 0.97x(?)
ObjectiveCBridgeStubFromNSStringRef 135 140 +3.7% 0.96x
ObjectiveCBridgeStubFromNSDate 3565 3717 +4.3% 0.96x
PolymorphicCalls 21 22 +4.8% 0.95x
**Unoptimized (Onone)**

Regression (3)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
StrToInt 5270 5942 +12.8% 0.89x
OpenClose 431 461 +7.0% 0.93x
ArrayAppend 3332 3553 +6.6% 0.94x

Improvement (4)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
PopFrontUnsafePointer 258656 242515 -6.2% 1.07x
StringHasSuffix 1731 1558 -10.0% 1.11x
ObjectiveCBridgeStubDateMutation 517 456 -11.8% 1.13x
TypeFlood 209 172 -17.7% 1.22x(?)

No Changes (129)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
StringHasPrefix 1638 1560 -4.8% 1.05x
ArrayOfGenericPOD 3599 3435 -4.6% 1.05x
ObjectiveCBridgeStubNSDateRefAccess 1258 1198 -4.8% 1.05x
SortStrings 2787 2684 -3.7% 1.04x
ObjectiveCBridgeStubToNSDate 13902 13333 -4.1% 1.04x(?)
ObjectiveCBridgeStubFromNSString 833 802 -3.7% 1.04x
ObjectiveCBridgeFromNSDictionaryAnyObjectToStringForced 145477 140376 -3.5% 1.04x(?)
StackPromo 131399 127209 -3.2% 1.03x(?)
ObjectiveCBridgeStubURLAppendPathRef 237092 229934 -3.0% 1.03x(?)
ObjectiveCBridgeStubURLAppendPath 237909 231491 -2.7% 1.03x
ObjectiveCBridgeStubDateAccess 1096 1067 -2.6% 1.03x
ArrayLiteral 1283 1246 -2.9% 1.03x(?)
ErrorHandling 3834 3769 -1.7% 1.02x(?)
ObjectiveCBridgeStubFromNSStringRef 171 167 -2.3% 1.02x
ObjectAllocation 567 558 -1.6% 1.02x(?)
AnyHashableWithAClass 79599 77791 -2.3% 1.02x(?)
ObjectiveCBridgeStubToArrayOfNSString 29665 29183 -1.6% 1.02x(?)
StringBuilder 2776 2720 -2.0% 1.02x(?)
ObjectiveCBridgeFromNSSetAnyObjectForced 7480 7367 -1.5% 1.02x(?)
ObjectiveCBridgeToNSString 1105 1095 -0.9% 1.01x(?)
135 5605941 5549494 -1.0% 1.01x
RecursiveOwnedParameter 10909 10846 -0.6% 1.01x
ObjectiveCBridgeFromNSArrayAnyObjectForced 10129 10065 -0.6% 1.01x(?)
ObjectiveCBridgeFromNSStringForced 3096 3080 -0.5% 1.01x(?)
SortLettersInPlace 2605 2589 -0.6% 1.01x(?)
ObjectiveCBridgeFromNSDictionaryAnyObject 211158 208143 -1.4% 1.01x(?)
SetIntersect_OfObjects 13964 13886 -0.6% 1.01x(?)
Join 1469 1461 -0.5% 1.01x(?)
NSError 701 693 -1.1% 1.01x(?)
PopFrontArrayGeneric 9620 9561 -0.6% 1.01x
CaptureProp 120031 118693 -1.1% 1.01x(?)
SortStringsUnicode 9222 9148 -0.8% 1.01x
NSDictionaryCastToSwift 8401 8288 -1.4% 1.01x(?)
ObjectiveCBridgeToNSSet 16025 15898 -0.8% 1.01x(?)
StringHasPrefixUnicode 15750 15520 -1.5% 1.01x
NSStringConversion 3020 2976 -1.5% 1.01x(?)
DeadArray 121609 120355 -1.0% 1.01x(?)
ArrayValueProp3 3085 3058 -0.9% 1.01x
StringWalk 21505 21396 -0.5% 1.01x(?)
ArraySubscript 5649 5665 +0.3% 1.00x(?)
DictionarySwapOfObjects 21118 21208 +0.4% 1.00x(?)
PopFrontArray 23883 23925 +0.2% 1.00x(?)
ObjectiveCBridgeStubToNSString 1338 1336 -0.1% 1.00x(?)
ClassArrayGetter 1267 1266 -0.1% 1.00x(?)
Array2D 815129 816253 +0.1% 1.00x(?)
Histogram 11727 11723 -0.0% 1.00x(?)
DictionaryBridge 3912 3900 -0.3% 1.00x(?)
ObjectiveCBridgeFromNSDictionaryAnyObjectForced 7614 7607 -0.1% 1.00x(?)
MonteCarloPi 53641 53783 +0.3% 1.00x(?)
StringWithCString 155007 154646 -0.2% 1.00x
Prims 13570 13620 +0.4% 1.00x(?)
DictionarySwap 7512 7526 +0.2% 1.00x(?)
Dictionary3OfObjects 2253 2252 -0.0% 1.00x(?)
RangeAssignment 27518 27538 +0.1% 1.00x(?)
ByteSwap 9 9 +0.0% 1.00x
SuperChars 261283 261745 +0.2% 1.00x(?)
XorLoop 20028 19996 -0.2% 1.00x
Integrate 365 365 +0.0% 1.00x
StaticArray 34966 35084 +0.3% 1.00x(?)
ProtocolDispatch 5744 5744 +0.0% 1.00x
ObjectiveCBridgeFromNSSetAnyObject 272389 273346 +0.3% 1.00x(?)
ProtocolDispatch2 447 445 -0.5% 1.00x
HashTest 5373 5385 +0.2% 1.00x(?)
Dictionary3 1609 1608 -0.1% 1.00x(?)
Dictionary2 4437 4415 -0.5% 1.00x(?)
StrComplexWalk 7181 7183 +0.0% 1.00x(?)
ArrayOfRef 9348 9388 +0.4% 1.00x(?)
DictionaryOfObjects 4875 4870 -0.1% 1.00x
ObjectiveCBridgeStubToNSDateRef 3343 3356 +0.4% 1.00x(?)
StringEqualPointerComparison 9817 9787 -0.3% 1.00x(?)
PolymorphicCalls 1139 1139 +0.0% 1.00x
RC4 9377 9366 -0.1% 1.00x(?)
ObjectiveCBridgeStubToNSStringRef 163 163 +0.0% 1.00x
MapReduce 45061 45134 +0.2% 1.00x
IterateData 10621 10598 -0.2% 1.00x(?)
Hanoi 20451 20537 +0.4% 1.00x(?)
DictionaryRemoveOfObjects 46982 46974 -0.0% 1.00x(?)
UTF8Decode 43472 43493 +0.1% 1.00x(?)
SetIsSubsetOf 2451 2455 +0.2% 1.00x(?)
Dictionary 2141 2131 -0.5% 1.00x(?)
NopDeinit 58130 58080 -0.1% 1.00x(?)
SetIntersect 15212 15169 -0.3% 1.00x(?)
ObjectiveCBridgeStubDataAppend 3330 3337 +0.2% 1.00x(?)
SetExclusiveOr 30476 30347 -0.4% 1.00x(?)
RGBHistogramOfObjects 93459 93194 -0.3% 1.00x(?)
ArrayInClass 3958 3958 +0.0% 1.00x
ArrayOfGenericRef 10034 10021 -0.1% 1.00x(?)
ObjectiveCBridgeFromNSString 5140 5124 -0.3% 1.00x(?)
Phonebook 64273 64248 -0.0% 1.00x(?)
Sim2DArray 14596 14575 -0.1% 1.00x
SetExclusiveOr_OfObjects 46841 46775 -0.1% 1.00x(?)
MonteCarloE 108714 108754 +0.0% 1.00x(?)
SetUnion_OfObjects 34571 34544 -0.1% 1.00x(?)
StringHasSuffixUnicode 64519 64478 -0.1% 1.00x(?)
Walsh 13185 13241 +0.4% 1.00x(?)
SetIsSubsetOf_OfObjects 2229 2221 -0.4% 1.00x
DictionaryRemove 17299 17330 +0.2% 1.00x(?)
LinkedList 27574 27576 +0.0% 1.00x(?)
ObjectiveCBridgeFromNSSetAnyObjectToStringForced 118846 118886 +0.0% 1.00x(?)
RGBHistogram 44012 44083 +0.2% 1.00x(?)
ArrayValueProp4 3012 3018 +0.2% 1.00x
SetUnion 18426 18405 -0.1% 1.00x(?)
Chars 4988 5006 +0.4% 1.00x
ArrayValueProp2 3144 3145 +0.0% 1.00x(?)
AngryPhonebook 3006 2992 -0.5% 1.00x(?)
ObjectiveCBridgeFromNSArrayAnyObjectToStringForced 91842 92151 +0.3% 1.00x(?)
ArrayValueProp 2625 2617 -0.3% 1.00x(?)
ObjectiveCBridgeStubFromArrayOfNSString 59102 58935 -0.3% 1.00x(?)
GlobalClass 0 0 +0.0% 1.00x
Memset 20651 20634 -0.1% 1.00x(?)
Dictionary2OfObjects 6277 6261 -0.2% 1.00x(?)
TwoSum 6522 6533 +0.2% 1.00x(?)
ObjectiveCBridgeStubFromNSDateRef 4076 4104 +0.7% 0.99x
StringInterpolation 15721 15808 +0.6% 0.99x(?)
ObjectiveCBridgeToNSArray 30613 30775 +0.5% 0.99x(?)
DictionaryLiteral 16682 16843 +1.0% 0.99x(?)
ObjectiveCBridgeFromNSArrayAnyObject 77819 78585 +1.0% 0.99x(?)
ObjectiveCBridgeToNSDictionary 16146 16377 +1.4% 0.99x(?)
BitCount 96 97 +1.0% 0.99x
ObjectiveCBridgeStubNSDataAppend 2781 2807 +0.9% 0.99x(?)
SevenBoom 1522 1532 +0.7% 0.99x(?)
ObjectiveCBridgeFromNSSetAnyObjectToString 159628 163651 +2.5% 0.98x(?)
ObjectiveCBridgeStubNSDateMutationRef 14576 14843 +1.8% 0.98x(?)
ObjectiveCBridgeStubFromNSDate 3808 3897 +2.3% 0.98x
ObjectiveCBridgeFromNSDictionaryAnyObjectToString 210977 216427 +2.6% 0.97x(?)
Calculator 981 1010 +3.0% 0.97x
ArrayOfPOD 2328 2392 +2.8% 0.97x
ArrayAppendReserved 3191 3301 +3.5% 0.97x
ObjectiveCBridgeFromNSArrayAnyObjectToString 120327 125542 +4.3% 0.96x(?)
**Hardware Overview** Model Name: Mac mini Model Identifier: Macmini7,1 Processor Name: Intel Core i5 Processor Speed: 2.8 GHz Number of Processors: 1 Total Number of Cores: 2 L2 Cache (per Core): 256 KB L3 Cache: 3 MB Memory: 16 GB
@moiseev

This comment has been minimized.

Copy link
Member

commented Oct 7, 2016

I fail to understand how this change affects StringWithCString benchmark.
@swift-ci Please benchmark

@swift-ci

This comment has been minimized.

Copy link
Contributor

commented Oct 7, 2016

Build comment file:

Optimized (O)

Regression (1)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
StringWithCString 153169 220956 +44.3% 0.69x

Improvement (0)

No Changes (135)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
ObjectiveCBridgeStubNSDateMutationRef 12231 11761 -3.8% 1.04x(?)
ObjectiveCBridgeStubFromNSString 780 758 -2.8% 1.03x(?)
SortStringsUnicode 8338 8191 -1.8% 1.02x
SetIsSubsetOf 508 499 -1.8% 1.02x(?)
Dictionary 750 736 -1.9% 1.02x(?)
ArrayInClass 63 62 -1.6% 1.02x(?)
Chars 640 629 -1.7% 1.02x(?)
ObjectiveCBridgeStubNSDataAppend 2443 2403 -1.6% 1.02x(?)
StringHasSuffix 719 708 -1.5% 1.02x(?)
Dictionary2OfObjects 3445 3389 -1.6% 1.02x(?)
SortStrings 1826 1808 -1.0% 1.01x
ObjectiveCBridgeFromNSStringForced 2687 2672 -0.6% 1.01x(?)
ObjectiveCBridgeFromNSDictionaryAnyObject 202173 200597 -0.8% 1.01x(?)
RangeAssignment 285 283 -0.7% 1.01x(?)
Join 471 468 -0.6% 1.01x(?)
ObjectiveCBridgeFromNSSetAnyObject 262812 261478 -0.5% 1.01x(?)
ProtocolDispatch2 159 158 -0.6% 1.01x(?)
ObjectiveCBridgeStubURLAppendPathRef 231501 229706 -0.8% 1.01x(?)
ObjectiveCBridgeStubToNSDateRef 3349 3302 -1.4% 1.01x(?)
DictionaryLiteral 2772 2742 -1.1% 1.01x(?)
Hanoi 3448 3429 -0.6% 1.01x
DictionaryRemoveOfObjects 20132 19955 -0.9% 1.01x(?)
ObjectiveCBridgeStubURLAppendPath 229651 228233 -0.6% 1.01x(?)
NopDeinit 38079 37874 -0.5% 1.01x(?)
ObjectiveCBridgeStubDataAppend 2988 2970 -0.6% 1.01x(?)
SetExclusiveOr 2618 2589 -1.1% 1.01x(?)
StringHasPrefixUnicode 13924 13823 -0.7% 1.01x(?)
NSStringConversion 824 814 -1.2% 1.01x(?)
SetUnion 3016 2984 -1.1% 1.01x(?)
StringBuilder 1326 1319 -0.5% 1.01x(?)
ObjectiveCBridgeStubFromArrayOfNSString 58842 58369 -0.8% 1.01x(?)
TwoSum 1589 1573 -1.0% 1.01x(?)
ArraySubscript 1420 1419 -0.1% 1.00x(?)
ObjectiveCBridgeToNSString 1073 1071 -0.2% 1.00x(?)
DictionarySwapOfObjects 6748 6754 +0.1% 1.00x(?)
StackPromo 21531 21573 +0.2% 1.00x(?)
PopFrontArray 1121 1120 -0.1% 1.00x(?)
RecursiveOwnedParameter 1934 1933 -0.1% 1.00x(?)
ObjectiveCBridgeStubToNSString 1285 1281 -0.3% 1.00x(?)
ObjectiveCBridgeFromNSDictionaryAnyObjectToString 207859 207371 -0.2% 1.00x(?)
ClassArrayGetter 12 12 +0.0% 1.00x
Histogram 657 656 -0.1% 1.00x(?)
DictionaryBridge 3822 3811 -0.3% 1.00x(?)
ObjectiveCBridgeFromNSDictionaryAnyObjectForced 5269 5279 +0.2% 1.00x(?)
ObjectiveCBridgeStubFromNSDateRef 3710 3711 +0.0% 1.00x(?)
MonteCarloPi 45025 45010 -0.0% 1.00x(?)
ObjectiveCBridgeFromNSArrayAnyObjectForced 6064 6085 +0.3% 1.00x(?)
Prims 717 717 +0.0% 1.00x
SortLettersInPlace 972 973 +0.1% 1.00x(?)
DictionarySwap 628 627 -0.2% 1.00x(?)
ArrayOfGenericPOD 220 220 +0.0% 1.00x
Dictionary3OfObjects 858 855 -0.3% 1.00x(?)
StringHasPrefix 609 608 -0.2% 1.00x(?)
ByteSwap 0 0 +0.0% 1.00x
SuperChars 210033 211067 +0.5% 1.00x(?)
ObjectiveCBridgeStubToNSDate 13061 13100 +0.3% 1.00x(?)
XorLoop 380 380 +0.0% 1.00x
StringInterpolation 10883 10873 -0.1% 1.00x(?)
Integrate 238 238 +0.0% 1.00x
StaticArray 2752 2760 +0.3% 1.00x(?)
ProtocolDispatch 3041 3040 -0.0% 1.00x(?)
ObjectAllocation 153 153 +0.0% 1.00x
TypeFlood 0 0 +0.0% 1.00x
ArrayLiteral 1171 1166 -0.4% 1.00x(?)
HashTest 1745 1738 -0.4% 1.00x(?)
Dictionary3 509 510 +0.2% 1.00x(?)
StrComplexWalk 2901 2894 -0.2% 1.00x(?)
SetIntersect_OfObjects 2269 2269 +0.0% 1.00x
ErrorHandling 2948 2960 +0.4% 1.00x(?)
ArrayOfRef 3535 3537 +0.1% 1.00x(?)
ObjectiveCBridgeStubDateMutation 273 273 +0.0% 1.00x
ObjectiveCBridgeToNSArray 30171 30237 +0.2% 1.00x(?)
DictionaryOfObjects 2288 2286 -0.1% 1.00x(?)
PopFrontArrayGeneric 1117 1116 -0.1% 1.00x(?)
PopFrontUnsafePointer 9133 9163 +0.3% 1.00x(?)
StringEqualPointerComparison 7304 7339 +0.5% 1.00x(?)
CaptureProp 4097 4079 -0.4% 1.00x(?)
RC4 165 165 +0.0% 1.00x
ObjectiveCBridgeStubToNSStringRef 123 123 +0.0% 1.00x
MapReduce 344 343 -0.3% 1.00x(?)
AngryPhonebook 2823 2814 -0.3% 1.00x(?)
OpenClose 48 48 +0.0% 1.00x
UTF8Decode 288 288 +0.0% 1.00x
ObjectiveCBridgeFromNSDictionaryAnyObjectToStringForced 134318 134646 +0.2% 1.00x(?)
ObjectiveCBridgeStubToArrayOfNSString 29427 29363 -0.2% 1.00x(?)
SetIntersect 1072 1072 +0.0% 1.00x
ObjectiveCBridgeStubNSDateRefAccess 308 308 +0.0% 1.00x
ArrayOfGenericRef 3605 3606 +0.0% 1.00x(?)
ObjectiveCBridgeFromNSString 1797 1796 -0.1% 1.00x(?)
Phonebook 7451 7425 -0.3% 1.00x(?)
ObjectiveCBridgeStubDateAccess 182 182 +0.0% 1.00x
Sim2DArray 277 277 +0.0% 1.00x
MonteCarloE 10554 10573 +0.2% 1.00x(?)
StringHasSuffixUnicode 63019 63115 +0.1% 1.00x(?)
Walsh 309 310 +0.3% 1.00x(?)
SetIsSubsetOf_OfObjects 522 522 +0.0% 1.00x
ArrayAppend 774 774 +0.0% 1.00x
LinkedList 7214 7208 -0.1% 1.00x(?)
RGBHistogramOfObjects 22115 22009 -0.5% 1.00x(?)
RGBHistogram 2843 2842 -0.0% 1.00x(?)
ObjectiveCBridgeFromNSSetAnyObjectForced 4564 4555 -0.2% 1.00x(?)
ArrayValueProp4 6 6 +0.0% 1.00x
ArrayOfPOD 182 182 +0.0% 1.00x
BitCount 1 1 +0.0% 1.00x
ArrayValueProp3 6 6 +0.0% 1.00x
ObjectiveCBridgeFromNSArrayAnyObjectToStringForced 90894 91087 +0.2% 1.00x(?)
StringWalk 5881 5884 +0.1% 1.00x(?)
ArrayValueProp 6 6 +0.0% 1.00x
GlobalClass 0 0 +0.0% 1.00x
Memset 235 235 +0.0% 1.00x
ArrayAppendReserved 536 536 +0.0% 1.00x
ArrayValueProp2 6 6 +0.0% 1.00x
135 3111551 3153299 +1.3% 0.99x
ObjectiveCBridgeFromNSArrayAnyObjectToString 117794 118856 +0.9% 0.99x(?)
Array2D 2018 2034 +0.8% 0.99x(?)
Dictionary2 2032 2046 +0.7% 0.99x(?)
ObjectiveCBridgeToNSSet 15364 15442 +0.5% 0.99x(?)
NSError 326 329 +0.9% 0.99x(?)
ObjectiveCBridgeFromNSArrayAnyObject 75558 76146 +0.8% 0.99x(?)
NSDictionaryCastToSwift 6901 6996 +1.4% 0.99x(?)
ObjectiveCBridgeFromNSSetAnyObjectToString 142315 143243 +0.7% 0.99x(?)
SetExclusiveOr_OfObjects 7686 7755 +0.9% 0.99x(?)
SetUnion_OfObjects 6439 6480 +0.6% 0.99x(?)
DictionaryRemove 4622 4649 +0.6% 0.99x(?)
ObjectiveCBridgeFromNSSetAnyObjectToStringForced 105925 106864 +0.9% 0.99x(?)
SevenBoom 1355 1368 +1.0% 0.99x(?)
IterateData 2545 2598 +2.1% 0.98x
ObjectiveCBridgeToNSDictionary 15642 15935 +1.9% 0.98x(?)
DeadArray 182 185 +1.6% 0.98x
AnyHashableWithAClass 62287 63462 +1.9% 0.98x(?)
Calculator 35 36 +2.9% 0.97x
StrToInt 5068 5226 +3.1% 0.97x
ObjectiveCBridgeStubFromNSStringRef 136 142 +4.4% 0.96x
ObjectiveCBridgeStubFromNSDate 3562 3716 +4.3% 0.96x
PolymorphicCalls 21 22 +4.8% 0.95x
**Unoptimized (Onone)**

Regression (4)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
StrToInt 5266 5944 +12.9% 0.89x
ProtocolDispatch2 446 491 +10.1% 0.91x
OpenClose 424 458 +8.0% 0.93x
ArrayAppend 3329 3550 +6.6% 0.94x

Improvement (3)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
StringWalk 22693 21314 -6.1% 1.06x
StringHasSuffix 1738 1557 -10.4% 1.12x
ObjectiveCBridgeStubDateMutation 517 456 -11.8% 1.13x

No Changes (129)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
StringHasPrefix 1637 1555 -5.0% 1.05x
PopFrontUnsafePointer 258607 245234 -5.2% 1.05x
ObjectiveCBridgeStubNSDateRefAccess 1258 1198 -4.8% 1.05x
SortStrings 2787 2688 -3.5% 1.04x
ArrayOfGenericPOD 3583 3431 -4.2% 1.04x
ObjectiveCBridgeStubFromNSString 831 803 -3.4% 1.03x
ObjectiveCBridgeToNSArray 30856 30053 -2.6% 1.03x(?)
ObjectiveCBridgeStubDateAccess 1095 1067 -2.6% 1.03x
ObjectiveCBridgeFromNSArrayAnyObjectForced 10284 10035 -2.4% 1.02x(?)
ObjectiveCBridgeStubFromNSStringRef 171 168 -1.8% 1.02x
StaticArray 35336 34639 -2.0% 1.02x(?)
ProtocolDispatch 5739 5617 -2.1% 1.02x(?)
CaptureProp 119771 117284 -2.1% 1.02x
NSStringConversion 3006 2960 -1.5% 1.02x(?)
ObjectiveCBridgeStubNSDateMutationRef 14723 14457 -1.8% 1.02x(?)
ObjectiveCBridgeToNSString 1103 1096 -0.6% 1.01x
StackPromo 127452 125792 -1.3% 1.01x(?)
ObjectiveCBridgeFromNSDictionaryAnyObjectForced 7695 7612 -1.1% 1.01x
SortLettersInPlace 2605 2575 -1.1% 1.01x(?)
ErrorHandling 3799 3779 -0.5% 1.01x(?)
SetIntersect_OfObjects 13985 13891 -0.7% 1.01x(?)
PopFrontArrayGeneric 9629 9568 -0.6% 1.01x(?)
AngryPhonebook 3030 3011 -0.6% 1.01x(?)
SortStringsUnicode 9211 9153 -0.6% 1.01x(?)
ObjectiveCBridgeFromNSArrayAnyObject 78875 78227 -0.8% 1.01x(?)
SetExclusiveOr 30692 30477 -0.7% 1.01x(?)
ObjectiveCBridgeFromNSString 5155 5097 -1.1% 1.01x(?)
StringHasPrefixUnicode 15744 15527 -1.4% 1.01x
ObjectiveCBridgeFromNSSetAnyObjectForced 7399 7356 -0.6% 1.01x(?)
DeadArray 120872 120263 -0.5% 1.01x(?)
ArraySubscript 5687 5696 +0.2% 1.00x(?)
DictionarySwapOfObjects 21146 21134 -0.1% 1.00x(?)
RecursiveOwnedParameter 10847 10823 -0.2% 1.00x(?)
ObjectiveCBridgeStubToNSString 1345 1345 +0.0% 1.00x
ObjectiveCBridgeFromNSDictionaryAnyObjectToString 218569 217496 -0.5% 1.00x(?)
ClassArrayGetter 1267 1267 +0.0% 1.00x
Array2D 812274 813217 +0.1% 1.00x
Histogram 11729 11684 -0.4% 1.00x(?)
DictionaryBridge 3925 3927 +0.1% 1.00x(?)
MonteCarloPi 53673 53832 +0.3% 1.00x(?)
StringWithCString 154736 154615 -0.1% 1.00x(?)
ObjectiveCBridgeFromNSStringForced 3092 3082 -0.3% 1.00x
DictionarySwap 7525 7519 -0.1% 1.00x(?)
PopFrontArray 24086 24082 -0.0% 1.00x(?)
Dictionary3OfObjects 2255 2257 +0.1% 1.00x(?)
RangeAssignment 27542 27650 +0.4% 1.00x(?)
ByteSwap 9 9 +0.0% 1.00x
SuperChars 261145 262059 +0.3% 1.00x(?)
XorLoop 19999 20011 +0.1% 1.00x(?)
StringInterpolation 15712 15750 +0.2% 1.00x(?)
Integrate 366 366 +0.0% 1.00x
ObjectiveCBridgeFromNSSetAnyObject 271952 271112 -0.3% 1.00x(?)
AnyHashableWithAClass 77559 77733 +0.2% 1.00x(?)
Dictionary3 1608 1609 +0.1% 1.00x(?)
Dictionary2 4420 4417 -0.1% 1.00x(?)
Join 1465 1458 -0.5% 1.00x(?)
ArrayOfRef 9351 9374 +0.2% 1.00x(?)
ObjectiveCBridgeStubURLAppendPathRef 232453 231830 -0.3% 1.00x(?)
NSError 682 683 +0.1% 1.00x(?)
DictionaryOfObjects 4879 4874 -0.1% 1.00x(?)
StringEqualPointerComparison 9824 9790 -0.3% 1.00x(?)
PolymorphicCalls 1138 1136 -0.2% 1.00x(?)
RC4 9371 9390 +0.2% 1.00x(?)
ObjectiveCBridgeStubToNSStringRef 164 164 +0.0% 1.00x
MapReduce 45052 45131 +0.2% 1.00x(?)
IterateData 10642 10637 -0.1% 1.00x(?)
Hanoi 20479 20529 +0.2% 1.00x(?)
DictionaryRemoveOfObjects 46991 46993 +0.0% 1.00x(?)
UTF8Decode 43482 43487 +0.0% 1.00x(?)
SetIsSubsetOf 2452 2450 -0.1% 1.00x(?)
Dictionary 2143 2135 -0.4% 1.00x
NopDeinit 58120 58164 +0.1% 1.00x(?)
SetIntersect 15244 15226 -0.1% 1.00x(?)
ObjectiveCBridgeStubDataAppend 3326 3325 -0.0% 1.00x(?)
RGBHistogramOfObjects 93457 93277 -0.2% 1.00x(?)
StringBuilder 2720 2716 -0.1% 1.00x(?)
ArrayInClass 3958 3958 +0.0% 1.00x
ArrayOfGenericRef 10034 10009 -0.2% 1.00x(?)
Sim2DArray 14577 14583 +0.0% 1.00x(?)
SetExclusiveOr_OfObjects 46752 46928 +0.4% 1.00x(?)
MonteCarloE 108708 108671 -0.0% 1.00x(?)
SetUnion_OfObjects 34348 34470 +0.4% 1.00x(?)
StringHasSuffixUnicode 64536 64444 -0.1% 1.00x(?)
Walsh 13230 13260 +0.2% 1.00x(?)
SetIsSubsetOf_OfObjects 2229 2222 -0.3% 1.00x(?)
DictionaryRemove 17310 17354 +0.2% 1.00x(?)
LinkedList 27621 27580 -0.1% 1.00x(?)
ObjectiveCBridgeFromNSSetAnyObjectToStringForced 118003 118220 +0.2% 1.00x(?)
ObjectiveCBridgeFromNSSetAnyObjectToString 157966 157395 -0.4% 1.00x(?)
RGBHistogram 44000 44036 +0.1% 1.00x(?)
ArrayValueProp4 3016 3012 -0.1% 1.00x(?)
SetUnion 18431 18408 -0.1% 1.00x(?)
Chars 4988 4999 +0.2% 1.00x
ArrayValueProp2 3144 3138 -0.2% 1.00x(?)
ArrayLiteral 1239 1238 -0.1% 1.00x(?)
SevenBoom 1515 1520 +0.3% 1.00x(?)
ObjectiveCBridgeFromNSArrayAnyObjectToStringForced 92071 92255 +0.2% 1.00x(?)
ArrayValueProp 2648 2651 +0.1% 1.00x(?)
GlobalClass 0 0 +0.0% 1.00x
Memset 20657 20657 +0.0% 1.00x
Dictionary2OfObjects 6258 6270 +0.2% 1.00x(?)
TwoSum 6521 6548 +0.4% 1.00x(?)
ObjectiveCBridgeStubNSDataAppend 2783 2778 -0.2% 1.00x(?)
ArrayValueProp3 3082 3069 -0.4% 1.00x(?)
ObjectiveCBridgeFromNSDictionaryAnyObject 207358 209479 +1.0% 0.99x(?)
135 5536502 5604257 +1.2% 0.99x
ObjectiveCBridgeStubFromNSDateRef 4068 4106 +0.9% 0.99x
ObjectiveCBridgeStubToNSDate 13752 13846 +0.7% 0.99x(?)
HashTest 5375 5405 +0.6% 0.99x
StrComplexWalk 7137 7178 +0.6% 0.99x(?)
ObjectiveCBridgeStubToNSDateRef 3333 3355 +0.7% 0.99x(?)
DictionaryLiteral 16760 16884 +0.7% 0.99x(?)
ObjectiveCBridgeStubURLAppendPath 235572 237308 +0.7% 0.99x(?)
ObjectiveCBridgeStubToArrayOfNSString 29590 29829 +0.8% 0.99x(?)
Phonebook 61725 62564 +1.4% 0.99x(?)
ObjectiveCBridgeToNSSet 15625 15781 +1.0% 0.99x(?)
BitCount 96 97 +1.0% 0.99x
ObjectiveCBridgeStubFromArrayOfNSString 58825 59199 +0.6% 0.99x(?)
Prims 13537 13808 +2.0% 0.98x
ObjectAllocation 559 570 +2.0% 0.98x(?)
ObjectiveCBridgeFromNSDictionaryAnyObjectToStringForced 139536 142143 +1.9% 0.98x(?)
NSDictionaryCastToSwift 8207 8384 +2.2% 0.98x(?)
ObjectiveCBridgeToNSDictionary 16256 16516 +1.6% 0.98x(?)
ObjectiveCBridgeStubFromNSDate 3811 3897 +2.3% 0.98x
ObjectiveCBridgeFromNSArrayAnyObjectToString 121761 125083 +2.7% 0.97x(?)
Calculator 981 1010 +3.0% 0.97x
ArrayOfPOD 2329 2389 +2.6% 0.97x
ArrayAppendReserved 3189 3300 +3.5% 0.97x
TypeFlood 175 184 +5.1% 0.95x(?)
**Hardware Overview** Model Name: Mac mini Model Identifier: Macmini7,1 Processor Name: Intel Core i5 Processor Speed: 2.8 GHz Number of Processors: 1 Total Number of Cores: 2 L2 Cache (per Core): 256 KB L3 Cache: 3 MB Memory: 16 GB
@natecook1000

This comment has been minimized.

Copy link
Member Author

commented Oct 7, 2016

@moiseev Same—the UTF8 view is the only one I didn't modify...

@moiseev

This comment has been minimized.

Copy link
Member

commented Oct 7, 2016

@gottesmm, @swiftix, @aschwaighofer can you take a look please? It is not at all obvious how this change resulted in a CString test regression.

@moiseev

This comment has been minimized.

Copy link
Member

commented Oct 12, 2016

Insanity is doing the same thing over and over and expecting different results
@swift-ci Please benchmark

@swift-ci

This comment has been minimized.

Copy link
Contributor

commented Oct 12, 2016

Build comment file:

Optimized (O)

Regression (3)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
StringWithCString 153051 220284 +43.9% 0.69x
MapReduce 347 372 +7.2% 0.93x
OpenClose 54 58 +7.4% 0.93x

Improvement (1)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
Calculator 33 31 -6.1% 1.06x

No Changes (132)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
Walsh 320 309 -3.4% 1.04x
ObjectiveCBridgeStubNSDateMutationRef 13805 13297 -3.7% 1.04x(?)
ObjectiveCBridgeToNSSet 16536 16263 -1.6% 1.02x(?)
IterateData 2601 2547 -2.1% 1.02x
SetIsSubsetOf_OfObjects 522 512 -1.9% 1.02x(?)
Chars 637 623 -2.2% 1.02x
Dictionary2OfObjects 3879 3785 -2.4% 1.02x(?)
ObjectiveCBridgeStubFromNSDate 3350 3290 -1.8% 1.02x(?)
ObjectiveCBridgeToNSString 1223 1210 -1.1% 1.01x(?)
SortLettersInPlace 1061 1054 -0.7% 1.01x(?)
StrComplexWalk 3122 3086 -1.1% 1.01x(?)
SuperChars 219803 218165 -0.8% 1.01x(?)
ObjectiveCBridgeStubToNSDate 15046 14848 -1.3% 1.01x(?)
ObjectiveCBridgeStubToNSString 1397 1388 -0.6% 1.01x(?)
StaticArray 3236 3189 -1.4% 1.01x(?)
Dictionary2 2215 2183 -1.4% 1.01x(?)
ObjectiveCBridgeStubURLAppendPathRef 240285 237117 -1.3% 1.01x(?)
Hanoi 3173 3153 -0.6% 1.01x(?)
SetExclusiveOr_OfObjects 7887 7833 -0.7% 1.01x(?)
ObjectiveCBridgeToNSDictionary 16750 16548 -1.2% 1.01x(?)
SetUnion_OfObjects 6627 6570 -0.9% 1.01x(?)
HashTest 1942 1927 -0.8% 1.01x(?)
DictionaryRemove 4705 4642 -1.3% 1.01x(?)
RGBHistogramOfObjects 24261 24011 -1.0% 1.01x(?)
SetUnion 3042 3025 -0.6% 1.01x(?)
ObjectiveCBridgeFromNSString 1808 1787 -1.2% 1.01x
DictionarySwapOfObjects 7203 7210 +0.1% 1.00x(?)
StackPromo 21523 21445 -0.4% 1.00x(?)
RecursiveOwnedParameter 1935 1935 +0.0% 1.00x
Integrate 238 238 +0.0% 1.00x
ClassArrayGetter 14 14 +0.0% 1.00x
Array2D 2019 2025 +0.3% 1.00x(?)
Histogram 656 656 +0.0% 1.00x
ObjectiveCBridgeStubFromNSDateRef 3736 3735 -0.0% 1.00x(?)
MonteCarloPi 44989 45141 +0.3% 1.00x(?)
ObjectiveCBridgeFromNSArrayAnyObjectForced 6172 6196 +0.4% 1.00x(?)
ObjectiveCBridgeFromNSStringForced 2652 2644 -0.3% 1.00x(?)
Prims 761 760 -0.1% 1.00x(?)
DictionarySwap 627 627 +0.0% 1.00x
PopFrontArray 1118 1119 +0.1% 1.00x(?)
Dictionary3OfObjects 907 910 +0.3% 1.00x(?)
RangeAssignment 274 273 -0.4% 1.00x(?)
ByteSwap 0 0 +0.0% 1.00x
XorLoop 381 381 +0.0% 1.00x
ErrorHandling 3358 3358 +0.0% 1.00x
ObjectiveCBridgeStubFromNSStringRef 128 128 +0.0% 1.00x
ProtocolDispatch 3039 3040 +0.0% 1.00x(?)
TypeFlood 0 0 +0.0% 1.00x
ArrayLiteral 1223 1218 -0.4% 1.00x(?)
StringHasPrefix 608 609 +0.2% 1.00x(?)
SetIntersect_OfObjects 2296 2292 -0.2% 1.00x(?)
ObjectiveCBridgeStubDateMutation 273 273 +0.0% 1.00x
NSError 375 374 -0.3% 1.00x(?)
DictionaryOfObjects 2564 2564 +0.0% 1.00x
ObjectiveCBridgeStubToNSDateRef 3272 3263 -0.3% 1.00x(?)
PopFrontArrayGeneric 1117 1117 +0.0% 1.00x
PopFrontUnsafePointer 9179 9202 +0.2% 1.00x(?)
CaptureProp 4105 4094 -0.3% 1.00x(?)
PolymorphicCalls 22 22 +0.0% 1.00x
RC4 165 165 +0.0% 1.00x
ObjectiveCBridgeStubToNSStringRef 116 116 +0.0% 1.00x
AngryPhonebook 3206 3205 -0.0% 1.00x(?)
DictionaryLiteral 3259 3262 +0.1% 1.00x(?)
ArrayOfGenericPOD 220 220 +0.0% 1.00x
DictionaryRemoveOfObjects 20648 20645 -0.0% 1.00x(?)
UTF8Decode 288 288 +0.0% 1.00x
SortStringsUnicode 8147 8154 +0.1% 1.00x(?)
SetIsSubsetOf 508 508 +0.0% 1.00x
Dictionary 752 750 -0.3% 1.00x(?)
NopDeinit 36659 36657 -0.0% 1.00x(?)
SetIntersect 1077 1076 -0.1% 1.00x(?)
ObjectiveCBridgeStubDataAppend 3070 3063 -0.2% 1.00x(?)
SetExclusiveOr 2638 2638 +0.0% 1.00x
ObjectiveCBridgeStubNSDateRefAccess 308 308 +0.0% 1.00x
ArrayInClass 63 63 +0.0% 1.00x
Phonebook 7069 7067 -0.0% 1.00x(?)
ObjectiveCBridgeStubDateAccess 182 182 +0.0% 1.00x
Sim2DArray 277 277 +0.0% 1.00x
MonteCarloE 10505 10515 +0.1% 1.00x(?)
StringHasSuffixUnicode 63353 63459 +0.2% 1.00x(?)
ArrayAppend 776 774 -0.3% 1.00x(?)
LinkedList 7206 7203 -0.0% 1.00x(?)
NSStringConversion 804 806 +0.2% 1.00x(?)
RGBHistogram 2919 2924 +0.2% 1.00x(?)
ArrayValueProp4 6 6 +0.0% 1.00x
ArrayOfPOD 182 182 +0.0% 1.00x
ArrayValueProp2 6 6 +0.0% 1.00x
BitCount 1 1 +0.0% 1.00x
SevenBoom 1602 1597 -0.3% 1.00x(?)
ObjectiveCBridgeFromNSArrayAnyObjectToStringForced 91127 91121 -0.0% 1.00x(?)
StringWalk 5881 5885 +0.1% 1.00x(?)
ArrayValueProp 6 6 +0.0% 1.00x
ObjectiveCBridgeStubFromArrayOfNSString 58940 58861 -0.1% 1.00x(?)
GlobalClass 0 0 +0.0% 1.00x
Memset 235 235 +0.0% 1.00x
ArrayAppendReserved 536 536 +0.0% 1.00x
TwoSum 1699 1698 -0.1% 1.00x(?)
ArrayValueProp3 6 6 +0.0% 1.00x
ArraySubscript 1442 1452 +0.7% 0.99x(?)
ObjectiveCBridgeFromNSDictionaryAnyObject 204750 206935 +1.1% 0.99x(?)
ObjectiveCBridgeFromNSDictionaryAnyObjectToString 214872 218007 +1.5% 0.99x(?)
SortStrings 1760 1776 +0.9% 0.99x
ObjectiveCBridgeFromNSDictionaryAnyObjectForced 5274 5316 +0.8% 0.99x(?)
ObjectAllocation 179 181 +1.1% 0.99x(?)
ObjectiveCBridgeFromNSSetAnyObject 272025 273854 +0.7% 0.99x(?)
ProtocolDispatch2 158 159 +0.6% 0.99x
Dictionary3 514 517 +0.6% 0.99x(?)
Join 504 511 +1.4% 0.99x(?)
ObjectiveCBridgeToNSArray 34961 35154 +0.6% 0.99x(?)
StringEqualPointerComparison 7298 7364 +0.9% 0.99x
ObjectiveCBridgeStubToArrayOfNSString 33990 34290 +0.9% 0.99x(?)
ObjectiveCBridgeFromNSArrayAnyObject 76068 76770 +0.9% 0.99x(?)
ObjectiveCBridgeFromNSSetAnyObjectToString 149110 150788 +1.1% 0.99x(?)
StrToInt 4998 5029 +0.6% 0.99x
StringHasSuffix 709 716 +1.0% 0.99x(?)
StringHasPrefixUnicode 14069 14142 +0.5% 0.99x(?)
ObjectiveCBridgeFromNSSetAnyObjectToStringForced 114661 115764 +1.0% 0.99x(?)
ObjectiveCBridgeFromNSSetAnyObjectForced 4518 4553 +0.8% 0.99x(?)
AnyHashableWithAClass 62429 62828 +0.6% 0.99x(?)
ObjectiveCBridgeStubNSDataAppend 2468 2486 +0.7% 0.99x(?)
ObjectiveCBridgeFromNSArrayAnyObjectToString 123911 126648 +2.2% 0.98x(?)
DictionaryBridge 4020 4111 +2.3% 0.98x(?)
StringInterpolation 11696 11900 +1.7% 0.98x(?)
ObjectiveCBridgeStubURLAppendPath 236242 240182 +1.7% 0.98x(?)
ObjectiveCBridgeFromNSDictionaryAnyObjectToStringForced 141379 143729 +1.7% 0.98x(?)
NSDictionaryCastToSwift 7799 7931 +1.7% 0.98x(?)
ArrayOfGenericRef 4345 4449 +2.4% 0.98x(?)
StringBuilder 1421 1451 +2.1% 0.98x(?)
DeadArray 182 185 +1.6% 0.98x
ArrayOfRef 4260 4410 +3.5% 0.97x
135 3206065 3334331 +4.0% 0.96x
ObjectiveCBridgeStubFromNSString 753 788 +4.7% 0.96x(?)
**Unoptimized (Onone)**

Regression (3)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
ObjectiveCBridgeStubNSDateMutationRef 16177 17950 +11.0% 0.90x
TypeFlood 181 199 +9.9% 0.91x(?)
StrToInt 5233 5762 +10.1% 0.91x

Improvement (4)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
PopFrontUnsafePointer 258623 242422 -6.3% 1.07x
ArrayAppend 3675 3358 -8.6% 1.09x
ObjectiveCBridgeStubDateMutation 547 486 -11.2% 1.13x
ArrayAppendReserved 3556 3100 -12.8% 1.15x

No Changes (129)

TEST OLD_MIN NEW_MIN DELTA (%) SPEEDUP
StrComplexWalk 8820 8409 -4.7% 1.05x(?)
ObjectiveCBridgeStubNSDateRefAccess 1260 1198 -4.9% 1.05x(?)
ObjectiveCBridgeStubFromNSDate 3803 3635 -4.4% 1.05x
ObjectiveCBridgeStubFromNSStringRef 166 160 -3.6% 1.04x(?)
ObjectiveCBridgeStubFromNSDateRef 4188 4065 -2.9% 1.03x(?)
ObjectiveCBridgeStubToNSDate 16402 15975 -2.6% 1.03x(?)
PopFrontArrayGeneric 9751 9476 -2.8% 1.03x
NSDictionaryCastToSwift 9459 9208 -2.6% 1.03x(?)
ArrayOfPOD 2512 2450 -2.5% 1.03x
StackPromo 140223 137258 -2.1% 1.02x(?)
ObjectiveCBridgeStubDateAccess 1115 1095 -1.8% 1.02x
DeadArray 134532 131426 -2.3% 1.02x(?)
ObjectiveCBridgeStubNSDataAppend 2988 2934 -1.8% 1.02x(?)
BitCount 104 102 -1.9% 1.02x
ObjectiveCBridgeFromNSArrayAnyObjectForced 10452 10367 -0.8% 1.01x(?)
DictionarySwap 7575 7520 -0.7% 1.01x(?)
SuperChars 273312 271684 -0.6% 1.01x(?)
StaticArray 39637 39222 -1.1% 1.01x(?)
StringHasPrefix 1647 1629 -1.1% 1.01x
ObjectiveCBridgeStubToNSDateRef 3346 3327 -0.6% 1.01x(?)
PolymorphicCalls 1138 1132 -0.5% 1.01x(?)
Calculator 1045 1038 -0.7% 1.01x
ObjectiveCBridgeStubDataAppend 3437 3405 -0.9% 1.01x(?)
StringBuilder 3065 3048 -0.6% 1.01x(?)
Phonebook 72874 72259 -0.8% 1.01x(?)
ObjectiveCBridgeToNSDictionary 17266 17166 -0.6% 1.01x(?)
PopFrontArray 23945 23834 -0.5% 1.00x
135 5757810 5734779 -0.4% 1.00x
RecursiveOwnedParameter 10880 10882 +0.0% 1.00x(?)
Integrate 368 368 +0.0% 1.00x
ClassArrayGetter 1267 1270 +0.2% 1.00x(?)
Array2D 812708 812067 -0.1% 1.00x(?)
Histogram 12078 12051 -0.2% 1.00x(?)
ObjectiveCBridgeFromNSDictionaryAnyObjectForced 7608 7631 +0.3% 1.00x(?)
OpenClose 435 433 -0.5% 1.00x(?)
StringWithCString 154622 154379 -0.2% 1.00x(?)
Prims 13665 13637 -0.2% 1.00x(?)
SortLettersInPlace 2712 2699 -0.5% 1.00x(?)
RangeAssignment 30392 30362 -0.1% 1.00x(?)
ByteSwap 9 9 +0.0% 1.00x
XorLoop 21221 21236 +0.1% 1.00x(?)
ObjectiveCBridgeStubToNSString 1500 1496 -0.3% 1.00x(?)
Join 1494 1499 +0.3% 1.00x
ProtocolDispatch 5778 5775 -0.1% 1.00x(?)
ObjectiveCBridgeFromNSSetAnyObject 285199 285677 +0.2% 1.00x(?)
ProtocolDispatch2 443 444 +0.2% 1.00x(?)
Walsh 13086 13098 +0.1% 1.00x(?)
Dictionary3 1638 1641 +0.2% 1.00x(?)
ErrorHandling 4222 4233 +0.3% 1.00x(?)
ArrayOfRef 10196 10206 +0.1% 1.00x(?)
NSError 778 777 -0.1% 1.00x(?)
DictionaryOfObjects 5160 5158 -0.0% 1.00x(?)
RC4 9369 9372 +0.0% 1.00x(?)
ObjectiveCBridgeStubToNSStringRef 156 156 +0.0% 1.00x
MapReduce 45176 45018 -0.3% 1.00x(?)
IterateData 10670 10661 -0.1% 1.00x(?)
UTF8Decode 43534 43530 -0.0% 1.00x(?)
Dictionary 2124 2129 +0.2% 1.00x
NopDeinit 59632 59601 -0.1% 1.00x(?)
ObjectiveCBridgeFromNSArrayAnyObject 79510 79318 -0.2% 1.00x(?)
SetExclusiveOr 31955 31989 +0.1% 1.00x(?)
ArrayInClass 3956 3953 -0.1% 1.00x(?)
Sim2DArray 14569 14587 +0.1% 1.00x(?)
SetExclusiveOr_OfObjects 46770 46805 +0.1% 1.00x(?)
MonteCarloPi 53740 53711 -0.1% 1.00x(?)
MonteCarloE 109594 109662 +0.1% 1.00x(?)
SetUnion_OfObjects 34471 34458 -0.0% 1.00x(?)
StringHasSuffixUnicode 65011 64863 -0.2% 1.00x(?)
HashTest 5641 5621 -0.3% 1.00x(?)
DictionaryRemove 17431 17359 -0.4% 1.00x(?)
LinkedList 27598 27595 -0.0% 1.00x(?)
ObjectiveCBridgeFromNSSetAnyObjectToStringForced 127909 127568 -0.3% 1.00x(?)
ArrayLiteral 1327 1329 +0.1% 1.00x(?)
RGBHistogram 45963 46023 +0.1% 1.00x(?)
ObjectiveCBridgeFromNSSetAnyObjectForced 7422 7397 -0.3% 1.00x(?)
SetUnion 18514 18451 -0.3% 1.00x(?)
Chars 5008 4991 -0.3% 1.00x
AnyHashableWithAClass 77347 77494 +0.2% 1.00x(?)
SevenBoom 1729 1725 -0.2% 1.00x(?)
ObjectiveCBridgeFromNSArrayAnyObjectToStringForced 92620 93081 +0.5% 1.00x(?)
GlobalClass 0 0 +0.0% 1.00x
Memset 20655 20667 +0.1% 1.00x(?)
ArrayValueProp4 3339 3336 -0.1% 1.00x(?)
TwoSum 6677 6683 +0.1% 1.00x(?)
ArrayValueProp2 3411 3412 +0.0% 1.00x(?)
ArraySubscript 5645 5681 +0.6% 0.99x(?)
ObjectiveCBridgeToNSString 1269 1282 +1.0% 0.99x(?)
DictionarySwapOfObjects 21553 21808 +1.2% 0.99x(?)
SortStrings 2703 2736 +1.2% 0.99x
ObjectiveCBridgeFromNSStringForced 3047 3080 +1.1% 0.99x(?)
Dictionary3OfObjects 2285 2318 +1.4% 0.99x(?)
ObjectAllocation 630 638 +1.3% 0.99x(?)
AngryPhonebook 3383 3408 +0.7% 0.99x(?)
Dictionary2 4595 4620 +0.5% 0.99x(?)
SetIntersect_OfObjects 13874 14024 +1.1% 0.99x(?)
ObjectiveCBridgeStubURLAppendPathRef 243985 247561 +1.5% 0.99x(?)
ObjectiveCBridgeToNSArray 35721 36135 +1.2% 0.99x(?)
StringEqualPointerComparison 9723 9778 +0.6% 0.99x
DictionaryLiteral 17461 17590 +0.7% 0.99x(?)
ObjectiveCBridgeStubURLAppendPath 244032 245452 +0.6% 0.99x(?)
SortStringsUnicode 9178 9268 +1.0% 0.99x(?)
SetIsSubsetOf 2478 2510 +1.3% 0.99x
SetIntersect 15623 15754 +0.8% 0.99x(?)
RGBHistogramOfObjects 96729 97352 +0.6% 0.99x(?)
SetIsSubsetOf_OfObjects 2212 2237 +1.1% 0.99x
ObjectiveCBridgeFromNSSetAnyObjectToString 165334 166533 +0.7% 0.99x(?)
ArrayValueProp 2775 2800 +0.9% 0.99x
ObjectiveCBridgeStubFromArrayOfNSString 59593 60252 +1.1% 0.99x(?)
Dictionary2OfObjects 6611 6669 +0.9% 0.99x(?)
ArrayValueProp3 3383 3422 +1.1% 0.99x
DictionaryBridge 4123 4225 +2.5% 0.98x(?)
ObjectiveCBridgeFromNSDictionaryAnyObject 210673 215294 +2.2% 0.98x(?)
StringInterpolation 16688 17014 +1.9% 0.98x(?)
CaptureProp 115630 118129 +2.2% 0.98x
Hanoi 20303 20699 +1.9% 0.98x
DictionaryRemoveOfObjects 48123 49211 +2.3% 0.98x(?)
ObjectiveCBridgeFromNSDictionaryAnyObjectToStringForced 147457 149758 +1.6% 0.98x(?)
ObjectiveCBridgeStubToArrayOfNSString 33690 34528 +2.5% 0.98x(?)
ArrayOfGenericRef 10786 10983 +1.8% 0.98x
ObjectiveCBridgeToNSSet 16630 16912 +1.7% 0.98x(?)
StringHasPrefixUnicode 15390 15756 +2.4% 0.98x
NSStringConversion 3412 3496 +2.5% 0.98x(?)
ObjectiveCBridgeFromNSDictionaryAnyObjectToString 224197 232171 +3.6% 0.97x(?)
ObjectiveCBridgeFromNSArrayAnyObjectToString 125749 129814 +3.2% 0.97x(?)
ObjectiveCBridgeStubFromNSString 809 836 +3.3% 0.97x
ObjectiveCBridgeFromNSString 5204 5338 +2.6% 0.97x(?)
StringHasSuffix 1690 1743 +3.1% 0.97x
ArrayOfGenericPOD 3523 3671 +4.2% 0.96x
StringWalk 21170 22283 +5.3% 0.95x
**Hardware Overview** Model Name: Mac mini Model Identifier: Macmini7,1 Processor Name: Intel Core i5 Processor Speed: 2.8 GHz Number of Processors: 1 Total Number of Cores: 2 L2 Cache (per Core): 256 KB L3 Cache: 3 MB Memory: 16 GB
@moiseev

This comment has been minimized.

Copy link
Member

commented Oct 13, 2016

FWIW, I tried to compile this little snippet with and without changes in this PR:

// This is exactly what StringWithCString test looks like
func foonction(_ N: Int) {
  let str = String(repeating: "x", count: 100 * (1 << 16))
  for _ in 0 ..< N {
    str.withCString { _ in }
  }
}
foonction(100)

Without the changes compiler was able to inline the String.init(repeating:count:) and all the calls to += inside it, whereas version with the changes applied simply calls the initializer.

@moiseev

This comment has been minimized.

Copy link
Member

commented Oct 13, 2016

Filed rdar://problem/28746323 to investigate the regression.

@moiseev

This comment has been minimized.

Copy link
Member

commented Oct 13, 2016

Talked to @aschwaighofer about it. He's OK with the merge.

@moiseev moiseev merged commit c2bc72d into apple:master Oct 13, 2016

5 checks passed

Swift Benchmark on OS X Platform Build finished.
Details
Swift Test Linux Platform Build finished.
Details
Swift Test Linux Platform (smoke test)
Details
Swift Test OS X Platform Build finished. 43475 tests run, 0 skipped, 0 failed.
Details
Swift Test OS X Platform (smoke test)
Details
@aschwaighofer

This comment has been minimized.

Copy link
Member

commented Oct 13, 2016

To give a little bit more context. We have seen this behavior in the past. Because of slight changes to code size the inliner makes a different inlining decision which has effects on ARC (retain/release call) removal.

We plan to address the ARC "fragility" by semantic ARC. We will come back to this benchmark to make sure this regression has recovered once this work is progressed (we expect such cases to be fixed by semantic ARC changes).

@natecook1000 natecook1000 deleted the natecook1000:nc-stringdex branch Oct 13, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.