Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Port Port System.Linq.Performance.Tests #90

Merged
merged 4 commits into from
Jul 10, 2018
Merged

Port Port System.Linq.Performance.Tests #90

merged 4 commits into from
Jul 10, 2018

Conversation

adamsitnik
Copy link
Member

Fixes #57

78 benchmarks, BDN needed 9 minutes, xunit 2 minutes. The huge difference is due to the fact that BDN run more iterations.

BDN produced more stable and accurate results.

These benchmarks need a redesign, for almost every single tested thing .ToArray() is called at the end. Which is a huge factor and can be a dominating cost. Those benchmarks should do sth similar to CoreCLR LINQ benchmarks: just iterate over the IEnumberable

@jorive there was no duplication between CoreCLR and CoreFX tests. Both benchmarks test different LINQ parts. there are some common parts but:

  1. CoreCLR LINQ test use complex types for tests, CoreFX mostly just ints
  2. CoreCLR LINQ test iterate over the results, CoreFX call .ToArray() (a bad thing)

So I did not remove any benchmarks, just added them to a LINQ category

@adamsitnik
Copy link
Member Author

System.Linq.Performance.Tests.dll Metric Unit Iterations Average STDEV.S Min Max
System.Linq.Tests.Perf_Linq.Cast_SameType(size: 100, iteration: 1000, wrapType: ICollection) Duration msec 1000 0.003 0.013 0.001 0.282
System.Linq.Tests.Perf_Linq.Cast_SameType(size: 100, iteration: 1000, wrapType: IEnumerable) Duration msec 1000 0.004 0.006 0.002 0.090
System.Linq.Tests.Perf_Linq.Cast_SameType(size: 100, iteration: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 0.003 0.006 0.002 0.072
System.Linq.Tests.Perf_Linq.Cast_SameType(size: 100, iteration: 1000, wrapType: NoWrap) Duration msec 1000 0.006 0.010 0.001 0.163
System.Linq.Tests.Perf_Linq.Cast_ToBaseClass(size: 100, iteration: 1000, wrapType: ICollection) Duration msec 1000 0.005 0.007 0.003 0.123
System.Linq.Tests.Perf_Linq.Cast_ToBaseClass(size: 100, iteration: 1000, wrapType: IEnumerable) Duration msec 1000 0.005 0.012 0.003 0.324
System.Linq.Tests.Perf_Linq.Cast_ToBaseClass(size: 100, iteration: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 0.005 0.006 0.003 0.076
System.Linq.Tests.Perf_Linq.Cast_ToBaseClass(size: 100, iteration: 1000, wrapType: NoWrap) Duration msec 1000 0.003 0.005 0.002 0.135
System.Linq.Tests.Perf_Linq.Contains_ElementNotFound(size: 100, iterationCount: 1000, wrapType: ICollection) Duration msec 1000 0.072 0.018 0.062 0.266
System.Linq.Tests.Perf_Linq.Contains_ElementNotFound(size: 100, iterationCount: 1000, wrapType: IEnumerable) Duration msec 1000 0.603 0.047 0.557 1.447
System.Linq.Tests.Perf_Linq.Contains_ElementNotFound(size: 100, iterationCount: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 0.605 0.057 0.557 1.997
System.Linq.Tests.Perf_Linq.Contains_ElementNotFound(size: 100, iterationCount: 1000, wrapType: NoWrap) Duration msec 1000 0.069 0.010 0.063 0.260
System.Linq.Tests.Perf_Linq.Contains_FirstElementMatches(size: 100, iterationCount: 1000, wrapType: ICollection) Duration msec 1000 0.017 0.022 0.012 0.526
System.Linq.Tests.Perf_Linq.Contains_FirstElementMatches(size: 100, iterationCount: 1000, wrapType: IEnumerable) Duration msec 1000 0.028 0.018 0.022 0.240
System.Linq.Tests.Perf_Linq.Contains_FirstElementMatches(size: 100, iterationCount: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 0.028 0.018 0.022 0.228
System.Linq.Tests.Perf_Linq.Contains_FirstElementMatches(size: 100, iterationCount: 1000, wrapType: NoWrap) Duration msec 1000 0.013 0.008 0.011 0.171
System.Linq.Tests.Perf_Linq.OrderBy(size: 100, iteration: 1000, wrapType: ICollection) Duration msec 1000 4.901 0.396 4.658 14.353
System.Linq.Tests.Perf_Linq.OrderBy(size: 100, iteration: 1000, wrapType: IEnumerable) Duration msec 1000 5.789 0.501 5.408 12.049
System.Linq.Tests.Perf_Linq.OrderBy(size: 100, iteration: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 5.789 0.345 5.404 9.461
System.Linq.Tests.Perf_Linq.OrderBy(size: 100, iteration: 1000, wrapType: NoWrap) Duration msec 1000 5.252 0.750 4.681 16.550
System.Linq.Tests.Perf_Linq.OrderByDescending(size: 100, iteration: 1000, wrapType: ICollection) Duration msec 1000 4.149 1.247 3.470 26.906
System.Linq.Tests.Perf_Linq.OrderByDescending(size: 100, iteration: 1000, wrapType: IEnumerable) Duration msec 1000 4.732 0.699 4.160 15.021
System.Linq.Tests.Perf_Linq.OrderByDescending(size: 100, iteration: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 4.726 0.482 4.174 8.600
System.Linq.Tests.Perf_Linq.OrderByDescending(size: 100, iteration: 1000, wrapType: NoWrap) Duration msec 1000 3.739 0.545 3.436 14.870
System.Linq.Tests.Perf_Linq.OrderByThenBy(size: 100, iteration: 1000, wrapType: ICollection) Duration msec 1000 5.300 1.557 4.918 44.528
System.Linq.Tests.Perf_Linq.OrderByThenBy(size: 100, iteration: 1000, wrapType: IEnumerable) Duration msec 1000 6.279 0.469 5.750 14.104
System.Linq.Tests.Perf_Linq.OrderByThenBy(size: 100, iteration: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 6.063 0.583 5.677 18.765
System.Linq.Tests.Perf_Linq.OrderByThenBy(size: 100, iteration: 1000, wrapType: NoWrap) Duration msec 1000 5.502 0.528 4.945 11.614
System.Linq.Tests.Perf_Linq.Range(size: 100, iteration: 1000) Duration msec 1000 0.101 0.049 0.078 0.507
System.Linq.Tests.Perf_Linq.Repeat(size: 100, iteration: 1000) Duration msec 1000 0.108 0.077 0.078 0.728
System.Linq.Tests.Perf_Linq.Reverse(size: 100, iteration: 1000, wrapType: ICollection) Duration msec 1000 0.171 0.079 0.132 1.085
System.Linq.Tests.Perf_Linq.Reverse(size: 100, iteration: 1000, wrapType: IEnumerable) Duration msec 1000 1.274 0.769 1.044 24.593
System.Linq.Tests.Perf_Linq.Reverse(size: 100, iteration: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 1.290 0.293 1.049 5.755
System.Linq.Tests.Perf_Linq.Reverse(size: 100, iteration: 1000, wrapType: NoWrap) Duration msec 1000 0.194 0.087 0.147 1.651
System.Linq.Tests.Perf_Linq.Select(size: 100, iteration: 1000, wrapType: ICollection) Duration msec 1000 1.239 0.190 1.116 4.284
System.Linq.Tests.Perf_Linq.Select(size: 100, iteration: 1000, wrapType: IEnumerable) Duration msec 1000 1.269 0.181 1.116 3.378
System.Linq.Tests.Perf_Linq.Select(size: 100, iteration: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 1.226 0.149 1.116 3.794
System.Linq.Tests.Perf_Linq.Select(size: 100, iteration: 1000, wrapType: NoWrap) Duration msec 1000 0.236 0.050 0.209 0.601
System.Linq.Tests.Perf_Linq.SelectSelect(size: 100, iteration: 1000, wrapType: ICollection) Duration msec 1000 1.484 0.191 1.309 4.391
System.Linq.Tests.Perf_Linq.SelectSelect(size: 100, iteration: 1000, wrapType: IEnumerable) Duration msec 1000 1.502 0.223 1.311 4.101
System.Linq.Tests.Perf_Linq.SelectSelect(size: 100, iteration: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 1.446 0.169 1.309 3.458
System.Linq.Tests.Perf_Linq.SelectSelect(size: 100, iteration: 1000, wrapType: NoWrap) Duration msec 1000 0.478 0.070 0.436 1.283
System.Linq.Tests.Perf_Linq.Skip(size: 100, iteration: 1000, wrapType: ICollection) Duration msec 1000 1.050 0.134 0.959 2.647
System.Linq.Tests.Perf_Linq.Skip(size: 100, iteration: 1000, wrapType: IEnumerable) Duration msec 1000 1.056 0.161 0.957 3.517
System.Linq.Tests.Perf_Linq.Skip(size: 100, iteration: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 1.051 0.124 0.957 1.853
System.Linq.Tests.Perf_Linq.Skip(size: 100, iteration: 1000, wrapType: NoWrap) Duration msec 1000 0.365 0.052 0.336 0.746
System.Linq.Tests.Perf_Linq.SkipTake(size: 100, iteration: 1000, wrapType: ICollection) Duration msec 1000 1.135 0.159 0.965 2.480
System.Linq.Tests.Perf_Linq.SkipTake(size: 100, iteration: 1000, wrapType: IEnumerable) Duration msec 1000 1.162 0.271 0.987 6.898
System.Linq.Tests.Perf_Linq.SkipTake(size: 100, iteration: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 1.153 0.243 0.959 4.079
System.Linq.Tests.Perf_Linq.SkipTake(size: 100, iteration: 1000, wrapType: NoWrap) Duration msec 1000 0.402 0.149 0.335 3.066
System.Linq.Tests.Perf_Linq.Take(size: 100, iteration: 1000, wrapType: ICollection) Duration msec 1000 1.049 0.253 0.955 6.750
System.Linq.Tests.Perf_Linq.Take(size: 100, iteration: 1000, wrapType: IEnumerable) Duration msec 1000 1.055 0.197 0.955 4.117
System.Linq.Tests.Perf_Linq.Take(size: 100, iteration: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 1.041 0.118 0.954 1.989
System.Linq.Tests.Perf_Linq.Take(size: 100, iteration: 1000, wrapType: NoWrap) Duration msec 1000 0.392 0.256 0.337 3.145
System.Linq.Tests.Perf_Linq.ToArray(size: 100, iteration: 1000, wrapType: ICollection) Duration msec 1000 0.097 0.049 0.073 0.451
System.Linq.Tests.Perf_Linq.ToArray(size: 100, iteration: 1000, wrapType: IEnumerable) Duration msec 1000 1.129 0.149 0.978 3.390
System.Linq.Tests.Perf_Linq.ToArray(size: 100, iteration: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 1.150 0.203 0.976 3.032
System.Linq.Tests.Perf_Linq.ToArray(size: 100, iteration: 1000, wrapType: NoWrap) Duration msec 1000 0.119 0.057 0.088 0.518
System.Linq.Tests.Perf_Linq.ToDictionary(size: 100, iteration: 1000, wrapType: ICollection) Duration msec 1000 1.723 0.214 1.510 5.488
System.Linq.Tests.Perf_Linq.ToDictionary(size: 100, iteration: 1000, wrapType: IEnumerable) Duration msec 1000 3.003 0.324 2.706 9.649
System.Linq.Tests.Perf_Linq.ToDictionary(size: 100, iteration: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 2.973 0.367 2.695 10.180
System.Linq.Tests.Perf_Linq.ToDictionary(size: 100, iteration: 1000, wrapType: NoWrap) Duration msec 1000 1.262 0.337 1.023 6.509
System.Linq.Tests.Perf_Linq.ToList(size: 100, iteration: 1000, wrapType: ICollection) Duration msec 1000 0.100 0.073 0.070 0.704
System.Linq.Tests.Perf_Linq.ToList(size: 100, iteration: 1000, wrapType: IEnumerable) Duration msec 1000 0.823 0.127 0.696 1.590
System.Linq.Tests.Perf_Linq.ToList(size: 100, iteration: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 0.829 0.152 0.695 3.327
System.Linq.Tests.Perf_Linq.ToList(size: 100, iteration: 1000, wrapType: NoWrap) Duration msec 1000 0.118 0.075 0.085 0.748
System.Linq.Tests.Perf_Linq.Where(size: 100, iteration: 1000, wrapType: ICollection) Duration msec 1000 1.302 0.128 1.193 2.269
System.Linq.Tests.Perf_Linq.Where(size: 100, iteration: 1000, wrapType: IEnumerable) Duration msec 1000 1.319 0.357 1.194 6.405
System.Linq.Tests.Perf_Linq.Where(size: 100, iteration: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 1.297 0.134 1.193 2.929
System.Linq.Tests.Perf_Linq.Where(size: 100, iteration: 1000, wrapType: NoWrap) Duration msec 1000 0.636 0.128 0.563 2.335
System.Linq.Tests.Perf_Linq.WhereSelect(size: 100, iteration: 1000, wrapType: ICollection) Duration msec 1000 1.421 0.417 1.232 11.234
System.Linq.Tests.Perf_Linq.WhereSelect(size: 100, iteration: 1000, wrapType: IEnumerable) Duration msec 1000 1.374 0.150 1.231 3.576
System.Linq.Tests.Perf_Linq.WhereSelect(size: 100, iteration: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 1.443 0.361 1.231 5.952
System.Linq.Tests.Perf_Linq.WhereSelect(size: 100, iteration: 1000, wrapType: NoWrap) Duration msec 1000 0.860 0.114 0.759 1.627
System.Linq.Tests.Perf_Linq.WhereWhere(size: 100, iteration: 1000, wrapType: ICollection) Duration msec 1000 1.694 0.331 1.447 5.179
System.Linq.Tests.Perf_Linq.WhereWhere(size: 100, iteration: 1000, wrapType: IEnumerable) Duration msec 1000 1.676 0.277 1.463 6.612
System.Linq.Tests.Perf_Linq.WhereWhere(size: 100, iteration: 1000, wrapType: IReadOnlyCollection) Duration msec 1000 1.648 0.194 1.443 3.716
System.Linq.Tests.Perf_Linq.WhereWhere(size: 100, iteration: 1000, wrapType: NoWrap) Duration msec 1000 0.979 0.162 0.841 2.701

@adamsitnik
Copy link
Member Author

Method size iteration wrapType iterationCount Mean Error StdDev Median Min Max Gen 0 Gen 1 Allocated
Select 100 1000 ICollection ? 1,145,107.3 ns 5,300.923 ns 4,426.510 ns 1,144,155.4 ns 1,140,155.0 ns 1,153,064.9 ns 191.9643 - 1240024 B
SelectSelect 100 1000 ICollection ? 1,406,478.5 ns 5,370.190 ns 4,192.691 ns 1,406,681.4 ns 1,400,646.7 ns 1,413,130.4 ns 218.7500 - 1392024 B
Where 100 1000 ICollection ? 1,195,513.8 ns 27,981.816 ns 31,101.711 ns 1,210,983.0 ns 1,156,872.3 ns 1,265,933.9 ns 192.3077 - 1240024 B
WhereWhere 100 1000 ICollection ? 1,537,650.8 ns 102,973.473 ns 118,584.394 ns 1,466,122.5 ns 1,447,484.8 ns 1,782,944.4 ns 218.7500 - 1392024 B
WhereSelect 100 1000 ICollection ? 1,355,332.4 ns 27,501.171 ns 30,567.476 ns 1,338,110.1 ns 1,322,199.3 ns 1,397,316.6 ns 203.1250 - 1304024 B
Cast_ToBaseClass 100 1000 ICollection ? 3,020.3 ns 63.731 ns 70.836 ns 2,975.6 ns 2,950.0 ns 3,135.0 ns 0.2932 - 1864 B
Cast_SameType 100 1000 ICollection ? 462.2 ns 9.291 ns 10.700 ns 457.3 ns 450.4 ns 483.2 ns 0.0533 - 344 B
OrderBy 100 1000 ICollection ? 5,164,876.1 ns 19,754.804 ns 17,512.118 ns 5,157,574.6 ns 5,142,805.6 ns 5,195,733.8 ns 291.6667 - 1960024 B
OrderByDescending 100 1000 ICollection ? 3,645,758.0 ns 170,203.756 ns 189,181.006 ns 3,582,862.1 ns 3,464,412.4 ns 4,040,941.1 ns 300.0000 - 1960024 B
OrderByThenBy 100 1000 ICollection ? 5,610,662.8 ns 429,106.186 ns 476,950.346 ns 5,382,218.4 ns 5,155,626.1 ns 6,867,703.6 ns 386.3636 - 2496024 B
Reverse 100 1000 ICollection ? 146,934.3 ns 3,835.404 ns 4,263.041 ns 144,602.8 ns 143,691.5 ns 159,258.0 ns 74.2925 - 472024 B
Skip 100 1000 ICollection ? 1,042,872.2 ns 25,673.100 ns 28,535.580 ns 1,051,517.3 ns 1,005,587.1 ns 1,094,285.1 ns 195.8333 - 1240024 B
Take 100 1000 ICollection ? 1,041,617.0 ns 32,815.263 ns 36,474.074 ns 1,034,996.4 ns 1,006,091.7 ns 1,130,013.0 ns 175.0000 - 1128112 B
SkipTake 100 1000 ICollection ? 1,078,180.1 ns 6,986.908 ns 5,454.918 ns 1,076,899.2 ns 1,074,058.9 ns 1,093,149.4 ns 183.0357 - 1168112 B
ToArray 100 1000 ICollection ? 77,258.5 ns 550.721 ns 459.877 ns 77,155.8 ns 76,713.6 ns 78,114.6 ns 67.2832 - 424024 B
ToList 100 1000 ICollection ? 77,857.5 ns 4,794.010 ns 5,520.789 ns 75,240.4 ns 73,431.9 ns 88,500.2 ns 73.4925 - 464024 B
ToDictionary 100 1000 ICollection ? 1,600,955.1 ns 38,571.644 ns 42,872.276 ns 1,621,779.8 ns 1,552,061.4 ns 1,703,289.1 ns 362.5000 - 2304120 B
Contains_ElementNotFound 100 ? ICollection 1000 64,215.8 ns 1,703.076 ns 1,892.965 ns 64,076.5 ns 62,172.9 ns 69,174.2 ns - - 24 B
Contains_FirstElementMatches 100 ? ICollection 1000 10,855.9 ns 273.070 ns 314.468 ns 10,972.3 ns 10,503.4 ns 11,564.8 ns - - 24 B
Select 100 1000 IEnumerable ? 1,176,298.6 ns 37,966.543 ns 42,199.708 ns 1,157,250.5 ns 1,139,365.8 ns 1,260,362.0 ns 192.3077 - 1240024 B
SelectSelect 100 1000 IEnumerable ? 1,507,154.7 ns 72,227.879 ns 83,177.726 ns 1,477,366.3 ns 1,404,043.7 ns 1,695,006.2 ns 215.9091 - 1392024 B
Where 100 1000 IEnumerable ? 1,255,724.5 ns 93,328.199 ns 103,734.036 ns 1,235,520.4 ns 1,153,645.3 ns 1,434,950.1 ns 192.7083 - 1240024 B
WhereWhere 100 1000 IEnumerable ? 1,502,797.8 ns 33,879.090 ns 37,656.516 ns 1,528,133.1 ns 1,455,061.1 ns 1,553,060.1 ns 218.7500 - 1392024 B
WhereSelect 100 1000 IEnumerable ? 1,362,668.3 ns 30,736.616 ns 34,163.664 ns 1,360,195.0 ns 1,319,174.3 ns 1,424,998.1 ns 203.1250 - 1304024 B
Cast_ToBaseClass 100 1000 IEnumerable ? 3,001.7 ns 152.387 ns 169.377 ns 2,914.5 ns 2,895.6 ns 3,408.6 ns 0.2812 - 1864 B
Cast_SameType 100 1000 IEnumerable ? 1,289.1 ns 33.364 ns 35.699 ns 1,277.4 ns 1,258.3 ns 1,393.6 ns 0.1952 - 1264 B
OrderBy 100 1000 IEnumerable ? 5,820,689.2 ns 179,897.827 ns 192,488.642 ns 5,749,498.8 ns 5,610,751.5 ns 6,222,321.8 ns 416.6667 - 2720024 B
OrderByDescending 100 1000 IEnumerable ? 4,519,684.4 ns 114,071.341 ns 131,364.714 ns 4,578,771.0 ns 4,367,362.4 ns 4,875,625.8 ns 421.8750 - 2720024 B
OrderByThenBy 100 1000 IEnumerable ? 6,208,875.7 ns 125,630.367 ns 139,637.807 ns 6,271,196.4 ns 5,977,201.5 ns 6,374,991.8 ns 500.0000 - 3256024 B
Reverse 100 1000 IEnumerable ? 1,110,837.1 ns 42,264.941 ns 45,223.009 ns 1,086,248.7 ns 1,071,048.5 ns 1,228,355.5 ns 191.9643 - 1232024 B
Skip 100 1000 IEnumerable ? 1,050,167.6 ns 12,665.250 ns 10,576.056 ns 1,052,103.5 ns 1,016,226.8 ns 1,059,009.0 ns 195.8333 - 1240024 B
Take 100 1000 IEnumerable ? 1,045,070.0 ns 28,297.368 ns 31,452.447 ns 1,036,845.0 ns 1,012,219.2 ns 1,133,061.4 ns 175.0000 - 1128112 B
SkipTake 100 1000 IEnumerable ? 1,116,714.1 ns 53,417.013 ns 57,155.600 ns 1,099,832.0 ns 1,024,324.8 ns 1,228,990.6 ns 182.6923 - 1168112 B
ToArray 100 1000 IEnumerable ? 1,090,977.1 ns 54,829.066 ns 60,942.356 ns 1,064,392.2 ns 1,012,104.3 ns 1,211,199.0 ns 183.3333 - 1184024 B
ToList 100 1000 IEnumerable ? 763,415.9 ns 44,969.087 ns 49,983.017 ns 737,892.3 ns 722,625.7 ns 889,122.8 ns 191.1765 - 1224024 B
ToDictionary 100 1000 IEnumerable ? 2,672,472.3 ns 51,394.250 ns 54,991.267 ns 2,694,167.8 ns 2,583,504.7 ns 2,750,064.4 ns 1177.0833 31.2500 7424120 B
Contains_ElementNotFound 100 ? IEnumerable 1000 611,980.5 ns 24,168.804 ns 25,860.347 ns 608,231.0 ns 586,904.4 ns 676,249.3 ns 2.4038 - 32024 B
Contains_FirstElementMatches 100 ? IEnumerable 1000 20,987.4 ns 519.805 ns 577.762 ns 20,705.1 ns 20,508.8 ns 22,363.3 ns 5.0360 - 32024 B
Select 100 1000 IReadOnlyCollection ? 1,179,965.4 ns 36,790.104 ns 39,364.996 ns 1,177,146.4 ns 1,139,737.6 ns 1,265,062.4 ns 192.3077 - 1240024 B
SelectSelect 100 1000 IReadOnlyCollection ? 1,423,812.6 ns 30,481.587 ns 31,302.343 ns 1,407,780.0 ns 1,400,322.0 ns 1,493,131.2 ns 215.9091 - 1392024 B
Where 100 1000 IReadOnlyCollection ? 1,199,630.1 ns 30,639.062 ns 34,055.233 ns 1,208,402.0 ns 1,155,404.3 ns 1,277,735.5 ns 192.3077 - 1240024 B
WhereWhere 100 1000 IReadOnlyCollection ? 1,499,973.1 ns 29,591.093 ns 27,679.524 ns 1,516,356.6 ns 1,446,938.8 ns 1,527,569.5 ns 215.9091 - 1392024 B
WhereSelect 100 1000 IReadOnlyCollection ? 1,364,027.0 ns 35,655.110 ns 38,150.564 ns 1,343,651.1 ns 1,326,406.4 ns 1,436,892.0 ns 204.5455 - 1304024 B
Cast_ToBaseClass 100 1000 IReadOnlyCollection ? 3,040.1 ns 108.271 ns 120.343 ns 3,022.3 ns 2,914.1 ns 3,280.3 ns 0.2859 - 1864 B
Cast_SameType 100 1000 IReadOnlyCollection ? 1,286.2 ns 10.864 ns 9.072 ns 1,282.9 ns 1,277.4 ns 1,309.6 ns 0.1949 - 1264 B
OrderBy 100 1000 IReadOnlyCollection ? 5,859,538.0 ns 110,847.107 ns 92,562.345 ns 5,893,888.8 ns 5,656,611.6 ns 5,942,647.4 ns 416.6667 - 2720024 B
OrderByDescending 100 1000 IReadOnlyCollection ? 4,579,361.0 ns 102,550.719 ns 113,984.842 ns 4,592,541.2 ns 4,383,649.6 ns 4,848,617.4 ns 421.8750 - 2720024 B
OrderByThenBy 100 1000 IReadOnlyCollection ? 6,093,095.0 ns 117,017.603 ns 114,926.922 ns 6,043,334.9 ns 5,989,304.8 ns 6,312,932.7 ns 500.0000 - 3256024 B
Reverse 100 1000 IReadOnlyCollection ? 1,149,198.0 ns 68,933.851 ns 79,384.318 ns 1,117,080.7 ns 1,075,048.6 ns 1,350,446.0 ns 192.3077 - 1232024 B
Skip 100 1000 IReadOnlyCollection ? 1,051,414.0 ns 43,215.316 ns 49,766.817 ns 1,031,693.8 ns 1,006,960.5 ns 1,163,240.4 ns 191.9643 - 1240024 B
Take 100 1000 IReadOnlyCollection ? 1,071,910.3 ns 41,176.622 ns 40,440.945 ns 1,057,160.3 ns 1,010,322.5 ns 1,167,957.3 ns 175.0000 - 1128112 B
SkipTake 100 1000 IReadOnlyCollection ? 1,058,318.6 ns 34,607.977 ns 38,466.671 ns 1,037,593.7 ns 1,025,520.7 ns 1,163,109.2 ns 183.3333 - 1168112 B
ToArray 100 1000 IReadOnlyCollection ? 1,045,904.1 ns 30,572.107 ns 33,980.813 ns 1,027,467.5 ns 1,007,814.3 ns 1,117,354.6 ns 183.3333 - 1184024 B
ToList 100 1000 IReadOnlyCollection ? 758,056.7 ns 19,933.265 ns 20,469.994 ns 758,867.4 ns 723,761.4 ns 799,649.6 ns 193.7500 - 1224024 B
ToDictionary 100 1000 IReadOnlyCollection ? 2,704,994.6 ns 105,595.109 ns 117,368.674 ns 2,704,238.2 ns 2,586,799.4 ns 2,948,779.6 ns 1177.0833 31.2500 7424120 B
Contains_ElementNotFound 100 ? IReadOnlyCollection 1000 603,110.6 ns 17,320.898 ns 19,252.131 ns 592,938.1 ns 587,834.0 ns 650,498.3 ns 2.5000 - 32024 B
Contains_FirstElementMatches 100 ? IReadOnlyCollection 1000 21,069.9 ns 424.310 ns 471.619 ns 21,337.1 ns 20,407.8 ns 21,687.0 ns 5.0376 - 32024 B
Select 100 1000 NoWrap ? 257,517.7 ns 4,895.105 ns 3,821.776 ns 258,899.6 ns 247,290.2 ns 259,920.0 ns 73.7705 - 472000 B
SelectSelect 100 1000 NoWrap ? 543,665.7 ns 13,443.092 ns 15,481.083 ns 548,621.8 ns 523,147.2 ns 584,822.9 ns 96.9828 - 616000 B
Where 100 1000 NoWrap ? 690,735.8 ns 13,062.545 ns 14,518.983 ns 686,517.2 ns 673,919.2 ns 719,637.9 ns 171.1957 - 1088000 B
WhereWhere 100 1000 NoWrap ? 968,365.1 ns 32,268.755 ns 34,527.203 ns 967,769.5 ns 920,629.5 ns 1,041,477.6 ns 191.4063 - 1232000 B
WhereSelect 100 1000 NoWrap ? 837,236.9 ns 16,727.185 ns 18,592.220 ns 842,462.4 ns 805,622.2 ns 870,787.1 ns 180.9211 - 1144000 B
Cast_ToBaseClass 100 1000 NoWrap ? 867.4 ns 16.534 ns 16.239 ns 858.5 ns 854.6 ns 900.1 ns 0.0791 - 520 B
Cast_SameType 100 1000 NoWrap ? 404.3 ns 9.309 ns 10.347 ns 399.7 ns 396.3 ns 431.0 ns 0.0488 - 320 B
OrderBy 100 1000 NoWrap ? 5,008,206.9 ns 109,650.950 ns 126,274.186 ns 4,965,858.1 ns 4,867,804.9 ns 5,303,932.7 ns 296.8750 - 1960000 B
OrderByDescending 100 1000 NoWrap ? 3,743,382.2 ns 138,796.792 ns 154,272.253 ns 3,672,056.7 ns 3,623,938.4 ns 4,136,425.8 ns 300.0000 - 1960000 B
OrderByThenBy 100 1000 NoWrap ? 5,444,729.5 ns 154,993.181 ns 165,840.952 ns 5,500,701.0 ns 5,241,969.8 ns 5,886,377.8 ns 380.9524 - 2496000 B
Reverse 100 1000 NoWrap ? 160,406.8 ns 4,095.321 ns 4,716.177 ns 158,802.9 ns 155,873.0 ns 171,147.0 ns 74.2188 - 472000 B
Skip 100 1000 NoWrap ? 406,563.3 ns 10,366.947 ns 11,092.516 ns 409,186.0 ns 390,540.8 ns 425,532.0 ns 74.0132 - 472000 B
Take 100 1000 NoWrap ? 402,439.6 ns 7,790.743 ns 6,906.290 ns 402,382.1 ns 388,284.5 ns 415,227.2 ns 73.7179 - 472088 B
SkipTake 100 1000 NoWrap ? 414,054.3 ns 13,139.820 ns 14,059.459 ns 415,331.2 ns 396,127.5 ns 446,638.6 ns 80.5921 - 512088 B
ToArray 100 1000 NoWrap ? 96,641.6 ns 2,403.956 ns 2,572.206 ns 95,388.2 ns 94,748.9 ns 104,546.9 ns 66.9071 - 424000 B
ToList 100 1000 NoWrap ? 93,738.2 ns 5,627.957 ns 6,021.851 ns 91,867.6 ns 89,344.3 ns 109,288.8 ns 73.2887 - 464000 B
ToDictionary 100 1000 NoWrap ? 1,109,690.9 ns 11,928.408 ns 9,960.760 ns 1,112,395.3 ns 1,078,521.4 ns 1,117,288.3 ns 358.3333 4.1667 2272096 B
Range 100 1000 ? ? 92,177.3 ns 1,791.154 ns 1,916.514 ns 91,183.5 ns 90,600.8 ns 96,524.4 ns 73.4848 - 464000 B
Repeat 100 1000 ? ? 89,531.2 ns 1,902.283 ns 2,114.382 ns 88,283.0 ns 87,592.8 ns 94,722.4 ns 72.3110 - 456000 B
Contains_ElementNotFound 100 ? NoWrap 1000 64,699.0 ns 1,254.476 ns 1,047.544 ns 65,139.9 ns 62,403.3 ns 65,347.2 ns - - 0 B
Contains_FirstElementMatches 100 ? NoWrap 1000 10,643.2 ns 246.283 ns 283.620 ns 10,738.7 ns 10,298.6 ns 11,327.4 ns - - 0 B

note: some benchmarks have argument called iteration, some iterationCount - this is why we have so many columns..

jorive
jorive reviewed Jul 9, 2018
View reviewed changes
src/benchmarks/corefx/System.Linq/Perf.Linq.cs
=> Perf_LinqTestBase.Measure(_sizeToPreallocatedArray[size], iteration, wrapType, col => col.Where(o => o >= 0).Select(o => o + 1));

[Benchmark]
[ArgumentsSource(nameof(IterationSizeWrapperData))] // for some reason the size and iteration arguments are ignored for this benchmark
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// for some reason the size and iteration arguments are ignored for this benchmark [](start = 60, length = 82)

Is this a bug?

jorive
jorive reviewed Jul 9, 2018
View reviewed changes
src/benchmarks/corefx/System.Linq/Perf.Linq.cs
}

[Benchmark]
[ArgumentsSource(nameof(IterationSizeWrapperData))] // for some reason the size and iteration arguments are ignored for this benchmark
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

// for some reason the size and iteration arguments are ignored for this benchmark [](start = 60, length = 82)

Is this a bug?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will use GIT blame and contact the author

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are not new bugs, right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are not new bugs, right?

Not new, I just decided to add the comment because it was weird and I wanted to make it clear why the benchmarks are not using argument values, but instead some hardcoded values.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/dotnet/corefx/blob/711c42810b8b9f8b7736b7b175cf9e330932b5f3/src/System.Linq/tests/Performance/Perf.Linq.cs#L99

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should open an issue to track this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jorive I have opened #91 and put two other things there ;)

jorive
jorive reviewed Jul 9, 2018
View reviewed changes
src/benchmarks/Categories.cs
public const string V8 = "V8";
public const string Perflab = "Perflab";

public const string CoreFX = "CoreFX";

public const string LINQ = "LINQ";
Copy link
Member

@jorive jorive Jul 9, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

public const string LINQ = "LINQ"; [](start = 8, length = 34)

Are there duplicated benchmarks here? #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there was no duplication between CoreCLR and CoreFX tests. Both benchmarks test different LINQ parts. there are some common parts but:

CoreCLR LINQ test use complex types for tests, CoreFX mostly just ints
CoreCLR LINQ test iterate over the results, CoreFX call .ToArray() (a bad thing)
So I did not remove any benchmarks, just added them to a LINQ category

Copy link
Member

@jorive jorive Jul 9, 2018 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Thanks! :) #Closed

jorive
jorive approved these changes Jul 9, 2018
View reviewed changes
Copy link
Member

@jorive jorive left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@adamsitnik adamsitnik mentioned this pull request Jul 10, 2018
Closed
@adamsitnik adamsitnik merged commit 4ff744b into dotnet:master Jul 10, 2018
@adamsitnik adamsitnik deleted the linq branch October 17, 2018 15:05
@adamsitnik adamsitnik mentioned this pull request Mar 11, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jorive jorive jorive approved these changes

@ViktorHofer ViktorHofer Awaiting requested review from ViktorHofer

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@adamsitnik @jorive