Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update benchmark results #1315

Merged
merged 1 commit into from
Oct 14, 2022
Merged

Update benchmark results #1315

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
2 changes: 1 addition & 1 deletion Jint.Benchmark/Jint.Benchmark.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,6 @@
<PackageReference Include="Jurassic" Version="3.2.6" />
<PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
<PackageReference Include="NiL.JS" Version="2.5.1600" />
<PackageReference Include="YantraJS.Core" Version="1.2.50" />
<PackageReference Include="YantraJS.Core" Version="1.2.51" />
</ItemGroup>
</Project>
122 changes: 62 additions & 60 deletions Jint.Benchmark/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,91 +9,93 @@ dotnet run -c Release --allCategories EngineComparison
* tests are run in global engine strict mode, as YantraJS always uses strict mode which improves performance
* `Jint` and `Jint_ParsedScript` shows the difference between always parsing the script source file and reusing parsed `Script` instance.

Last updated 2022-09-11
Last updated 2022-10-14

* Jint main
* Jurassic 3.2.6
* NiL.JS 2.5.1600
* YantraJS.Core 1.2.50
* YantraJS.Core 1.2.51


``` ini

BenchmarkDotNet=v0.13.2, OS=Windows 11 (10.0.22621.457)
BenchmarkDotNet=v0.13.2, OS=Windows 11 (10.0.22621.675)
AMD Ryzen 9 5950X, 1 CPU, 32 logical and 16 physical cores
.NET SDK=6.0.400
[Host] : .NET 6.0.8 (6.0.822.36306), X64 RyuJIT AVX2
DefaultJob : .NET 6.0.8 (6.0.822.36306), X64 RyuJIT AVX2
.NET SDK=7.0.100-rc.2.22477.23
[Host] : .NET 6.0.10 (6.0.1022.47605), X64 RyuJIT AVX2
DefaultJob : .NET 6.0.10 (6.0.1022.47605), X64 RyuJIT AVX2


```
| Method | FileName | Mean | StdDev | Rank | Allocated |
|------------------ |--------------------- |-----------------:|---------------:|-----:|--------------:|
| **Jint** | **array-stress** | **11,379.224 μs** | **15.0971 μs** | **4** | **8535395 B** |
| Jint_ParsedScript | array-stress | 11,074.970 μs | 18.4598 μs | 3 | 8508563 B |
| Jurassic | array-stress | 10,979.843 μs | 8.6910 μs | 3 | 11926434 B |
| NilJS | array-stress | 5,642.704 μs | 4.1736 μs | 2 | 14746566 B |
| YantraJS | array-stress | 5,496.992 μs | 23.4065 μs | 1 | 6318056 B |
| **Jint** | **array-stress** | **10,215.696 μs** | **14.2525 μs** | **4** | **7577235 B** |
| Jint_ParsedScript | array-stress | 9,888.694 μs | 20.5964 μs | 3 | 7550403 B |
| Jurassic | array-stress | 11,085.546 μs | 18.8502 μs | 5 | 11926461 B |
| NilJS | array-stress | 5,636.598 μs | 5.7896 μs | 2 | 14746566 B |
| YantraJS | array-stress | 5,076.093 μs | 29.8884 μs | 1 | 6314988 B |
| | | | | | |
| **Jint** | **dromaeo-3d-cube** | **24,146.511 μs** | **26.1510 μs** | **4** | **7259082 B** |
| Jint_ParsedScript | dromaeo-3d-cube | 23,483.111 μs | 32.3476 μs | 3 | 6952138 B |
| Jurassic | dromaeo-3d-cube | 39,587.090 μs | 61.9786 μs | 5 | 10926049 B |
| NilJS | dromaeo-3d-cube | 9,424.855 μs | 14.3491 μs | 2 | 4240315 B |
| YantraJS | dromaeo-3d-cube | 4,912.535 μs | 21.6984 μs | 1 | 8786605 B |
| **Jint** | **dromaeo-3d-cube** | **24,287.978 μs** | **43.1781 μs** | **4** | **6726806 B** |
| Jint_ParsedScript | dromaeo-3d-cube | 23,464.073 μs | 28.5274 μs | 3 | 6419862 B |
| Jurassic | dromaeo-3d-cube | 39,704.937 μs | 68.4706 μs | 5 | 10926049 B |
| NilJS | dromaeo-3d-cube | 9,205.576 μs | 14.4517 μs | 2 | 4240315 B |
| YantraJS | dromaeo-3d-cube | 4,528.786 μs | 16.8094 μs | 1 | 8783571 B |
| | | | | | |
| **Jint** | **dromaeo-core-eval** | **5,728.484 μs** | **9.5477 μs** | **2** | **355438 B** |
| Jint_ParsedScript | dromaeo-core-eval | 5,855.588 μs | 11.6084 μs | 3 | 335590 B |
| Jurassic | dromaeo-core-eval | 13,536.638 μs | 9.6263 μs | 5 | 2971094 B |
| NilJS | dromaeo-core-eval | 2,227.991 μs | 6.6451 μs | 1 | 1636803 B |
| YantraJS | dromaeo-core-eval | 8,280.202 μs | 47.5025 μs | 4 | 37131592 B |
| **Jint** | **dromaeo-core-eval** | **5,432.432 μs** | **10.3033 μs** | **2** | **355182 B** |
| Jint_ParsedScript | dromaeo-core-eval | 5,458.993 μs | 10.4310 μs | 2 | 335335 B |
| Jurassic | dromaeo-core-eval | 13,607.238 μs | 43.2512 μs | 4 | 2971060 B |
| NilJS | dromaeo-core-eval | 2,104.603 μs | 4.1631 μs | 1 | 1636803 B |
| YantraJS | dromaeo-core-eval | 8,146.161 μs | 27.3940 μs | 3 | 37128558 B |
| | | | | | |
| **Jint** | **dromaeo-object-array** | **77,560.010 μs** | **232.4946 μs** | **4** | **108722570 B** |
| Jint_ParsedScript | dromaeo-object-array | 77,566.594 μs | 301.2466 μs | 4 | 108677466 B |
| Jurassic | dromaeo-object-array | 41,083.167 μs | 50.7771 μs | 1 | 26433962 B |
| NilJS | dromaeo-object-array | 57,373.831 μs | 858.8375 μs | 3 | 184600105 B |
| YantraJS | dromaeo-object-array | 50,393.076 μs | 533.5444 μs | 2 | 24735072 B |
| **Jint** | **dromaeo-object-array** | **67,516.323 μs** | **206.4093 μs** | **5** | **104398774 B** |
| Jint_ParsedScript | dromaeo-object-array | 64,823.739 μs | 140.2733 μs | 4 | 104351697 B |
| Jurassic | dromaeo-object-array | 40,609.674 μs | 59.2756 μs | 1 | 26433897 B |
| NilJS | dromaeo-object-array | 55,900.598 μs | 138.4180 μs | 3 | 184599926 B |
| YantraJS | dromaeo-object-array | 46,439.103 μs | 75.7535 μs | 2 | 24731921 B |
| | | | | | |
| **Jint** | **droma(...)egexp [21]** | **298,896.106 μs** | **8,135.1484 μs** | **1** | **242978848 B** |
| Jint_ParsedScript | droma(...)egexp [21] | 302,652.271 μs | 7,025.5037 μs | 1 | 243414424 B |
| Jurassic | droma(...)egexp [21] | 654,490.231 μs | 10,258.3833 μs | 2 | 846502528 B |
| NilJS | droma(...)egexp [21] | 674,164.900 μs | 6,095.0307 μs | 3 | 799695424 B |
| YantraJS | droma(...)egexp [21] | 1,006,796.520 μs | 10,505.9810 μs | 4 | 966405696 B |
| **Jint** | **droma(...)egexp [21]** | **242,184.582 μs** | **2,540.4568 μs** | **2** | **177719160 B** |
| Jint_ParsedScript | droma(...)egexp [21] | 224,182.131 μs | 3,085.4110 μs | 1 | 177744853 B |
| Jurassic | droma(...)egexp [21] | 668,193.653 μs | 13,262.9937 μs | 3 | 843971400 B |
| NilJS | droma(...)egexp [21] | 667,795.640 μs | 12,173.5871 μs | 3 | 799041960 B |
| YantraJS | droma(...)egexp [21] | 1,017,836.340 μs | 14,200.0239 μs | 4 | 962355680 B |
| | | | | | |
| **Jint** | **droma(...)tring [21]** | **525,297.081 μs** | **21,438.2488 μs** | **2** | **1363376760 B** |
| Jint_ParsedScript | droma(...)tring [21] | 526,864.316 μs | 17,744.2228 μs | 2 | 1363452752 B |
| Jurassic | droma(...)tring [21] | 547,799.136 μs | 17,846.3415 μs | 3 | 1492885824 B |
| NilJS | droma(...)tring [21] | 394,707.218 μs | 14,726.4115 μs | 1 | 1418391032 B |
| YantraJS | droma(...)tring [21] | 2,310,121.450 μs | 68,560.6400 μs | 4 | 16092741168 B |
| **Jint** | **droma(...)tring [21]** | **523,799.294 μs** | **17,239.4929 μs** | **2** | **1358370968 B** |
| Jint_ParsedScript | droma(...)tring [21] | 587,293.097 μs | 59,293.9537 μs | 4 | 1358240112 B |
| Jurassic | droma(...)tring [21] | 555,974.511 μs | 25,269.6502 μs | 3 | 1492965432 B |
| NilJS | droma(...)tring [21] | 388,563.930 μs | 10,811.7627 μs | 1 | 1418390840 B |
| YantraJS | droma(...)tring [21] | 2,981,204.866 μs | 92,355.9688 μs | 5 | 16097451064 B |
| | | | | | |
| **Jint** | **droma(...)ase64 [21]** | **67,005.288 μs** | **64.0066 μs** | **4** | **8051766 B** |
| Jint_ParsedScript | droma(...)ase64 [21] | 65,846.243 μs | 120.6587 μs | 3 | 7953457 B |
| Jurassic | droma(...)ase64 [21] | 70,798.359 μs | 238.4890 μs | 5 | 76105352 B |
| NilJS | droma(...)ase64 [21] | 40,621.460 μs | 82.0179 μs | 1 | 23963164 B |
| YantraJS | droma(...)ase64 [21] | 42,127.730 μs | 301.0910 μs | 2 | 778590774 B |
| **Jint** | **droma(...)ase64 [21]** | **68,781.989 μs** | **155.3771 μs** | **4** | **7914940 B** |
| Jint_ParsedScript | droma(...)ase64 [21] | 66,471.309 μs | 241.4502 μs | 3 | 7817337 B |
| Jurassic | droma(...)ase64 [21] | 67,141.458 μs | 187.7398 μs | 3 | 76103581 B |
| NilJS | droma(...)ase64 [21] | 41,305.925 μs | 94.0063 μs | 2 | 23963321 B |
| YantraJS | droma(...)ase64 [21] | 38,200.686 μs | 122.9918 μs | 1 | 778587651 B |
| | | | | | |
| **Jint** | **evaluation** | **28.213 μs** | **0.1590 μs** | **2** | **35232 B** |
| Jint_ParsedScript | evaluation | 11.793 μs | 0.0203 μs | 1 | 25504 B |
| Jurassic | evaluation | 1,277.875 μs | 3.9611 μs | 5 | 430506 B |
| NilJS | evaluation | 48.449 μs | 0.1407 μs | 3 | 23792 B |
| YantraJS | evaluation | 179.522 μs | 0.6104 μs | 4 | 178518 B |
| **Jint** | **evaluation** | **27.771 μs** | **0.0516 μs** | **2** | **34880 B** |
| Jint_ParsedScript | evaluation | 11.306 μs | 0.0192 μs | 1 | 25152 B |
| Jurassic | evaluation | 1,288.449 μs | 1.1880 μs | 5 | 430506 B |
| NilJS | evaluation | 44.453 μs | 0.0368 μs | 3 | 23792 B |
| YantraJS | evaluation | 122.653 μs | 0.2922 μs | 4 | 175486 B |
| | | | | | |
| **Jint** | **linq-js** | **1,806.243 μs** | **1.2092 μs** | **3** | **1300129 B** |
| Jint_ParsedScript | linq-js | 86.629 μs | 0.0921 μs | 1 | 210256 B |
| Jurassic | linq-js | 37,427.618 μs | 606.2405 μs | 4 | 9538437 B |
| **Jint** | **linq-js** | **1,699.223 μs** | **1.3486 μs** | **2** | **1295937 B** |
| Jint_ParsedScript | linq-js | 82.746 μs | 0.1714 μs | 1 | 206064 B |
| Jurassic | linq-js | 39,257.886 μs | 618.5241 μs | 3 | 9540326 B |
| NilJS | linq-js | NA | NA | ? | - |
| YantraJS | linq-js | 393.218 μs | 1.1641 μs | 2 | 490377 B |
| YantraJS | linq-js | NA | NA | ? | - |
| | | | | | |
| **Jint** | **minimal** | **5.423 μs** | **0.0044 μs** | **3** | **14248 B** |
| Jint_ParsedScript | minimal | 3.650 μs | 0.0062 μs | 1 | 12752 B |
| Jurassic | minimal | 237.757 μs | 0.3432 μs | 5 | 395506 B |
| NilJS | minimal | 5.119 μs | 0.0149 μs | 2 | 4816 B |
| YantraJS | minimal | 174.715 μs | 0.4206 μs | 4 | 174671 B |
| **Jint** | **minimal** | **5.252 μs** | **0.0302 μs** | **3** | **13992 B** |
| Jint_ParsedScript | minimal | 3.597 μs | 0.0130 μs | 1 | 12496 B |
| Jurassic | minimal | 237.108 μs | 0.2916 μs | 5 | 395505 B |
| NilJS | minimal | 4.659 μs | 0.0061 μs | 2 | 4816 B |
| YantraJS | minimal | 118.594 μs | 0.1637 μs | 4 | 171638 B |
| | | | | | |
| **Jint** | **stopwatch** | **384,627.940 μs** | **1,009.2949 μs** | **4** | **38904816 B** |
| Jint_ParsedScript | stopwatch | 386,851.529 μs | 1,015.5684 μs | 4 | 38875776 B |
| Jurassic | stopwatch | 206,273.171 μs | 562.2965 μs | 2 | 160703632 B |
| NilJS | stopwatch | 243,991.748 μs | 2,371.4106 μs | 3 | 76377896 B |
| YantraJS | stopwatch | 81,158.826 μs | 214.1868 μs | 1 | 259048157 B |
| **Jint** | **stopwatch** | **384,738.880 μs** | **920.0042 μs** | **4** | **38907168 B** |
| Jint_ParsedScript | stopwatch | 383,749.193 μs | 787.7331 μs | 4 | 38877848 B |
| Jurassic | stopwatch | 207,564.195 μs | 656.0519 μs | 2 | 160703632 B |
| NilJS | stopwatch | 240,530.110 μs | 713.6901 μs | 3 | 76378037 B |
| YantraJS | stopwatch | 74,697.115 μs | 247.4887 μs | 1 | 259044927 B |

Benchmarks with issues:
EngineComparisonBenchmark.NilJS: DefaultJob [FileName=linq-js]
EngineComparisonBenchmark.YantraJS: DefaultJob [FileName=linq-js]

2 changes: 1 addition & 1 deletion Jint.Tests.CommonScripts/Jint.Tests.CommonScripts.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

<ItemGroup>
<PackageReference Include="GitHubActionsTestLogger" Version="2.0.1" PrivateAssets="all" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.2" />
<PackageReference Include="NUnit" Version="3.13.3" />
<PackageReference Include="NUnit3TestAdapter" Version="4.2.1" />
</ItemGroup>
Expand Down
2 changes: 1 addition & 1 deletion Jint.Tests.PublicInterface/Jint.Tests.PublicInterface.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
<ItemGroup>
<PackageReference Include="Flurl.Http.Signed" Version="3.2.0" />
<PackageReference Include="GitHubActionsTestLogger" Version="2.0.1" PrivateAssets="all" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.2" />
<PackageReference Include="MongoDB.Bson.signed" Version="2.14.1" />
<PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
<PackageReference Include="xunit" Version="2.4.2" />
Expand Down
2 changes: 1 addition & 1 deletion Jint.Tests.Test262/Jint.Tests.Test262.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@

<ItemGroup>
<PackageReference Include="GitHubActionsTestLogger" Version="2.0.1" PrivateAssets="all" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.2" />
<PackageReference Include="NUnit" Version="3.13.3" />
<PackageReference Include="NUnit3TestAdapter" Version="4.2.1" />
<PackageReference Include="Test262Harness" Version="0.0.22" />
Expand Down
2 changes: 1 addition & 1 deletion Jint.Tests/Jint.Tests.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
<ItemGroup>
<PackageReference Include="Flurl.Http.Signed" Version="3.2.0" />
<PackageReference Include="GitHubActionsTestLogger" Version="2.0.1" PrivateAssets="all" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.1" />
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.3.2" />
<PackageReference Include="MongoDB.Bson.signed" Version="2.14.1" />
<PackageReference Include="Newtonsoft.Json" Version="13.0.1" />
<PackageReference Include="xunit" Version="2.4.2" />
Expand Down
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ The entire execution engine was rebuild with performance in mind, in many cases
#### ECMAScript 2022

- ❌ Class Fields
- ❌ RegExp Match Indices
- ✔ RegExp Match Indices
- ❌ Top-level await
- ❌ Ergonomic brand checks for Private Fields
- ✔ `.at()`
Expand Down