Skip to content
This repository was archived by the owner on Jan 23, 2023. It is now read-only.

Commit bbfc885

Browse files
authored
Update Performance Test Execution Instructions (#27475)
1 parent 1faf626 commit bbfc885

File tree

1 file changed

+13
-31
lines changed

1 file changed

+13
-31
lines changed

Documentation/project-docs/performance-tests.md

Lines changed: 13 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,47 +1,29 @@
1-
Performance Tests
1+
Performance Tests
22
======================
33

44
This document contains instructions for building, running, and adding Performance tests.
55

6-
Requirements
7-
--------------------
8-
### Windows
9-
To run performance tests on Windows, .NET portable v5.0 is required. This library is included in [the Visual Studio Community 2015 download](https://www.visualstudio.com/products/visual-studio-community-vs). To get the correct packages during installation, follow these steps after opening the installer:
10-
1. Select "Custom Installation" if no installation is present, or "Modify" otherwise
11-
2. Check the "Universal Windows App Development Tools" box under the "Windows and Web Development" menu
12-
3. Install
13-
### Linux
14-
Performance tests on Linux require all of the same steps as they do for regular xunit tests - see the linux instructions [here](https://github.com/dotnet/corefx/blob/master/Documentation/building/unix-instructions.md). Once you can have a directory on your Linux machine with a working corerun and xunit.console.netcore.exe (as well as the test dll containing your perf tests!), you only need to run the following command:
15-
16-
`dnu commands install Microsoft.DotNet.xunit.performance.runner.dnx 1.0.0-alpha-build0021 -f https://dotnet.myget.org/F/dotnet-buildtools/api/v3/index.json`
17-
18-
Be careful that your mscorlib, libcoreclr, and test dlls were compiled using the "/p:Configuration=Release" property. Otherwise you may get skewed results.
19-
20-
Running the tests
6+
Building and Running Tests
217
-----------
22-
### Windows
23-
Performance test files (if present) are stored within a library's ```tests/Performance``` directory and contain test methods that are all marked with a perf-specific *Benchmark* attribute. The performance tests will only be run if the ```performance``` property is set to ```true```.
8+
Performance test files (if present) are stored within a library's ```tests/Performance``` directory and contain test methods that are all marked with a perf-specific *Benchmark* attribute.
249

25-
Before running the performance tests you must run ```build -release``` from the root folder.
10+
**Step # 1:** Prior to running performance tests, a full build from the repo root must be completed: ```build -release```
2611

27-
To build and run the tests using msbuild for a project, run ```msbuild /t:BuildAndTest /p:Performance=true /p:ConfigurationGroup=Release /p:TargetOS=Windows_NT``` from the Performance directory with Admin privileges. If the v5.0 assemblies aren't installed on your system, an error will be raised and no tests will be run.
12+
**Step # 2:** Change directory to the performance tests directory: ```cd path/to/library/tests/Performance```
2813

29-
Note: Because build.cmd runs tests concurrently, it's not recommended that you execute the perf tests using it.
14+
**Step # 3:** Build and run the tests:
15+
- Windows (using admin command shell): ```msbuild /t:BuildAndTest /p:Performance=true /p:ConfigurationGroup=Release /p:TargetOS=Windows_NT```
16+
- Linux: ```<repo-root>/Tools/msbuild.sh /t:BuildAndTest /p:Performance=true /p:ConfigurationGroup=Release /p:TargetOS=Linux```
3017

31-
results will be in: corefx/bin/tests/Windows_NT.AnyCPU.Release/TESTNAME/netcoreapp1.0
32-
### Linux
33-
From your tests directory, run:
34-
```
35-
xunit.performance System.Collections.Tests.dll -trait Benchmark=true -verbose -runner ./xunit.console.netcore.exe -runnerhost ./corerun -runid System.Collections.Tests.dll-Linux -outdir results
36-
```
18+
**Note: Because build-tests.cmd/sh runs tests concurrently, do not use it for executing performance tests.**
3719

38-
This will run the perf tests for System.Collections.Tests.dll and output the results in results/System.Collections.Tests.dll-Linux.xml and results/System.Collections.Tests.dll-Linux.csv
20+
The results files will be dropped in corefx/bin/tests/FLAVOR/TESTLIBRARY/TARGETFRAMEWORK. The console output will also specify the location of these files.
3921

40-
Adding new Performance tests
22+
Adding New Performance Tests
4123
-----------
42-
Performance tests for CoreFX are built on top of xunit and [the Microsoft xunit-performance runner](https://github.com/Microsoft/xunit-performance/).
24+
Performance tests for CoreFX are built on top of xunit and [the Microsoft xunit-performance framework](https://github.com/Microsoft/xunit-performance/).
4325

44-
For the time being, perf tests should reside within their own "Performance" folder within the tests directory of a library (e.g. [corefx/src/System.IO.FileSystem/tests/Performance](https://github.com/dotnet/corefx/tree/master/src/System.IO.FileSystem/tests/Performance) contains perf tests for FileSystem).
26+
Performance tests should reside within their own "Performance" folder within the tests directory of a library (e.g. [corefx/src/System.IO.FileSystem/tests/Performance](https://github.com/dotnet/corefx/tree/master/src/System.IO.FileSystem/tests/Performance) contains perf tests for FileSystem).
4527

4628
It's easiest to copy and modify an existing example like the one above. Notice that you'll need these lines in the tests csproj:
4729
```

0 commit comments

Comments
 (0)