Skip to content

Commit

Permalink
Update contributing docs (#1939)
Browse files Browse the repository at this point in the history
* update testing docs

* update disassembler doc

* update debugging docs

* remove development.md (it's outdated)

* update building docs
  • Loading branch information
adamsitnik committed Apr 4, 2022
1 parent 74086ac commit 5fdafbc
Show file tree
Hide file tree
Showing 6 changed files with 28 additions and 82 deletions.
10 changes: 4 additions & 6 deletions docs/articles/contributing/building.md
Expand Up @@ -2,17 +2,15 @@

There are two recommended options to build BenchmarkDotNet from source:

## Option A (Windows only) - Visual Studio
## Visual Studio

- [Visual Studio 2017 version 15.7 Preview 4 or higher](https://www.visualstudio.com/downloads/) (Community, Professional, Enterprise).
- [Visual Studio](https://www.visualstudio.com/downloads/) (Community, Professional, Enterprise) with .NET 4.6.1 SDK and F# support.

- Visual Studio 2017 doesn't have installed necessary the .NET Core SDK 2.1 Preview 1, but you can get it from the [Installing the .Net Core SDK page](https://www.microsoft.com/net/download/dotnet-core/sdk-2.1.300-preview1). You may also need [.NET Core Runtime 2.1 Preview 1](https://www.microsoft.com/net/download/dotnet-core/runtime-2.1.0-preview1).

- Visual Studio 2017 should have installed “F# language support” feature. You can also install the support [directly as a separate download](https://www.microsoft.com/en-us/download/details.aspx?id=48179).
- [.NET 5 SDK](https://dotnet.microsoft.com/download).

Once all the necessary tools are in place, building is trivial. Simply open solution file **BenchmarkDotNet.sln** that lives at the base of the repository and run Build action.

## Option B (Windows, Linux, macOS) - Cake (C# Make)
## Cake (C# Make)

[Cake (C# Make)](http://cakebuild.net/) is a cross platform build automation system with a C# DSL to do things like compiling code, copy files/folders, running unit tests, compress files and build NuGet packages.

Expand Down
2 changes: 0 additions & 2 deletions docs/articles/contributing/debugging.md
Expand Up @@ -3,5 +3,3 @@
There should be two debug profiles available in VS drop down

![](https://cloud.githubusercontent.com/assets/6011991/15627671/89f2405a-24eb-11e6-8bd1-c9d45613e0f6.png "Debug profiles")

However due to VS 2017 RC4 bug it seems that it's impossible to choose as of 2/19/2017. If you want to change it then please edit the project file (.csproj) and set your order - the first framework moniker is always used. `<TargetFrameworks>netcoreapp1.1;net461</TargetFrameworks>`
56 changes: 0 additions & 56 deletions docs/articles/contributing/development.md

This file was deleted.

15 changes: 7 additions & 8 deletions docs/articles/contributing/disassembler.md
Expand Up @@ -7,23 +7,22 @@ The disassembler might looks scarry, but once you know how it works and how to d
We have 3 disassemblers:

- Mono
- x64 for Windows
- x64 for Windows and Linux
- x86 for Windows

The MonoDisassembler is very simple: it spawns Mono with the right arguments to get the asm, Mono prints the output to the console and we just parse it. Single class does the job: `MonoDisassembler`.

When it comes to Windows disassemblers it's not so easy. To obtain the disassm we are using ClrMD. ClrMD can attach only to the process of same bitness (architecture).
This is why we have two dissasemblers: x64 and x86. The code is the same (single class, linked in two projects) but compiled for two different architectures.
This is why we have two dissasemblers: x64 and x86. The code is the same (single class, linked in two projects) but compiled for two different architectures. We keep both disassemblers in the resources of the BenchmarkDotNet.dll. When we need the disassembler, we search for it in the resources, copy it to the disk and run (it's an exe).

On Linux it's simpler (only x64 is supported) and we don't spawn a new process (everything is done in-proc).

Unfortunately ClrMD is not a signed dll. This is why, we keep both diassemblers in the resources of the BenchmarkDotNet.dll.
When we need the disassembler, we search for it in the resources, copy it to the disk and run (it's an exe).

### How to debug the disassembler

You need to create a new project which executes the code that you would like to disassemble. It can be a simple console app.
In this app, you need to run the desired code (to get it jitted) and just don't exit. Before you exit, you have to attach with Disassembler to given process.
You need to create a new console app project which executes the code that you would like to disassemble. In this app, you need to run the desired code (to get it jitted) and just don't exit before attaching the disassembler and getting the disassembly.

Disassembler requires some arguments to run: id of the process to attach, full type name of the type which contains desired method, name of desired method and what should be disassembled: asm, IL, C#, prolog & epilog.
Disassembler requires some arguments to run: id of the process to attach, full type name of the type which contains desired method, name of desired method and few other (see the example below).

Personally I use following code to run the console app and print arguments that are required to attach to it:

Expand Down Expand Up @@ -76,7 +75,7 @@ namespace Sample

Once you configure your app, you should run it. It will give you an output similar to this:

`13672 ConsoleApp1.RandomSort ArraySort True 7 C:\Users\adsitnik\AppData\Local\Temp\tmpDCB9.tmp.xml`
`13672 Sample.Program Benchmark True 7 C:\Users\adsitnik\AppData\Local\Temp\tmpDCB9.tmp.xml`

Now you go to BenchmarkDotNet solution, select desired Disassembler project in the Solution Explorer and Set it as Startup project. After this you go to the project's properties and in the Debug tab copy-paste the arguments for the disassembler. Now when you start debugging, your IDE will spawn new process of the disassembler with the right arguments to attach to the desired exe. You should be able to debug it like any other app.

Expand Down
25 changes: 17 additions & 8 deletions docs/articles/contributing/running-tests.md
@@ -1,12 +1,21 @@
# Running Tests

* To run "Classic" tests build the solution and run runClassicTests.cmd in the tests directory or comment out the `netcoreapp1.0` part of all project.json files
that belong to the testing projects.
* To run "Core" tests you just need to open Test Explorer in Visual Studio and rebuild the solution. Then tests show up in Test Explorer and you can simply run them.
To run all tests just run the following command in the repo root:

> [!IMPORTANT]
> Remember to do both before pulling a PR or publishing new version
```cmd
dotnet test -c Release BenchmarkDotNet.sln
```

* For some unit tests (e.g. for exporter tests) BenchmarkDotNet uses [approval tests'](http://approvaltests.sourceforge.net/) implementation for .NET: [ApprovalTests.Net](https://github.com/approvals/ApprovalTests.Net).
* The expected value for each test is stored in a `*.approved.txt` file located near the test source file in the repository. ApprovalTests.NET generates approved file's names automatically according test name and its parameters. This files must be added under the source control.
* It also creates a `*.received` file for each failed test. You can use different reporters for convenient file comparison. By default we use XUnit2Reporter, so you can find test run results on the test runner console as usual. You can add [UseReporter(typeof(KDiffReporter))] on test class and then ApprovalTests will open KDiff for each failed test. This way you can easily understand what's the difference between approved and received values and choose the correct one.
Most of the tests projects target `net461` and `net5.0`. If the change that you want to test is not specific to any particular runtime, you can run the tests for one of them.

```cmd
dotnet test -c Release -f net5.0 BenchmarkDotNet.sln
```

You should be able to run all of tests from your IDE as well.

## Approval Tests

For some unit tests (e.g. for exporter tests) BenchmarkDotNet uses [approval tests'](http://approvaltests.sourceforge.net/) implementation for .NET: [ApprovalTests.Net](https://github.com/approvals/ApprovalTests.Net).
* The expected value for each test is stored in a `*.approved.txt` file located near the test source file in the repository. ApprovalTests.NET generates approved file's names automatically according test name and its parameters. This files must be added under the source control.
* It also creates a `*.received` file for each failed test. You can use different reporters for convenient file comparison. By default we use XUnit2Reporter, so you can find test run results on the test runner console as usual. You can add [UseReporter(typeof(KDiffReporter))] on test class and then ApprovalTests will open KDiff for each failed test. This way you can easily understand what's the difference between approved and received values and choose the correct one.
2 changes: 0 additions & 2 deletions docs/articles/contributing/toc.yml
Expand Up @@ -4,8 +4,6 @@
href: debugging.md
- name: Running tests
href: running-tests.md
- name: Development
href: development.md
- name: Miscellaneous topics
href: miscellaneous.md
- name: Disassembler
Expand Down

0 comments on commit 5fdafbc

Please sign in to comment.