Skip to content

Commit

Permalink
docs: samples for setup and cleanup
Browse files Browse the repository at this point in the history
  • Loading branch information
AndreyAkinshin committed Jun 11, 2018
1 parent 04c0ad1 commit b16b83f
Show file tree
Hide file tree
Showing 8 changed files with 247 additions and 153 deletions.
171 changes: 19 additions & 152 deletions docs/articles/features/setup-and-cleanup.md
@@ -1,164 +1,31 @@
---
uid: docs.setup-and-cleanup
name: Setup And Cleanup
---

# Setup And Cleanup

Sometimes we want to write some logic which should be executed *before* or *after* a benchmark, but we don't want to measure it.
For this purpose, BenchmarkDotNet provides a set of attributes: `[GlobalSetup]`, `[GlobalCleanup]`, `[IterationSetup]`, `[IterationCleanup]`.

## GlobalSetup

A method which is marked by the `[GlobalSetup]` attribute will be executed only once per a benchmarked method
after initialization of benchmark parameters and before all the benchmark method invocations.

```cs
public class GlobalSetupExample
{
[Params(10, 100, 1000)]
public int N;

private int[] data;

[GlobalSetup]
public void GlobalSetup()
{
data = new int[N]; // executed once per each N value
}

[Benchmark]
public int Logic()
{
int res = 0;
for (int i = 0; i < N; i++)
res += data[i];
return res;
}
}
```

## GlobalCleanup

A method which is marked by the `[GlobalCleanup]` attribute will be executed only once per a benchmarked method
after all the benchmark method invocations.
If you are using some unmanaged resources (e.g., which were created in the `GlobalSetup` method), they can be disposed in the `GlobalCleanup` method.

```cs
public void GlobalCleanup()
{
// Disposing logic
}
```

## IterationSetup

A method which is marked by the `[IterationSetup]` attribute will be executed only once *before each an iteration*.
It's not recommended to use this attribute in microbenchmarks because it can spoil the results.
However, if you are writing a macrobenchmark (e.g. a benchmark which takes at least 100ms) and
you want to prepare some data before each iteration, `[IterationSetup]` can be useful.
BenchmarkDotNet doesn't support setup/cleanup method for a single method invocation (*an operation*), but you can perform only one operation per iteration.
It's recommended to use `RunStrategy.Monitoring` for such cases.
Be careful: if you allocate any objects in the `[IterationSetup]` method, the MemoryDiagnoser results can also be spoiled.

## IterationCleanup
A method which is marked by the `[IterationCleanup]` attribute will be executed only once *after each an iteration*.
This attribute has the same set of constraint with `[IterationSetup]`: it's not recommended to use `[IterationCleanup]` in microbenchmarks or benchmark which also

## An example

```cs
[SimpleJob(RunStrategy.Monitoring, launchCount: 1, warmupCount: 2, targetCount: 3)]
public class SetupAndCleanupExample
{
private int setupCounter;
private int cleanupCounter;

[IterationSetup]
public void IterationSetup() => Console.WriteLine("// " + "IterationSetup" + " (" + ++setupCounter + ")");

[IterationCleanup]
public void IterationCleanup() => Console.WriteLine("// " + "IterationCleanup" + " (" + ++cleanupCounter + ")");

[GlobalSetup]
public void GlobalSetup() => Console.WriteLine("// " + "GlobalSetup");

[GlobalCleanup]
public void GlobalCleanup() => Console.WriteLine("// " + "GlobalCleanup");

[Benchmark]
public void Benchmark() => Console.WriteLine("// " + "Benchmark");
}
```

The order of method calls:

```
// GlobalSetup
// IterationSetup (1) // IterationSetup Jitting
// IterationCleanup (1) // IterationCleanup Jitting
// IterationSetup (2) // MainWarmup1
// Benchmark // MainWarmup1
// IterationCleanup (2) // MainWarmup1
// IterationSetup (3) // MainWarmup2
// Benchmark // MainWarmup2
// IterationCleanup (3) // MainWarmup2
// IterationSetup (4) // MainTarget1
// Benchmark // MainTarget1
// IterationCleanup (4) // MainTarget1
// IterationSetup (5) // MainTarget2
// Benchmark // MainTarget2
// IterationCleanup (5) // MainTarget2
// IterationSetup (6) // MainTarget3
// Benchmark // MainTarget3
// IterationCleanup (6) // MainTarget3
// GlobalCleanup
```

## Target

Sometimes it's useful to run setup or cleanups for specific benchmarks. All four setup and cleanup attributes have a Target property that allow the setup/cleanup method to be run for one or more specific benchmark methods.
For this purpose, BenchmarkDotNet provides a set of attributes:
[`[GlobalSetup]`](xref:BenchmarkDotNet.Attributes.GlobalSetupAttribute),
[`[GlobalCleanup]`](xref:BenchmarkDotNet.Attributes.GlobalCleanupAttribute),
[`[IterationSetup]`](xref:BenchmarkDotNet.Attributes.IterationSetupAttribute),
[`[IterationCleanup]`](xref:BenchmarkDotNet.Attributes.IterationCleanupAttribute).

```cs
[SimpleJob(RunStrategy.Monitoring, launchCount: 0, warmupCount: 0, targetCount: 1)]
public class SetupAndCleanupExample
{
[GlobalSetup(Target = nameof(BenchmarkA))]
public void GlobalSetupA() => Console.WriteLine("// " + "GlobalSetup A");

[Benchmark]
public void BenchmarkA() => Console.WriteLine("// " + "Benchmark A");

[GlobalSetup(Target = nameof(BenchmarkB) + "," + nameof(BenchmarkC))]
public void GlobalSetupB() => Console.WriteLine("// " + "GlobalSetup B");

[Benchmark]
public void BenchmarkB() => Console.WriteLine("// " + "Benchmark B");

[Benchmark]
public void BenchmarkC() => Console.WriteLine("// " + "Benchmark C");

[Benchmark]
public void BenchmarkD() => Console.WriteLine("// " + "Benchmark D");
}
```
---

The order of method calls:
[!include[IntroSetupCleanupGlobal](../samples/IntroSetupCleanupGlobal.md)]

```
// GlobalSetup A
The link to this sample: @BenchmarkDotNet.Samples.IntroSetupCleanupGlobal

// Benchmark A
---

// GlobalSetup B
[!include[IntroSetupCleanupIteration](../samples/IntroSetupCleanupIteration.md)]

// Benchmark B
The link to this sample: @BenchmarkDotNet.Samples.IntroSetupCleanupIteration

// GlobalSetup B
---

// Benchmark C
[!include[IntroSetupCleanupTarget](../samples/IntroSetupCleanupTarget.md)]

// Benchmark D
```
The link to this sample: @BenchmarkDotNet.Samples.IntroSetupCleanupTarget
22 changes: 22 additions & 0 deletions docs/articles/samples/IntroSetupCleanupGlobal.md
@@ -0,0 +1,22 @@
---
uid: BenchmarkDotNet.Samples.IntroSetupCleanupGlobal
---

## Sample: IntroSetupCleanupGlobal

A method which is marked by the [`[GlobalSetup]`](xref:BenchmarkDotNet.Attributes.GlobalSetupAttribute)
attribute will be executed only once per a benchmarked method
after initialization of benchmark parameters and before all the benchmark method invocations.
A method which is marked by the [`[GlobalCleanup]`](xref:BenchmarkDotNet.Attributes.GlobalCleanupAttribute)
attribute will be executed only once per a benchmarked method
after all the benchmark method invocations.
If you are using some unmanaged resources (e.g., which were created in the `GlobalSetup` method),
they can be disposed in the `GlobalCleanup` method.

### Source code

[!code-csharp[IntroSetupCleanupGlobal.cs](../../../samples/BenchmarkDotNet.Samples/IntroSetupCleanupGlobal.cs)]

### See also

* @docs.setup-and-cleanup
62 changes: 62 additions & 0 deletions docs/articles/samples/IntroSetupCleanupIteration.md
@@ -0,0 +1,62 @@
---
uid: BenchmarkDotNet.Samples.IntroSetupCleanupIteration
---

## Sample: IntroSetupCleanupIteration

A method which is marked by the [`[IterationSetup]`](xref:BenchmarkDotNet.Attributes.IterationSetupAttribute)
attribute will be executed only once *before each an iteration*.
It's not recommended to use this attribute in microbenchmarks because it can spoil the results.
However, if you are writing a macrobenchmark (e.g. a benchmark which takes at least 100ms) and
you want to prepare some data before each iteration,
[`[IterationSetup]`](xref:BenchmarkDotNet.Attributes.IterationSetupAttribute) can be useful.
BenchmarkDotNet doesn't support setup/cleanup method for a single method invocation (*an operation*),
but you can perform only one operation per iteration.
It's recommended to use `RunStrategy.Monitoring` for such cases.
Be careful: if you allocate any objects in
the [`[IterationSetup]`](xref:BenchmarkDotNet.Attributes.IterationSetupAttribute) method,
the MemoryDiagnoser results can also be spoiled.

A method which is marked by the [`[IterationCleanup]`](xref:BenchmarkDotNet.Attributes.IterationCleanupAttribute)
attribute will be executed only once *after each an iteration*.
This attribute has the same set of constraint with `[IterationSetup]`: it's not recommended to use
[`[IterationCleanup]`](xref:BenchmarkDotNet.Attributes.IterationCleanupAttribute) in microbenchmarks.

### Source code

[!code-csharp[IntroSetupCleanupIteration.cs](../../../samples/BenchmarkDotNet.Samples/IntroSetupCleanupIteration.cs)]

### THe order of method calls

```cs
// GlobalSetup
// IterationSetup (1) // IterationSetup Jitting
// IterationCleanup (1) // IterationCleanup Jitting
// IterationSetup (2) // MainWarmup1
// Benchmark // MainWarmup1
// IterationCleanup (2) // MainWarmup1
// IterationSetup (3) // MainWarmup2
// Benchmark // MainWarmup2
// IterationCleanup (3) // MainWarmup2
// IterationSetup (4) // MainTarget1
// Benchmark // MainTarget1
// IterationCleanup (4) // MainTarget1
// IterationSetup (5) // MainTarget2
// Benchmark // MainTarget2
// IterationCleanup (5) // MainTarget2
// IterationSetup (6) // MainTarget3
// Benchmark // MainTarget3
// IterationCleanup (6) // MainTarget3
// GlobalCleanup
```

### See also

* @docs.setup-and-cleanup
35 changes: 35 additions & 0 deletions docs/articles/samples/IntroSetupCleanupTarget.md
@@ -0,0 +1,35 @@
---
uid: BenchmarkDotNet.Samples.IntroSetupCleanupTarget
---

## Sample: IntroSetupCleanupTarget

Sometimes it's useful to run setup or cleanups for specific benchmarks.
All four setup and cleanup attributes have a Target property that allow
the setup/cleanup method to be run for one or more specific benchmark methods.

### Source code

[!code-csharp[IntroSetupCleanupTarget.cs](../../../samples/BenchmarkDotNet.Samples/IntroSetupCleanupTarget.cs)]

### THe order of method calls

```cs
// GlobalSetup A
// Benchmark A
// GlobalSetup B
// Benchmark B
// GlobalSetup B
// Benchmark C
// Benchmark D
```

### See also

* @docs.setup-and-cleanup
8 changes: 7 additions & 1 deletion docs/articles/samples/toc.yml
Expand Up @@ -17,4 +17,10 @@
- name: IntroJobBaseline
href: IntroJobBaseline.md
- name: IntroEnvVars
href: IntroEnvVars.md
href: IntroEnvVars.md
- name: IntroSetupCleanupGlobal
href: IntroSetupCleanupGlobal.md
- name: IntroSetupCleanupIteration
href: IntroSetupCleanupIteration.md
- name: IntroSetupCleanupTarget
href: IntroSetupCleanupTarget.md
33 changes: 33 additions & 0 deletions samples/BenchmarkDotNet.Samples/IntroSetupCleanupGlobal.cs
@@ -0,0 +1,33 @@
using BenchmarkDotNet.Attributes;

namespace BenchmarkDotNet.Samples
{
public class IntroSetupCleanupGlobal
{
[Params(10, 100, 1000)]
public int N;

private int[] data;

[GlobalSetup]
public void GlobalSetup()
{
data = new int[N]; // executed once per each N value
}

[Benchmark]
public int Logic()
{
int res = 0;
for (int i = 0; i < N; i++)
res += data[i];
return res;
}

[GlobalCleanup]
public void GlobalCleanup()
{
// Disposing logic
}
}
}
34 changes: 34 additions & 0 deletions samples/BenchmarkDotNet.Samples/IntroSetupCleanupIteration.cs
@@ -0,0 +1,34 @@
using System;
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Engines;

namespace BenchmarkDotNet.Samples
{
[SimpleJob(RunStrategy.Monitoring, launchCount: 1,
warmupCount: 2, targetCount: 3)]
public class IntroSetupCleanupIteration
{
private int setupCounter;
private int cleanupCounter;

[IterationSetup]
public void IterationSetup()
=> Console.WriteLine($"// IterationSetup ({++setupCounter})");

[IterationCleanup]
public void IterationCleanup()
=> Console.WriteLine($"// IterationCleanup ({++cleanupCounter})");

[GlobalSetup]
public void GlobalSetup()
=> Console.WriteLine("// " + "GlobalSetup");

[GlobalCleanup]
public void GlobalCleanup()
=> Console.WriteLine("// " + "GlobalCleanup");

[Benchmark]
public void Benchmark()
=> Console.WriteLine("// " + "Benchmark");
}
}

0 comments on commit b16b83f

Please sign in to comment.