Skip to content
A simple benchmarking and comparison tool
C# F# JavaScript TypeScript Python CSS Other

Mono Benchmarker

Build Status Go Report Card

Running benchmarks locally


brew install lynx jq


cd tools
nuget restore tools.sln
xbuild /target:compare


There are two ways to run the benchmarking suite locally, one higher-level than the other. If you want to benchmark finished Mono packages from a build bot, you should use If you want to benchmark a Mono that you've built on your own machine, use the lower-level compare.exe.

From a package

tools/ requires at least three pieces of information, two of which serve as documentation and for reproducibility:

  • The file or URL of the Mono package you want to benchmark
  • The URL of the build page for that package
  • The mono Git repository commit of that package

Example, on OS X:

./tools/ --commit 2d4f6b344205b410f21a3470d4152d78a28964d1 \
               --build-url '' \
               --pkg-url ''

It will ask for your password when it tries to install the package. The package is installed in a temporary disk image, so your system will not be affected.

On Linux, requires that you specify .deb package URLs. --help will tell you how.

From a local build

To run the suite on a locally built Mono, use tools/compare.exe:

mono tools/compare.exe --root <MONO-INSTALL-ROOT>

Your Mono executable must be in MONO-INSTALL-ROOT/bin/mono-sgen. The benchmark runner will try to figure out the Git commit of your Mono. If that fails, use the --commit and --git-repo options to help it.

Once all benchmarks have been run and the results uploaded it will print a URL that shows all the results.

Setting options

To give options to Mono or set environment variables you need to use a configuration files. Those files are kept in the config directory. The default configuration is default-sgen.conf. Make a copy of it and rename it, to avoid confusion and for documentation.


Each Mono configuration requires a .conf file. The files in the configs directory are examples. The JSON structure is as follows:

  • Name: name of the config (must be unique across all configs and benchmarks)
  • Count: number of time to run the benchmark (optional, default: 10)
  • Mono: path to the mono executable (optional, default to system one)
  • MonoOptions: command line parameters to pass to the mono runtime (optional)
  • MonoEnvironmentVariables: environment variables to set to run the benchmark (optional)
  • UnsavedMonoEnvironmentVariables: environment variables to set to run the benchmarks, but not saved in the database (optional)


Each benchmark requires a .benchmark file. The files in the benchmarks directory are examples. The JSON structure is as follows:

  • Name: name of the benchmark (must be unique across all configs and benchmarks)
  • TestDirectory: path to the working directory to run the benchmark, relative to the benchmarker repository root directory
  • CommandLine: command line to run the benchnark, does not contain the runtime (mono) executable
  • Timeout: benchmark specific timeout, override the command line one

Building the front-end


brew install npm
npm -g install webpack


make -C front-end
Something went wrong with that request. Please try again.