Permalink
Switch branches/tags
Nothing to show
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
136 lines (106 sloc) 4.5 KB
title description original-date date category obsolete
Benchmarking the Dart VM
Learn how to benchmark your applications
2012-10-01
2018-07-26
dart-vm
true

Written by John McCutchan
October 2012 (updated November 2012; note updated July 2018)

This article still has valid information, but it was initially targeted for the standalone VM under the 1.x Dart SDK. It does not reflect Flutter, nor does it reflect changes in [Dart 2](/dart-2). For example, this article mentions Dartium, which [doesn't work with Dart 2.]({{site.webdev}}/dart-2#tools) Also, checked mode and production mode no longer exist; development modes vary by platform, as you can see from the [assert section](/guides/language/language-tour#assert) of the language tour. For information about Flutter benchmarking, see the [Flutter performance page.]({{site.flutter}}/ui-performance/)

Programmers often create benchmarks that exercise an important algorithm in a larger application. The point of the benchmark is to have an easy to run, reproducible stress test of the performance-sensitive algorithm. The benchmark verifies that changes to the algorithm are indeed improvements, and not regressions.

Benchmarking Dart

Dart can go really fast, but you have to give the VM time to optimize your code. Most benchmarks we've seen are short and to the point—so short that they don’t trigger the VM’s optimizer. This isn’t an issue with real-world applications, which execute long enough to be optimized. Read on to find out how to use the benchmark_harness library to properly run a Dart benchmark.

Use the benchmark harness

The Dart team has provided an official benchmark harness that ensures your benchmark follows the benchmarking procedures necessary for the Dart VM's optimizer.

The harness is available as a pub package and is incredibly easy to use.

  1. Add the following to your pubspec.yaml, and run `pub get`:

    {% prettify yaml %} dependencies: benchmark_harness: ">=1.0.0 <2.0.0" {% endprettify %}

  2. Copy the following template, which creates a class extending `BenchmarkBase`:

    {% prettify dart %} // Import BenchmarkBase class. import 'package:benchmark_harness/benchmark_harness.dart';

    // Create a new benchmark by extending BenchmarkBase. class TemplateBenchmark extends BenchmarkBase { const TemplateBenchmark() : super("Template");

    static void main() { new TemplateBenchmark().report(); }

    // The benchmark code. void run() { }

    // Not measured: setup code executed before the benchmark runs. void setup() { }

    // Not measured: teardown code executed after the benchmark runs. void teardown() { } }

    // Main function runs the benchmark. main() { // Run TemplateBenchmark. TemplateBenchmark.main(); } {% endprettify %}

Output

The benchmark will output something like the following:

Template(RunTime): 0.1568472448997197 µs.
**Note:** µs is an abbreviation for microseconds. A microsecond is one millionth of a second.

Run in production mode

The Dart VM can run in two modes: checked and production mode. Checked mode is slower because the VM is checking types at runtime. Before benchmarking make sure that your code runs without issue in checked mode. If checked mode finds an issue, it will likely cause a performance problem in production mode. After making sure your program is correct, you should run your benchmark in production mode to get an accurate measurement of real world performance.

When executing from the command line checked mode is off by default and can be turned on by passing the --checked command line flag. WebStorm has checked mode on by default but can be turned off by going to Preferences > Languages & Frameworks > Dart and toggling the Start Dartium in checked mode checkbox.

Summary

When benchmarking your application be sure to follow these three rules:

  1. Use the official benchmarking harness.
  2. Ensure the code does not raise any errors when run in checked mode.
  3. Run your benchmark in production mode.

If you follow these rules you will be able to accurately measure how fast your code runs. Once you've sped up your application, share your secrets on the mailing list. Happy benchmarking!