Real-world tests to benchmark ITK performance.
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.circleci
CMake
doc
examples
include
src
test
.gitattributes
.gitignore
CMakeLists.txt
CTestConfig.cmake
LICENSE
README.rst
RunWithVisualStudio.cmd
evaluate-itk-performance.py
itk-module.cmake

README.rst

ITKPerformanceBenchmarking

https://circleci.com/gh/InsightSoftwareConsortium/ITKPerformanceBenchmarking/tree/master.svg?style=svg

Real-world tests to benchmark ITK performance.

New classes increase operating system process priority to minimize the impact of other processes running on the system.

These classes are used by a suite of example ITK benchmarks to quantify toolkit performance.

For more information, see the Insight Journal article:

McCormick M., Kang H.J., Barre S.
Performance Benchmarking the Insight Toolkit
The Insight Journal. January-December. 2016.
http://hdl.handle.net/10380/3557
http://insight-journal.org/browse/publication/972

Since ITK 4.11.0, this module is available in the ITK source tree as a Remote module. To enable it, set:

Module_PerformanceBenchmarking:BOOL=ON

in ITK's CMake build configuration.

# Driving performance benchmarks for a given ITK source tree

See ITK_PerformanceTestingDriver.sh in this directory

Requirements

Notes for running the benchmarks

An example to call the benchmarking script is:

$ python ./evaluate-itk-performance.py run -g {ITK-version} {ITK-source} {ITK-build} {ITKPerformanceBenchmarking-build}

where {ITK-version} is the ITK version that the user wishes to evaluate.

Also, note that the ITK source folder (ITK-source}, where the specific version is fetched, needs to exist.

The generated/result JSON files are placed in:

./{ITKPerformanceBenchmarking-build}/BenchmarkResults/{machine-name}

Notes for benchmarking in Windows

CMake and Ninja need to be in the PATH. Also, the C++ compiler has to be in the PATH so that Ninja can find it.

For the Microsoft Visual Studio compiler, the C++ compiler is a file that is usually under:

C:/Program Files/Microsoft Visual Studio {version}/VC/bin/cl.exe

The user will need to change the path to the vcvars*.bat command file in the RunWithVisualStudio.cmd command line script to the specific location of their vcvars file, e.g.:

C:/Program Files/Microsoft Visual Studio {version}/VC/vcvarsall.bat

Finally, the user will need to start the Git bash by double-clicking on the RunWithVisualStudio.cmd script.

Note that the module is built with static libraries to allow for ITK benchmarking in Windows: the BUILD_SHARED_LIBS flag in the evaluate-itk-performance.py script is set to OFF.

License

This software is distributed under the Apache 2.0 license. Please see the LICENSE file for details.