A makefile used for running test executables.
Makefile.test can be used to run any type of test executables. It is not language specific nor it requires any changes to your code. Parallel, serial execution, various platforms and make versions are supported. The executables can be organized in any desired way. The user only lists the test files, the rest is taken care of Makefile.test.
Makefile.test does not contain any rules for compilation and other pre-processing steps. If your test executables are not scripts, but for example compiled binaries, you will need to extend Makefile.test with additional rules. Makefile.test can still be a good starting point for those scenarios.
Makefile.test runs on a single host and therefore its parallelization is limited with the resources of one machine. If your test suite requires multiple hosts to run, ClusterRunner can be a better tool for your use case.
Example: A repo that has a
src and a
A simple repository has a
src and a
test directory at its root. The
programmer places application code in
src and test executables in
Makefile.test, the executables in
test can be executed with ease.
The directory structure can look like the following:
ExampleRepo ├── Makefile.test ├── src │ └── ExampleApplication.sh └── test ├── ExampleTest1.sh ├── ExampleTest2.py └── Makefile
For a recommended way to place the Makefile.test into your own repo see the Installation section next.
Makefile file in the
test directory needs to list the executables in a
TESTS variable and include the
TESTS ?= \ ExampleTest1.sh \ ExampleTest2.py include ../Makefile.test
To run the tests, any of the following can be used from the repo root.
cd test && make -j make -C test -j make -f test/Makefile -j
The output looks as follows:
[ExampleTest1.sh] Running ExampleTest1 [ExampleTest2.py] Running ExampleTest2 PASSED: ExampleTest1.sh PASSED: ExampleTest2.py --------------------------------- All 2 tests passed ---------------------------------
Running one test at a time.
During development or debugging time, you may want to execute only one test at a time. In order to achive that without modifying any files, overwrite the TESTS environment variable from the command line:
Only runs the specified test:
[ExampleTest2.py] Running ExampleTest2 PASSED: ExampleTest2.py --------------------------------- All 1 tests passed ---------------------------------
bashneeds to be installed at
Using git submodules and symlink to the Makefile.test.
In the directory you want to place Makefile.test execute the following:
git submodule add https://github.com/box/Makefile.test.git .Makefile.test ln -s .Makefile.test/Makefile.test
First command creates a hidden dir with the submoduled repo. Second command symlinks the Makefile.test file.
The directory tree of
ExampleRepo with the submodule and the symlink looks like
ExampleRepo ├── .Makefile.test │ ├── ..... │ └── Makefile.test ├── Makefile.test -> .Makefile.test/Makefile.test ├── src │ └── .... └── test └── ....
In order to avoid temporary files that may be created by Makefile.test, you should
to update your
# Intermediate files created by Makefile.test **/.makefile_test_*ed_tests
If hung tests are encountered, one may want to kill the
For SIGTERM, the user should send SIGTERM to the process group of
something similar to:
kill -s TERM -- -<pgrp id of make>
If SIGTERM is only sent to
make child processes will be orphaned and left behind.
make is invoked interactively from a terminal,
CTRL-C should kill all running
Need to contact us directly? Email email@example.com and be sure to include the name of this project in the subject.
Copyright and License
Copyright 2017 Box, Inc. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.