A makefile used for running test executables.
Makefile.test can be used to run any type of test executables. It is not language specific nor it requires any changes to your code. Parallel, serial execution, various platforms and make versions are supported. The executables can be organized in any desired way. The user only lists the test files, the rest is taken care of Makefile.test.
Makefile.test does not contain any rules for compilation and other pre-processing steps. If your test executables are not scripts, but for example compiled binaries, you will need to extend Makefile.test with additional rules. Makefile.test can still be a good starting point for those scenarios.
Makefile.test runs on a single host and therefore its parallelization is limited with the resources of one machine. If your test suite requires multiple hosts to run, ClusterRunner can be a better tool for your use case.
A simple repository has a src and a test directory at its root. The
programmer places application code in src and test executables in test.
Using the Makefile.test, the executables in test can be executed with ease.
The directory structure can look like the following:
ExampleRepo
├── Makefile.test
├── src
│ └── ExampleApplication.sh
└── test
├── ExampleTest1.sh
├── ExampleTest2.py
└── Makefile
For a recommended way to place the Makefile.test into your own repo see the Installation section next.
The Makefile file in the test directory needs to list the executables in a
TESTS variable and include the Makefile.test
TESTS ?= \
ExampleTest1.sh \
ExampleTest2.py
include ../Makefile.test
To run the tests, any of the following can be used from the repo root.
cd test && make -j
make -C test -j
make -f test/Makefile -j
The output looks as follows:
[ExampleTest1.sh] Running ExampleTest1
[ExampleTest2.py] Running ExampleTest2
PASSED: ExampleTest1.sh
PASSED: ExampleTest2.py
---------------------------------
All 2 tests passed
---------------------------------
During development or debugging time, you may want to execute only one test at a time. In order to achive that without modifying any files, overwrite the TESTS environment variable from the command line:
TESTS=ExampleTest2.py make
Only runs the specified test:
[ExampleTest2.py] Running ExampleTest2
PASSED: ExampleTest2.py
---------------------------------
All 1 tests passed
---------------------------------
bashneeds to be installed at/bin/bash.
In the directory you want to place Makefile.test execute the following:
git submodule add https://github.com/box/Makefile.test.git .Makefile.test
ln -s .Makefile.test/Makefile.test
First command creates a hidden dir with the submoduled repo. Second command symlinks the Makefile.test file.
The directory tree of ExampleRepo with the submodule and the symlink looks like
this:
ExampleRepo
├── .Makefile.test
│ ├── .....
│ └── Makefile.test
├── Makefile.test -> .Makefile.test/Makefile.test
├── src
│ └── ....
└── test
└── ....
In order to avoid temporary files that may be created by Makefile.test, you should
to update your .gitignore file.
# Intermediate files created by Makefile.test
**/.makefile_test_*ed_tests
If hung tests are encountered, one may want to kill the make execution.
For SIGTERM, the user should send SIGTERM to the process group of make. Using
something similar to:
kill -s TERM -- -<pgrp id of make>
If SIGTERM is only sent to make child processes will be orphaned and left behind.
If make is invoked interactively from a terminal, CTRL-C should kill all running
processes cleanly.
Need to contact us directly? Email oss@box.com and be sure to include the name of this project in the subject.
Copyright 2017 Box, Inc. All rights reserved.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.