Build systems are software tools designed to automate the process of program compilation. Some commonly used build systems include:
Flow Diagram of GNU Build System
Build System Philosophy
At their core, build systems are a functional based languages mapping a set of source resources (in most cases, files) to a target (executable). The primary assumption of the build system is each invocation of a build command with the same input and options will create the same output. This assumption allows the build system to memoize the actions that is has already performed, and only perform build actions on resources that have changed.
How Build System Figures Out Dependencies
For a build system to correctly do partial rebuilds of only the components that have changed, it must fully understand the dependencies of the build. Generally, there are two methods to specify the dependencies: implicitly or explicitly.
Explicit dependencies are the most straight forward to implement, the relationship is explicitly encoded in the build script. Most build systems have the ability to explicitly specify a dependency, even if they try to find the majority of dependencies implicitly.
Example : Makefile
edit : main.o kbd.o command.o display.o \ insert.o search.o files.o utils.o cc -o edit main.o kbd.o command.o display.o \ insert.o search.o files.o utils.o main.o : main.c defs.h cc -c main.c kbd.o : kbd.c defs.h command.h cc -c kbd.c command.o : command.c defs.h command.h cc -c command.c display.o : display.c defs.h buffer.h cc -c display.c insert.o : insert.c defs.h buffer.h cc -c insert.c search.o : search.c defs.h buffer.h cc -c search.c files.o : files.c defs.h buffer.h command.h cc -c files.c utils.o : utils.c defs.h cc -c utils.c clean : rm edit main.o kbd.o command.o display.o \ insert.o search.o files.o utils.o
Also see our ISC'17 application miniDFT's makefile.
Build systems find implicit dependencies a number of ways. First is by file association. Most build tools use common file extensions to identify file types. So, given the file
hello.c many build systems will infer a build dependency from
Further, additional dependencies can be inferred from contents of the file. The build system can scan the
.c file for
#include directives, and then add a new dependency from associated header file to the source file. This technique does not always work, however, because include directives can be guarded by the preprocessor. In general it is undecidable to discover whether a source file will actually include a given header.
Example : CMake
CMake is a system to generate make files based on the platform (i.e. CMake is cross platform) which you can then
make using the generated makefiles.
MAKE_DEPENDENT_OPTION(NEKTAR_SOLVER_ADR "Build the ADR Solver." ON "NEKTAR_BUILD_SOLVERS" OFF) IF (NOT NEKTAR_BUILD_SOLVERS) SET(NEKTAR_SOLVER_ADR OFF CACHE INTERNAL "") ENDIF() IF( NEKTAR_SOLVER_ADR ) ADD_SOLVER_EXECUTABLE(ADRSolver SOURCES ./ADRSolver.cpp ./EquationSystems/Helmholtz.cpp ./EquationSystems/Laplace.cpp ./EquationSystems/Poisson.cpp ./EquationSystems/Projection.cpp ...
Example taken from ISC'18 application Nektar++.
Popular Build Systems
Here comes some examples:
Make allows for explicit mapping between target files and source files/dependencies (foo.o: foo.c), as well as general mappings (%.o: %.c). Make also allows for "phony" target, which where the command executed does not actually create the target. This allows Makefiles to perform additional functionality. Here is the simple hello example in GNU Make:
OBJS=hello.o hello_io.o CC=gcc hello: $(OBJS) $(CC) -o $@ $(OBJS) %.o: %.c $(CC) -c $<
Scons is a python-based build tool, focusing on flexibility and scripting. Scons SConstruct files are interpreted as python scripts. Here is the hello build in Scons:
env = Environment() env.Program(target="hello", source=["hello.c", "hello_io.c"])
Scons is focused on flexibility and platform independence, and the cost of performance. Because scons is based on a general-purpose language it offers a wide range of extensible features, including custom builders and file scanning.
CMake is "cross platform make." CMake specifics dependencies in it's own configuration file, which is then used to create "native" makefile in the target environment. Here is the
CMakeLists.txt for the example:
add_executable (hello hello.c hello_io.c)
Autoconf is a tool for producing shell scripts that automatically configure software source code packages to adapt to many kinds of Posix-like systems. The configuration scripts produced by Autoconf are independent of Autoconf when they are run, so their users do not need to have Autoconf.
The configuration scripts that Autoconf produces are by convention called
configure. When run,
configure creates several files, replacing configuration parameters in them with appropriate values.
Example : PETsc
./configure --with-blas-lib=/path/to/BLAS/lib/libopenblas.a \ --with-lapack-lib=/path/to/lapack/lib/libscalapack.a \ --with-cc=mpicc --with-cxx=mpicxx --with-fc=mpif90
- Login to NSCC with NTU account
- Submit an interactive job request of 1.5h with 12 CPU cores
# this command may take a while $ qsub -I -l select=1:ncpus=12 -P personal -l walltime=01:30:00
- After the job starts, issue the following commands if you do have a copy of this repo on NSCC (otherwise, go to step 4):
# ssh into nscc04 $ ssh nscc04-ib0 $ git clone https://github.com/ntuhpc/training-ay1819.git
- Go to directory for this workshop
$ cd 2-Build_System
Hands-on Session ：MrBayes
MrBayes is a program for Bayesian inference and model choice across a wide range of phylogenetic and evolutionary models. MrBayes uses Markov chain Monte Carlo (MCMC) methods to estimate the posterior distribution of model parameters.
We will use MrBayes as an example to illustrate the use of build system.
# download source code $ wget https://github.com/NBISweden/MrBayes/archive/v3.2.6.tar.gz # prepare directory and go to source code directory $ tar zxvf v3.2.6.tar.gz $ cd MrBayes-3.2.6 $ tar zxvf mrbayes-3.2.6.tar.gz && cd mrbayes-3.2.6/src # load intel compiler and mpi library $ module load intelcc intelmpi
Then take a look at
# config the build $ autoconf
Notice that 4 new files are generated, namely
autom4te.cache(storing additional info for future use),
config.h.in. Take a look at them, think about the relationship with help of the chart above.
$ mkdir install $ ./configure --enable-mpi=yes --without-beagle --prefix=/path/to/your/MrBayes/install CC=mpiicc CXX=icpc # build MrBayes in parallel. may also set a value to -j like -j8 $ make install -j