This tutorial will guide you through the process of using the spack develop
command to develop software from local source code within a spack environment. With this command spack will manage your dependencies while you focus on testing changes to your library and/or application.
The spack install
command, as you know, fetches source code from a mirror or the internet before building and installing your package. As developers, we want to build from local source, which we will constantly change, build, and test.
Let's imagine for a second we're working on scr
. scr
is a library used to implement scalable checkpointing in application codes. It supports writing/reading checkpoints quickly and efficiently using MPI and high-bandwidth file I/O. We'd like to test changes to scr within an actual application so we'll test with macsio
, a proxy application written to mimic typical HPC I/O workloads. We've chosen scr
and macsio
because together they are quick to build.
We'll start by making an environment for our development. We need to build macsio
with scr
support, and we'd like everything to be built without fortran support for the time being. Let's set up that development workflow.
outputs/dev/setup-scr.out
Before we do any work, we verify that this all builds. Spack ends up building the entire development tree below, and links everything together for you.
- digraph G {
labelloc = "b" rankdir = "TB" ranksep = "1" edge[ penwidth=4 ] node[ fontname=Monaco, penwidth=4, fontsize=24, margin=.2, shape=box, fillcolor=lightblue, style="rounded,filled" ]
"4sh6pymrm2ms4auu3ajbjjr6fiuhz5g7" [label="pkgconf"] "7tkgwjvu2mi4ea2wsdetunq7g4k4r2nh" [label="json-cwx"] "4ihuiazsglf22f3pntq5hc4kyszqzexn" [label="berkeley-db"] "jearpk4xci4zc7dkrza4fufaqfkq7rfl" [label="libiconv"] "d2krmb5gweivlnztcymhklzsqbrpatt6" [label="automake"] "gs6ag7ktdoiirb62t7bcagjw62szrrg2" [label="util-macros"] "yn2r3wfhiilelyulh5toteicdtxjhw7d" [label="libxml2"] "lbrx7lnfz46ukewxbhxnucmx76g23c6q" [label="libsigsegv"] "bob4o5m3uku6vtdil5imasprgy775zg7" [label="libpciaccess"] "pmsyupw6w3gql4loaor25gfumlmvkl25" [label="openmpi"] "mkc3u4x2p2wie6jfhuku7g5rkovcrxps" [label="m4"] "jdxbjftheiotj6solpomva7dowrhlerl" [label="libtool"] "mm33a3ocsv3jsh2tfxc4mlab4xsurtdd" [label="autoconf"] "zfdvt2jjuaees43ffrrtphqs2ky3o22t" [label="perl"] "t54jzdy2jj4snltjazlm3br2urcilc6v" [label="readline"] "4av4gywgpaspkhy3dvbb62nulqogtzbb" [label="gdbm"] "crhlefo3dv7lmsv5pf4icsy4gepkdorm" [label="ncurses"] "bltycqwh5oofai4f6o42q4uuj4w5zb3j" [label="cmake"] "zqwfzhw5k2ollygh6nrjpsi7u4d4g6lu" [label="hwloc"] "vedchc5aoqyu3ydbp346qrbpe6kg46rq" [label="hdf5"] "wbqbc5vw5sxzwhvu56p6x5nd5n4abrvh" [label="numactl"] "komekkmyciga3kl24edjmredhj3uyt7v" [label="xz"] "es377uqsqougfc67jyg7yfjyyuukin52" [label="openssl"] "vfrf7asfclt7epufnoxibfqbkntbk5k3" [label="silo"] "smoyzzo2qhzpn6mg6rd3l2p7b23enshg" [label="zlib"] "sz72vygmht66khd5aa4kihz5alg4nrbm" [label="macsio"]
"wbqbc5vw5sxzwhvu56p6x5nd5n4abrvh" -> "jdxbjftheiotj6solpomva7dowrhlerl" "zqwfzhw5k2ollygh6nrjpsi7u4d4g6lu" -> "4sh6pymrm2ms4auu3ajbjjr6fiuhz5g7" "sz72vygmht66khd5aa4kihz5alg4nrbm" -> "vfrf7asfclt7epufnoxibfqbkntbk5k3" "vfrf7asfclt7epufnoxibfqbkntbk5k3" -> "t54jzdy2jj4snltjazlm3br2urcilc6v" "crhlefo3dv7lmsv5pf4icsy4gepkdorm" -> "4sh6pymrm2ms4auu3ajbjjr6fiuhz5g7" "7tkgwjvu2mi4ea2wsdetunq7g4k4r2nh" -> "mkc3u4x2p2wie6jfhuku7g5rkovcrxps" "sz72vygmht66khd5aa4kihz5alg4nrbm" -> "pmsyupw6w3gql4loaor25gfumlmvkl25" "zqwfzhw5k2ollygh6nrjpsi7u4d4g6lu" -> "wbqbc5vw5sxzwhvu56p6x5nd5n4abrvh" "7tkgwjvu2mi4ea2wsdetunq7g4k4r2nh" -> "d2krmb5gweivlnztcymhklzsqbrpatt6" "es377uqsqougfc67jyg7yfjyyuukin52" -> "smoyzzo2qhzpn6mg6rd3l2p7b23enshg" "bltycqwh5oofai4f6o42q4uuj4w5zb3j" -> "crhlefo3dv7lmsv5pf4icsy4gepkdorm" "mm33a3ocsv3jsh2tfxc4mlab4xsurtdd" -> "zfdvt2jjuaees43ffrrtphqs2ky3o22t" "es377uqsqougfc67jyg7yfjyyuukin52" -> "zfdvt2jjuaees43ffrrtphqs2ky3o22t" "7tkgwjvu2mi4ea2wsdetunq7g4k4r2nh" -> "jdxbjftheiotj6solpomva7dowrhlerl" "mkc3u4x2p2wie6jfhuku7g5rkovcrxps" -> "lbrx7lnfz46ukewxbhxnucmx76g23c6q" "bltycqwh5oofai4f6o42q4uuj4w5zb3j" -> "es377uqsqougfc67jyg7yfjyyuukin52" "vedchc5aoqyu3ydbp346qrbpe6kg46rq" -> "smoyzzo2qhzpn6mg6rd3l2p7b23enshg" "wbqbc5vw5sxzwhvu56p6x5nd5n4abrvh" -> "d2krmb5gweivlnztcymhklzsqbrpatt6" "zfdvt2jjuaees43ffrrtphqs2ky3o22t" -> "4av4gywgpaspkhy3dvbb62nulqogtzbb" "vedchc5aoqyu3ydbp346qrbpe6kg46rq" -> "pmsyupw6w3gql4loaor25gfumlmvkl25" "d2krmb5gweivlnztcymhklzsqbrpatt6" -> "mm33a3ocsv3jsh2tfxc4mlab4xsurtdd" "bob4o5m3uku6vtdil5imasprgy775zg7" -> "jdxbjftheiotj6solpomva7dowrhlerl" "yn2r3wfhiilelyulh5toteicdtxjhw7d" -> "komekkmyciga3kl24edjmredhj3uyt7v" "pmsyupw6w3gql4loaor25gfumlmvkl25" -> "smoyzzo2qhzpn6mg6rd3l2p7b23enshg" "wbqbc5vw5sxzwhvu56p6x5nd5n4abrvh" -> "mm33a3ocsv3jsh2tfxc4mlab4xsurtdd" "vfrf7asfclt7epufnoxibfqbkntbk5k3" -> "vedchc5aoqyu3ydbp346qrbpe6kg46rq" "bob4o5m3uku6vtdil5imasprgy775zg7" -> "gs6ag7ktdoiirb62t7bcagjw62szrrg2" "d2krmb5gweivlnztcymhklzsqbrpatt6" -> "zfdvt2jjuaees43ffrrtphqs2ky3o22t" "7tkgwjvu2mi4ea2wsdetunq7g4k4r2nh" -> "mm33a3ocsv3jsh2tfxc4mlab4xsurtdd" "vfrf7asfclt7epufnoxibfqbkntbk5k3" -> "smoyzzo2qhzpn6mg6rd3l2p7b23enshg" "zfdvt2jjuaees43ffrrtphqs2ky3o22t" -> "4ihuiazsglf22f3pntq5hc4kyszqzexn" "bob4o5m3uku6vtdil5imasprgy775zg7" -> "4sh6pymrm2ms4auu3ajbjjr6fiuhz5g7" "vfrf7asfclt7epufnoxibfqbkntbk5k3" -> "pmsyupw6w3gql4loaor25gfumlmvkl25" "zqwfzhw5k2ollygh6nrjpsi7u4d4g6lu" -> "bob4o5m3uku6vtdil5imasprgy775zg7" "yn2r3wfhiilelyulh5toteicdtxjhw7d" -> "jearpk4xci4zc7dkrza4fufaqfkq7rfl" "sz72vygmht66khd5aa4kihz5alg4nrbm" -> "bltycqwh5oofai4f6o42q4uuj4w5zb3j" "pmsyupw6w3gql4loaor25gfumlmvkl25" -> "wbqbc5vw5sxzwhvu56p6x5nd5n4abrvh" "sz72vygmht66khd5aa4kihz5alg4nrbm" -> "7tkgwjvu2mi4ea2wsdetunq7g4k4r2nh" "yn2r3wfhiilelyulh5toteicdtxjhw7d" -> "smoyzzo2qhzpn6mg6rd3l2p7b23enshg" "t54jzdy2jj4snltjazlm3br2urcilc6v" -> "crhlefo3dv7lmsv5pf4icsy4gepkdorm" "pmsyupw6w3gql4loaor25gfumlmvkl25" -> "zqwfzhw5k2ollygh6nrjpsi7u4d4g6lu" "4av4gywgpaspkhy3dvbb62nulqogtzbb" -> "t54jzdy2jj4snltjazlm3br2urcilc6v" "jdxbjftheiotj6solpomva7dowrhlerl" -> "mkc3u4x2p2wie6jfhuku7g5rkovcrxps" "yn2r3wfhiilelyulh5toteicdtxjhw7d" -> "4sh6pymrm2ms4auu3ajbjjr6fiuhz5g7" "mm33a3ocsv3jsh2tfxc4mlab4xsurtdd" -> "mkc3u4x2p2wie6jfhuku7g5rkovcrxps" "zqwfzhw5k2ollygh6nrjpsi7u4d4g6lu" -> "yn2r3wfhiilelyulh5toteicdtxjhw7d" "pmsyupw6w3gql4loaor25gfumlmvkl25" -> "4sh6pymrm2ms4auu3ajbjjr6fiuhz5g7" "wbqbc5vw5sxzwhvu56p6x5nd5n4abrvh" -> "mkc3u4x2p2wie6jfhuku7g5rkovcrxps"
}
Now we are ready to begin work on the actual application.
Let's assume that scr has a bug, and we'd like to patch scr to find out what the problem is. First, we tell spack that we'd like to check out a development version of scr:
outputs/dev/develop-1.out
The spack develop command marks the package as being a "development" package in the spack.yaml. This adds a special dev_path=
attribute to the spec for the package, so spack remembers where the source code for this package is located. The develop command also downloads/checks out the source code for the package. By default, the source code is downloaded into a subdirectory of the environment. You can change the location of this source directory by modifying the path:
attribute of the develop configuration in the environment.
There are a two gotchas with the spack develop command
- You need to manually specify the package version when specifying a package as a dev package. Spack needs to know the version of the dev package so it can supply the correct flags for the package's build system.
- You'll need to re-concretize the environment so that the version number and the
dev_path=
attributes are properly added to the cached spec inspack.lock
.
outputs/dev/develop-conc.out
Now that we have this done, we tell spack to rebuild both scr
and macsio
by running spack install
.
outputs/dev/develop-2.out
This rebuilds scr
from the subdirectory we specified. If your package uses cmake, spack will build the package in a build directory that matches the hash for your package. From here you can change into the appropriate directory and perform your own build/test cycles.
Now, we can develop our code. For the sake of this demo, we're just going to intentionally introduce an error. Let's edit a file and remove the first semi-colon we find.
outputs/dev/edit-1.out
Once you have a development package, spack install
also works much like "make". Since spack knows the source code directory of the package, it checks the filetimes on the source directory to see if we've made recent changes. If the file times are newer, it will rebuild scr
and any other package that depends on scr
.
outputs/dev/develop-3.out
Here, the build failed as expected. We can look at the output for the build in scr/spack-build-out.txt
to find out why, or we can launch a shell directly with the appropriate environment variables to figure out what went wrong by using spack build-env scr@2.0 -- bash
. If that's too much to remember, then sourcing scr/spack-build-env.txt
will also set all the appropriate environment variables so we can diagnose the build ourselves. Now let's fix it and rebuild directly.
outputs/dev/develop-4.out
You'll notice here that spack rebuilt both scr
and macsio
, as expected.
Taking advantage of iterative builds with spack requires cooperation from your build system. When spack performs a rebuild on a development package, it reruns all the build stages for your package without cleaning the source and build directories to a pristine state. If your build system can take advantage of the previously compiled object files then you'll end up with an iterative build.
- If your package just uses make, you also should get iterative builds for free when running
spack develop
. - If your package uses cmake with the typical
cmake
/build
/install
build stages, you'll get iterative builds for free with spack because cmake doesn’t modify the filetime on theCMakeCache.txt
file if your cmake flags haven't changed. - If your package uses autoconf, then rerunning the typical
autoreconf
stage typically modifies the filetime ofconfig.h
, which can trigger a cascade of rebuilding.
Multiple packages can also be marked as develop. If we were co-developing macsio
, we could run
outputs/dev/develop-5.out
Using development workflows also lets us ship our whole development process to another developer on the team. They can simply take our spack.yaml, create a new environment, and use this to replicate our build process. For example, we'll make another development environment here.
outputs/dev/otherdevel.out
Here, spack develop
with no arguments will check out or download the source code and place it in the appropriate places.
When we're done developing, we simply tell spack that it no longer needs to keep a development version of the package.
outputs/dev/wrapup.out
Use the spack develop
command with an environment to make a reproducible build environment for your development workflow. Spack will set up all the dependencies for you and link all your packages together. Within a development environment, spack install
works similar to make
in that it will check file times to rebuild the minimum number of spack packages necessary to reflect the changes to your build.