mystack.py

mystack.py

Script format

As the low-level stringing-together build mechanism, we have an API that allows writing Python scripts like this ("mystack.py"):

system = hashdist.AlreadyInstalledOnSystemSource() spkg = hashdist.SpkgPackageProvider('/home/dagss/...') pypi = hashdist.PyPIPackageProvider('http://sciencepypi')

# Lacking: An environment mechanism like waf and scons

blas = spkg.reference_blas()

ATLAS = spkg.ATLAS() ATLAS = system.ATLAS('/sysadmins/stupid/path/for/ATLAS')

petsc = pkgs.petsc(blas=ATLAS, compiler=intel) numpy = pkgs.numpy(blas=ATLAS, compiler=intel, CFLAGS='-O0')

mypackage = pkgs.mypackage(use_frobnicator=True,

CFLAGS='-O3', numpy=numpy)

profile = hashdist.profile([petsc, numpy])

hashdist.distribution_command_line(profile,

distro_dir='/home/dagss/my_hashdist')

Note that this scripts embed a specific, chosen configuration. For a final distribution system, this file would either be generated, or just the same actions would be specified from higher-level configuration files. However, this format allows for rapid development and testing.

For Python-HPCMP, for instance, one could have one such script per cluster configuration to replace the current setup.

Script commands

The script can be invoked like this:

python mystack.py shell

to get a shell with the profile prefixed to PATH.

Command list:

• shell - Described above
• bacspec numpy - Get the spec.json to build numpy with BAC using this configuration
• debug numpy - Get a shell that drops into the build directory after BAC has done its unpacking/copying/environment setup; then echo the commands that would have been run to the console; and ask the user to investigate