Setting up your own easyblocks repository
Clone this wiki locally
Note: using the
--include-easyblocks configure option is highly recommended over manually setting up an easyblocks repository, see http://easybuild.readthedocs.org/en/latest/Including_additional_Python_modules.html
EasyBuild allows you to host your own set of easyblocks, next to the ones that ship with the EasyBuild version you have installed.
You can provide these easyblocks to EasyBuild by simply making sure your Python path is set correctly.
EasyBuild will then use these easyblocks when needed, i.e., when requested to build/install software that require them.
Setting up your own easyblocks repository
The following steps outline how to set up your own easyblocks repository. Of course, you will only need to do this procedure once on your system.
We present the path in which will host your set of easyblocks by the environment variable $MYEBDIR, e.g.,
Step 1: Directory structure
In your easyblocks repository, you need to make sure you adhere to the EasyBuild namespace, i.e.,
a top-level directory
easybuild with a subdirectory
mkdir -p $MYEBDIR/easybuild/easyblocks
The Python modules representing your easyblocks should then be hosted directly in the
directory (not in subdirectories like
b, etc., see below why). For example:
$ ls $MYEBDIR/easybuild/easyblocks bar.py foo.py linuxfromscratch.py
Step 2: Setting up Python packages
easybuild/easyblocks path should be a Python package, so it should contain
__init__.py file. This file can, and should in this case, be empty; see the note on searching
Python modules in Python packages for the reason why they should be empty.
Step 3: Setting Python path
After creating your easyblocks repository, all you need to do is make sure that EasyBuild is able to find your easyblocks, by adding the path to the Python path.
# extend path for easyblocks for EasyBuild (pro tip: add this to your .bashrc) export PYTHONPATH=$PYTHONPATH:$MYEBDIR
Note: Add the path to your easyblocks at the end, not at the beginning, to avoid messing with the
easybuild namespace (see
below as to why).
The reason why some notes were made in the steps above is the way in which searching for Python modules
in the Python path, and some custom hacking in
__init__.py of the
easyblocks package that comes
with an EasyBuild installation.
Flattening the easyblocks namespace in EasyBuild
We organise Python modules representing easyblocks in subdirectories by their first letter, i.e.:
$ ls easybuild/easyblocks/*/* easybuild/easyblocks/a/acml.py easybuild/easyblocks/a/armadillo.py easybuild/easyblocks/a/atlas.py easybuild/easyblocks/b/boost.py ...
__init__.py file, we then flatten this again as if all these Python modules were located directly
easybuild/easyblocks namespace. That allows for cleaner import statements, i.e.:
from easybuild.easyblocks.atlas import EB_ATLAS
Package initialisation by Python
When Python wanders through the Python path, it will initialise any Python package it comes across by
__init__.py file. And here's the catch: Python will only actually execute the first
file it comes across for each package.
That means that if a Python package is spread across multiple directories, as is the case with the
easybuild/easyblocks path in the EasyBuild installation path and your own set of easyblocks in a custom directory,
you need to make sure the right
__init__.py file gets executed.
That is the reason for adding the path to your set of easyblocks at the end of
PYTHONPATH, and why your
files should be empty (they won't get executed anyway, so why bother putting something in there). It also explains why
you should place your easyblocks directly in the
easybuild/easyblocks subdirectory of your easyblocks repository,
since you can only do the flattening trick once, and EasyBuild relies on the fact that it is able to import the easyblock
for software package Foo with:
from easybuild.easyblocks.foo import EB_Foo
When you start working on a new easyblock, you have a couple of options:
- copying and adjusting an existing easyblock, e.g., one shipped with EasyBuild that shows similarities with the support you need to implement
- starting from scratch, and write an entirely new easyblock (good exercise, but time-consuming to do it every time)
- using the
mk_tmpl_easyblock_for.pyscript provided with EasyBuild (available as of EasyBuild v1.1 in the
easybuild/scriptssubdirectory) to generate a template easyblock specific to the software package you want to support
We recommend the last option, since that will allow you to start with a clean slate and limit the amount of things to get right at the same time.
Once you finish the work on an easyblock, i.e., it allows you to build a previously unsupported software package and (preferably) you have done some effort to verify the build, we encourage you to contribute the easyblock and at least one example easyconfig back to EasyBuild.
See the wiki page on Contributing back for more information.