Scripts build a custom set of conda packages from a common environment and publish to a custom conda channel.
Originally developed to manage the dependency tree for ilastik, which we handle by using the conda package manager. The build process is automated by the scripts in this repository. For an example of a large package set which can be built using this tool, see ilastik-conda-recipes.
You have a list of conda recipes (spread across one or more git repos), and you want to make sure your channel has an up-to-date build of each corresponding package. For each of your recipes,
publish-conda-stack will perform the following steps:
- Clone the recipe repo to a local cache.
- Check out the desired tag (with submodules, if any).
- Determine the exact package version (and build string) that the recipe would produce, if it were built (via
- Check your channel to see if that exact version already exists.
- If it doesn't exist on your channel yet, build the recipe and upload the resulting package to your channel.
conda install -n base -c ilastik-forge -c conda-forge publish-conda-stack
This also installs the
publish-conda-stack main entry-point and makes it available in the respective conda environment.
conda install anaconda-client. You need to be logged in to your https://anaconda.org account by running
publish-conda-stack builds packages specified in a
yaml config file. An example:
# common configuration for all packages defined in shared-config: shared-config: # will translate to --python for every conda-build, deprecated, use pin-file python: '3.6' # will translate to --numpy for every conda-build, deprecated, use pin-file numpy: '1.13' # Path to store git repositories containing recipes. Relative to this yaml file's directory. repo-cache-dir: ./repo-cache # Optional path to master conda_build_config file to unify build-environment and package pins across recipes master-conda-build-config: ./my-pins.yaml # Channels to check for dependencies of the built package source-channels: - my-personal-channel - conda-forge # channel to upload recipes to destination-channel: my-personal-channel
recipe-specs: - name: PACKAGE_NAME recipe-repo: PACKAGE_RECIPE_GIT_URL tag: RECIPE_GIT_TAG_OR_BRANCH recipe-subdir: RECIPE_DIR_IN_GIT_REPO # Optional specs environment: MY_VAR: VALUE conda-build-flags: STRING_THAT_WILL_BE_APPENDED_TO_CONDA_BUILD # by default a package is built on all 3 platforms, you can restrict that by specifying the following build-on: - linux - win - osx - name: NEXT_PACKAGE ...
Optional Master conda-build-config
You can add a master
conda_build_config.yaml file that will be passed to
conda-build, hence, it supports all syntax as described in the
This field is optional.
If the field is left blank, it will be simply ignored.
- python: 3.6 pin_run_as_build: python: x.x
# on Linux and Windows: publish-conda-stack my-recipe-specs.yaml # on Mac: MACOSX_DEPLOYMENT_TARGET=10.9 publish-conda-stack my-recipe-specs.yaml
build-recipes.py script parses the packages from
my-recipe-specs.yaml, and for each package checks whether an up-to-date version is already available on the
destination-channel listed in
my-recipe-specs.yaml. If the packages don't yet exist in that channel set, it will build the package and upload it.