diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 3748cc7bdd..6f52d8afa2 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,7 +1,3 @@ # Contributing to bioconda -Contibuting to bioconda is easy. Just create a pull request. -When writing a recipe, ensure to read the [guidelines](GUIDELINES.md) before submitting a PR. If you don't have already, consider becoming part of our comminity by requesting access in [issue #1](https://github.com/bioconda/bioconda-recipes/issues/1). - -* If the PR adds a new recipe, please mention `@bioconda/core` to get a review. -* If the PR just updates an existing recipe, feel free to merge without asking for review. +Please visit https://bioconda.github.io/contributing.html for the updated docs. diff --git a/FAQs.md b/FAQs.md index 73fa4cdbfc..9b75b2e1e2 100644 --- a/FAQs.md +++ b/FAQs.md @@ -1,176 +1,3 @@ # FAQs - - -## How do I get set up to test recipes locally? - -### One-time setup - -#### Install prerequisites - -- Install conda (use the Python 3 version). - -- Install conda-build in the root environment. - -- Install Docker (optional, but allows you to simulate most closely the Travis-CI tests). - -- Identify the branch of bioconda-utils currently used. It is indicated in -`bioconda-recipes/.travis.yml` (look for the `BIOCONDA_UTILS_TAG` env var). -Export that as an environment variable: - -``` -export BIOCONDA_UTILS_TAG= -``` - -- Install the requirements for that version of bioconda-utils: - -``` -conda install --file https://raw.githubusercontent.com/bioconda/bioconda-utils/${BIOCONDA_UTILS_TAG}/conda-requirements.txt -``` - -- Install that version of bioconda-utils: - -``` -pip install git+https://github.com/bioconda/bioconda-utils.git@${BIOCONDA_UTILS_TAG} -``` - - -#### Git and GitHub setup - -- Create a fork of bioconda-recipes on GitHub and clone it locally. Even if you -are a member of the bioconda team with push access, this will allow testing of -your recipes on travis-ci without consuming resources allocated by travis-ci to -the `bioconda` group: - -``` -git clone https://github.com//bioconda-recipes.git -``` - -- Add the main bioconda-recipes repo as an upstream remote: - -``` -git remote add upstream https://github.com/bioconda/bioconda-recipes.git -``` - -- Connect the fork to travis-ci, following steps 1 and 2 from the [travis-ci -docs](https://docs.travis-ci.com/user/getting-started/#To-get-started-with-Travis-CI%3A) - - -### Write a recipe - -Check out a new branch (e.g., `git checkout -b my-recipe`) and write one or -more recipes. - -See the [conda docs](http://conda.pydata.org/docs/building/recipe.html) for -writing recipes, and see `GUIDELINES.md` in the bioconda-recipes repo for more -info. - -### Test locally -The `simulate-travis.py` script reads the config files in the repo and sets -things up as closely as possible to how the builds will be run on travis-ci. It -should be run in the top-level dir of the repo. Any arguments are passed on to -the `bioconda-utils build` command, so check `bioconda-utils build -h` for help -and more options. - -Some example commands: - -This tests everything, using the installed conda-build. It will check all -recipes to see what needs to be built and so it is the most comprehensive: - -``` -./simulate-travis.py -``` - -Same thing but using `--docker`. If you're on OSX and have docker installed, -you can use this to test the recipe under Linux: - -``` -./simulate-travis.py --docker -``` - -Use the `--quick` option which will just check recipes that have changed since -the last commit to master branch or that have been newly removed from any -configured blacklists: - -``` -./simulate-travis.py --docker --quick -``` - -Or specify exactly which packages you want to try building. Note that the -arguments to `--package` can be globs and are of package *names* rather than -*paths* to recipe directories. For example, to consider all R and Bioconductor -packages: - -``` -./simulate-travis.py --docker --package r-* bioconductor-* -``` - - -### Push to fork, wait for Travis-CI, submit pull request. - -Push your changes to your fork on github, and watch the Travis-CI logs. Keep -making changes on your fork and pushing them until the travis-ci builds pass. - -Open a [pull request](https://help.github.com/articles/about-pull-requests/) on -the bioconda-recipes repo. If it's your first recipe or the recipe is doing -something non-standard, please ask `@bioconda/core` for a review. - -### Use your new recipe - -When the PR is merged with the master branch, travis-ci will again do the -builds but at the end will upload the packages to anaconda.org. Once the merge -build completes, your new package is installable by anyone using: - -``` -conda install my-package-name -c bioconda -``` - - -## How is Travis-CI set up and configured? - -- `.travis.yml` is read by the Travis-CI worker. - -- the worker runs `scripts/travis-setup.sh`, which installs conda, adds channels, and installs `bioconda-utils` - -- the worker runs `scripts/travis-run.sh`. If the system is Linux, then the -build is performed in a docker container (the one listed in `.travis.yml`). If -OSX, then the build is performed without docker. - -- Only if the build is on the master branch will it be uploaded to anaconda.org -## What does "SUBDAG" mean on Travis-CI? - -We have limited resources on Travis-CI on which to build packages. In an -attempt to speed up builds, we split the full DAG of recipes that need to be -built in to multiple independent sub-DAGs. These are named `SUBDAG 0` and -`SUBDAG 1`. Each sub-DAG is considered an independent build and they are built -in parallel. If you submit a single recipe, which sub-DAG it is built on is -nondeterministic so if you don't see log output for the recipe in one sub-DAG, -check the other. - -## How are environmental variables defined and used? - -It is possible to use jinja2 templating in recipes to use a uniform set of -versions for core packages used by bioconda packages, for example see [this -`meta.yaml`](https://github.com/bioconda/bioconda-recipes/blob/f5eb63e30a76fd13c28663786d219c9f7750267c/recipes/gfold/meta.yaml). - -This works by the following: - -- [`config.yml` indicates an -`env_matrix`](https://github.com/bioconda/bioconda-recipes/blob/0be2881ef95be68feb09fae7814e0217aca57285/config.yml#L1) -in which CONDA_GSL is defined - -- when figuring out which recipes need to be built, the filtering step attaches -each [unique env to a Target -object](https://github.com/bioconda/bioconda-utils/blob/63aacdedf583b6cda770c84f241120d48dfb77f5/bioconda_utils/utils.py#L511) -(e.g., `CONDA_GSL=1.6; CONDA_PY=27, CONDA_R=3.3.1;` etc} -- that [env is provided to the build -function](https://github.com/bioconda/bioconda-utils/blob/63aacdedf583b6cda770c84f241120d48dfb77f5/bioconda_utils/build.py#L312) -which is either - - sent [directly to - docker](https://github.com/bioconda/bioconda-utils/blob/63aacdedf583b6cda770c84f241120d48dfb77f5/bioconda_utils/build.py#L81) - - or used to [temporarily update - os.environ](https://github.com/bioconda/bioconda-utils/blob/63aacdedf583b6cda770c84f241120d48dfb77f5/bioconda_utils/build.py#L94) - so that conda-build sees it. -- These environmental variables are then [used over in -conda-build](https://github.com/conda/conda-build/blob/9ace97969e77a10afc3c03022f8a1d0f47c2a043/conda_build/jinja_context.py#L205) -to fill in the templated variables via jinja2. +Please visit https://bioconda.github.io/faqs.html for the new FAQs page. diff --git a/GUIDELINES.md b/GUIDELINES.md index 6337e01b14..e08f73768f 100644 --- a/GUIDELINES.md +++ b/GUIDELINES.md @@ -1,363 +1,3 @@ # Guidelines for `bioconda` recipes -## Requirements of a good `bioconda` recipe - -- Follows the Filesystem Hierarchy Standard - ([FSH](https://en.wikipedia.org/wiki/Filesystem_Hierarchy_Standard)). This is - most important for libraries and Java packages; for these cases use one of the - recipes below as a guideline. - -- Has adequate tests (see "Testing" section below) - -- License allows redistribution and license is indicated in meta.yaml - -- Does not already exist in the main Anaconda channel or the `r` channel. - Exceptions: Anaconda version is too old or some sort of patch is required that - is not in the main Anaconda channel (TODO: examples of the latter?) - -- If the recipe installs custom wrapper scripts, usage notes should be added to ``extra -> notes`` in the `meta.yaml`. - -- If uploading of an unreleased version is necessary, please follow the versioning scheme of conda for pre- and post-releases (e.g. using a, b, rc, and dev suffixes, see [here](https://github.com/conda/conda/blob/d1348cf3eca0f78093c7c46157989509572e9c25/conda/version.py#L30)). - -## Examples - -The following recipes serve as examples of good recipes that can be used as -guides or templates when developing new recipes. - -### Python - -Use `conda skeleton pypi ` where `packagename` is a package -available on PyPI. Either run that command in the `recipes` dir or move the -recipe it creates to `recipes`. Typically can be used without modification, -though dependencies may also need recipes. - -If the recipe was created using `conda skeleton pypi`, then the default test is -probably sufficient. The exception is when the package also installs -a command-line tool, in which case that should be tested as well. - -- typical `import` check: [pysam](recipes/pysam) -- import and command-line tests: [chanjo](recipes/chanjo) - - -### R (CRAN) - -Use `conda skeleton cran ` where `packagename` is a package -available on CRAN and is case-sensitive. Either run that command in the -`recipes` dir or move the recipe it creates to `recipes`. The recipe name will -have an `r-` prefix and will be converted to lowercase. Typically can be used -without modification, though dependencies may also need recipes. - -If the recipe was created using `conda skeleton cran` or the -`scripts/bioconductor_skeleton.py` script, the default test is probably -sufficient. - -* typical R recipe from CRAN: [r-locfit](recipes/r-locfit) -* recipe for R package not on CRAN, also with patch: [spp](recipes/r-spp) - - -### R (Bioconductor) - -Use `scripts/bioconductor/bioconductor_skeleton.py ` where -`packagename` is a case-sensitive package available on Bioconductor. The recipe -name will have a `bioconductor-` prefix and will be converted to lowercase. -Typically can be used without modification, though dependencies may also need -recipes. Recipes for dependencies with an `r-` prefix should be created using -`conda skeleton cran`; see above. - -* typical bioconductor recipe: - [bioconductor-limma/meta.yaml](recipes/bioconductor-limma) - - -### Java - -Add a wrapper script if the software is typically called via `java -jar ...`. -For example, [fastqc](recipes/fastqc) already had a wrapper script, but -[peptide-shaker](recipes/peptide-shaker) did not. - -JAR files should go in `$PREFIX/share/$PKG_NAME-$PKG_VERSION-$PKG_BUILDNUM`. -A wrapper script should be placed here as well, and symlinked to `$PREFIX/bin`. - -* Example with added wrapper script: [peptide-shaker](recipes/peptide-shaker) -* Example with patch to fix memory: [fastqc](recipes/fastqc) - -### Perl - -Use `conda skeleton cpan ` to build a recipe for Perl and place -the recipe in the `recipes` dir. The recipe will have the `perl-` prefix. - -The recipe as generated by `conda skeleton cpan` must be changed. **The run and -build requirements must specify `perl-threaded` instead of `perl`**. Since some -bioconda packages depend on a version of Perl compiled with threading support, -a choice was made to have all recipes use `perl-threaded` to avoid maintaining -multiple versions of each Perl module. - -An example of such a package is [perl-module-build](recipes/perl-module-build). - -Alternatively, you can additionally ensure the build requirements for the -recipe include `perl-app-cpanminus`, and then the `build.sh` script can be -simplified. An example of this simplification is -[perl-time-hires](recipes/perl-time-hires). - -If the recipe was created with `conda skeleton cpan`, the tests are likely -sufficient. Otherwise, test the import of modules (see the `imports` section of -the `meta.yaml` files in above examples). - - -### C/C++ - -In general, standard `make` should work. Other build tools (e.g., `autoconf`) -and compilers (e.g., `gcc`) should be specified in the build requirements. - -We have not yet decided whether to have `gcc` as a conda package or to assume -it is in the build environment. Until this decision is made, please add `gcc` -(for Linux packages) and `llvm` (for OSX packages) to the `meta.yaml` as -follows: - -```yaml -requirements: - build: - - gcc # [not osx] - - llvm # [osx] - - run: - - libgcc # [not osx] -``` - -If the package uses `zlib`, then please see the `ZLIB` section below. - -* example requiring `autoconf`: [srprism](recipes/srprism) -* simple example: [samtools](recipes/samtools) - -If your package links dynamically against a particular library, it is often -necessary to pin the version against which it was compiled, in order to avoid -ABI incompatibilities. Instead of hardcoding a particular version in the -recipe, we use jinja templates to achieve this. For example, bioconda provides -an environnmnet variable `CONDA_BOOST` that contains the current major version -of boost. You should pin your boost dependency against that version. An example -is the [salmon recipe](recipes/salmon). You find the libraries you can -currently pin in -[scripts/env_matrix.yml](https://github.com/bioconda/bioconda-recipes/blob/master/scripts/env_matrix.yml). -If you need to pin another library, please notify @bioconda/core, and we will -set up a corresponding environment variable. - -It's not uncommon to have difficulty compiling package into a portable conda -package. Since there is no single solution, here are some examples of how -bioconda contributors have solved compiling issues to give you some ideas on -what to try: - -- [ococo](recipes/ococo) edits the source in `build.sh` to accommodate the C++ compiler on OSX -- [muscle](recipes/muscle) patches the makefile on OSX so it doesn't use static libs -- [metavelvet](recipes/metavelvet), [eautils](recipes/eautils), - [preseq](recipes/preseq) have several patches to their makefiles to fix - `LIBS` and `INCLUDES`, `INCLUDEARGS`, and `CFLAGS` -- [mapsplice](recipes/mapsplice) includes an older version of samtools; the - included samtools' makefile is patched to work in conda envs. -- [mosaik](recipes/mosaik) has platform-specific patches -- one removes - `-static` on linux, and the other sets `BLD_PLATFORM` correctly on OSX -- [mothur](recipes/mothur) and [soapdenovo](recipes/soapdenovo) have many fixes to makefiles - -## General command-line tools -If a command-line tool is installed, it should be tested. If it has a shebang -line, it should be patched to use `/usr/bin/env` for more general use. - -An example of this is [fastq-screen](recipes/fastq-screen). - -For command-line tools, running the program with no arguments, checking the -programs version (e.g. with `-v`) or checking the command-line help is -sufficient if doing so returns an exit code 0. We recommend to not redirect -output to `/dev/null` because it may hide helpful debugging information. - -Examples: - -* exit code 0: [bedtools](recipes/bedtools) - -* exit code 255 in a separate script: [ucsc-bedgraphtobigwig](recipes/ucsc-bedgraphtobigwig) - -* confirm expected text in stderr: [weblogo](recipes/weblogo) - -If a package depends on Python and has a custom build string, then -`py{{CONDA_PY}}` must be contained in that build string. Otherwise Python will -be automatically pinned to one minor version, resulting in dependency conflicts -with other packages. See [mapsplice](recipes/mapsplice) for an example of this. - -### Metapackages - -[Metapackages](http://conda.pydata.org/docs/building/meta-pkg.html) tie -together other packages. All they do is define dependencies. For example, the -[hubward-all](recipes/hubward-all) metapackage specifies the various other -conda packages needed to get full `hubward` installation running just by -installing one package. Other metapackages might tie together conda packages -with a theme. For example, all UCSC utilities related to bigBed files, or a set -of packages useful for variant calling. - -For packages that are not anchored to a particular package (as in the last -example above), we recommended [semantic versioning](http://semver.org/) -starting at 1.0.0 for metapackages. - -### Other examples - -Examples of somewhat non-standard recipes, in no particular order: - -* [blast](recipes/blast) has an OS-specific installation -- OSX copies - binaries while on Linux it is compiled. -* [graphviz](recipes/graphviz) has an OS-specific option to `configure` -* [crossmap](recipes/crossmap) removes libs that are shipped with the source - distribution -* [hisat2](recipes/hisat2) runs `2to3` to make it Python 3 compatible, and - copies over individual scripts to the bin dir -* [krona](recipes/krona) has a `post-link.sh` script that gets called after - installation to alert the user a manual step is required -* [htslib](recipes/htslib) has a small test script that creates example data - and runs multiple programs on it -* [spectacle](recipes/spectacle) runs `2to3` to make the wrapper script Python - 3 compatible, patches the wrapper script to have a shebang line, deletes - example data to avoid taking up space in the bioconda channel, and includes - a script for downloading the example data separately. -* [gatk](recipes/gatk) is a package for licensed software that cannot be - redistributed. The package installs a placeholder script (in this case - doubling as the `jar` - [wrapper](https://github.com/bioconda/bioconda-recipes/blob/master/GUIDELINES.md#java)) - to alert the user if the program is not installed, along with a separate - script (`gatk-register`) to copy in a user-supplied archive/binary to the - conda environment - -### Name collisions -In some cases, there may be a name collision when writing a recipe. For example the -[wget](recipes/wget) recipe is for the standard command-line tool. There is -also a Python package called `wget` [on -PyPI](https://pypi.python.org/pypi/wget). In this case, we prefixed the Python -package with `python-` (see [python-wget](recipes/python-wget)). A similar -collision was resolved with [weblogo](recipes/weblogo) and -[python-weblogo](recipes/python-weblogo). - -If in doubt about how to handle a naming collision, please submit an issue. - -### Fixes for missing URLs -Sometimes, URLs to source code may disappear -- particularly with R and -Bioconductor packages, where only the very latest source code is hosted on the -Bioconductor servers. This means that rebuilding an older package version fails -because the source can't be found. - -The solution to this is the [Cargo -Port](https://depot.galaxyproject.org/software/), developed and maintained by -the [Galaxy](https://galaxyproject.org/) team. The Galaxy Jenkins server -performs daily archives of the source code of packages in `bioconda`, and makes -these tarballs permanently available in Cargo Port. If you try rebuilding -a recipe and the source seems to have disappeared, do the following: - -- search for the package and version at https://depot.galaxyproject.org/software/ -- add the URL listed in the "Package Version" column to your `meta.yaml` file as - another entry in the `source: url` section. -- add the corresponding sha256 checksum displayed upon clicking the Info icon - in the "Help" column to the `source:` section. - -For example, if this stopped working: - -```yaml -source: - fn: argh-0.26.1.tar.gz - url: https://pypi.python.org/packages/source/a/argh/argh-0.26.1.tar.gz - md5: 5a97ce2ae74bbe3b63194906213f1184 -``` - -then change it to this: - -```yaml -source: - fn: argh-0.26.1.tar.gz - url: - - https://pypi.python.org/packages/source/a/argh/argh-0.26.1.tar.gz - - https://depot.galaxyproject.org/software/argh/argh_0.26.1_src_all.tar.gz - md5: 5a97ce2ae74bbe3b63194906213f1184 - sha256: 06a7442cb9130fb8806fe336000fcf20edf1f2f8ad205e7b62cec118505510db -``` - - - -## Tests -An adequate test must be included in the recipe. An "adequate" test depends on -the recipe, but must be able to detect a successful installation. While many -packages may ship their own test suite (unit tests or otherwise), including -these in the recipe is not recommended since it may timeout the build system on -Travis-CI. - -Note that a test must return an exit code of 0. The test can be in the `test` -field of `meta.yaml`, or can be a separate script (see the [relevant conda -docs](http://conda.pydata.org/docs/building/meta-yaml.html#test-section) for -testing). - -It is recommended to pipe unneeded stdout/stderr to /dev/null to avoid -cluttering the output in the Travis-CI build environment. - -## Link and unlink scripts (pre- and post- install hooks) -It is possible to include -[scripts](http://conda.pydata.org/docs/spec.html#link-and-unlink-scripts) that -are executed before or after installing a package, or before uninstalling -a package. These scripts can be helpful for alerting the user that manual -actions are required after adding or removing a package. For example, -a `post-link.sh` script may be used to alert the user that he or she will need -to create a database or modify a settings file. Any package that requires -a manual preparatory step before it can be used should consider alerting the -user via an `echo` statement in a `post-link.sh` script. These scripts may be -added at the same level as `meta.yaml` and `build.sh`: - -* `pre-link.sh` is executed *prior* to linking (installation). An error causes conda to stop. -* `post-link.sh` is executed *after* linking (installation). When the post-link - step fails, no package metadata is written, and the package is not considered - installed. -* `pre-unlink.sh` is executed *prior* to unlinking (uninstallation). Errors are - ignored. Used for cleanup. - -These scripts have access to the following environment variables: - -* `$PREFIX` The install prefix -* `$PKG_NAME` The name of the package -* `$PKG_VERSION` The version of the package -* `$PKG_BUILDNUM` The build number of the package - -## Versions - -If an older version is required, put it in a subdirectory of the recipe. -Examples of this can be found in [bowtie2](recipes/bowtie2), -[bx-python](recipes/bx-python), and others. - -## ZLIB - -If a package depends on zlib, then it will most likely also depend on `gcc` (on -Linux) and `llvm` (on OSX). The `meta.yaml` requirements section should -therefore at least have the following for a recipe that supports both Linux and -OSX: - - -```yaml -requirements: - build: - - gcc # [not osx] - - llvm # [osx] - - zlib - - run: - - libgcc # [not osx] - - zlib -``` - -When building the package, you may get an error saying that zlib.h can't be -found -- despite having zlib listed in the dependencies. The reason is that the -location of `zlib` often has to be specified in the `build.sh` script, which -can be done like this: - -```bash -export CFLAGS="-I$PREFIX/include" -export LDFLAGS="-L$PREFIX/lib" -``` - -Or sometimes: - -``` -export CPATH=${PREFIX}/include -``` - -Sometimes Makefiles may specify these locations, in which case they need to be -edited. See the [samtools](recipes/samtools) recipe for an example of this. It -may take some tinkering to get the recipe to build; if it doesn't seem to work -then please submit an issue. +Please visit https://bioconda.github.io/guidelines.html for the updated guidelines. diff --git a/README.md b/README.md index 4ccffe2b3c..f84937174a 100644 --- a/README.md +++ b/README.md @@ -19,248 +19,4 @@ Please visit https://bioconda.github.io for details. ## Developer guide -**Note: the `bioconda` build system has recently been changed to use CentOS6 -instead of CentOS5. See the conversation in -[#2274](https://github.com/bioconda/bioconda-recipes/pull/2274) for details on -the reasoning behind this. To better take advantage of the efforts of the -[`conda-forge`](https://conda-forge.github.io/) team, we are using the CentOS6 -Docker container ``condaforge/linux-anvil`` for building, and our -[``bioconda-utils``](https://github.com/bioconda/bioconda-utils) package for -managing the infrastructure. See below for updated testing instructions.** - - -If you want to contribute new packages to Bioconda, you are invited to join the -Bioconda team. Please post in the [team thread on -GitHub](https://github.com/bioconda/recipes/issues/1) to ask for permission. - -We build Linux packages inside a CentOS 6 docker container to maintain -compatibility across multiple systems. OSX packages are built using the [OSX -build environment](https://docs.travis-ci.com/user/osx-ci-environment/) on -[Travis CI](https://travis-ci.org/). - -The steps below describe how to contribute a new package. The following -prerequisites are assumed: - -- The [`conda`](http://anaconda.org) command line tool. This comes with the - full [Anaconda scientific Python stack](https://www.continuum.io/downloads) - installation, or the slimmed-down - [Miniconda](http://conda.pydata.org/miniconda.html). The Python 3 version is - recommended. -- [`docker`](https://www.docker.com/) -- [`git`](https://git-scm.com/) - -### Step 1: Create a new recipe - -Fork this repository or create a new branch to work in. Within the new branch, -[create a recipe](http://conda.pydata.org/docs/building/build.html) -(`your_package` in this example) in the `recipes` directory. See our -[guidelines](GUIDELINES.md) for best practices and examples. - -### Step 2: Test the recipe - -When the recipe is ready, first test it with your local conda installation via - - conda build recipes/your_package - -Bioconda uses the `bioconda`, `r`, and `conda-forge` channels. You can either -set them globally with: -``` -conda config --add channels conda-forge -conda config --add channels defaults -conda config --add channels r -conda config --add channels bioconda - -``` - -or by adding the channels to the build command: - - conda build recipes/your_package \ - --channel conda-forge \ - --channel defaults \ - --channel r \ - --channel bioconda - -Then, you can test the build in a docker container. The authoritative source -for how packages are built can be found in the `scripts/travis-run.sh` script, -the `config.yml` config file, and the `.travis.yml` config file. However, the -`simulate-travis.py` script can be used for conveniently testing on a local -machine. Any environmental variables will be passed to `scripts/travis-run.sh` and will -override any defaults detected in .travis.yml. Currently the only variables -useful to modify are TRAVIS_OS_NAME and BIOCONDA_UTILS_TAG and they can be used as - follows: - -``` -# run `mypackagename` -./simulate-travis.py --packages mypackagename --loglevel=debug -# use the linux build system from a non-linux machine -TRAVIS_OS_NAME=linux ./simulate-travis.py -# specify the bioconda-utils commit to use for your builds -BIOCONDA_UTILS_TAG=63543b34 ./simulate-travis.py -``` - -If rebuilding a previously-built package and the version number hasn't changed, -be sure to increment the build number in `meta.yaml` (the default build number -is 0): - - build: - number: 1 - -See below for building on OSX. - -### Step 3: Submit a pull request - -Once these local tests pass, submit a [pull -request](https://help.github.com/articles/using-pull-requests) to this -repository. The [travis-ci](https://travis-ci.org) continuous -integration service will automatically test the pull request. - -When the PR tests pass, the PR can be merged into the master branch. - -Travis-CI will again run tests. However, when testing the master branch, new, -successfully-built packages will be uploaded to the `bioconda` conda channel. -Once these tests pass, your new package can now be installed from anywhere -using: - - conda install -c bioconda your_package - -### Step 4: - -If you want to promote the Bioconda installation of your package, we recommend -to add the following badge to your homepage: - -``` -[![bioconda-badge](https://img.shields.io/badge/install%20with-bioconda-brightgreen.svg?style=flat-square)](http://bioconda.github.io) -``` - -This will display as -[![bioconda-badge](https://img.shields.io/badge/install%20with-bioconda-brightgreen.svg?style=flat-square)](https://bioconda.github.io). -For other styles, replace ``?style=flat-square`` with ``?style=flat`` or -``?style=plastic``. - -### Building packages for Mac OSX -**By default, recipes will be built for both Linux and OSX** -(see "The bioconda build system" section below) upon submitting -a pull request. Many recipes build cleanly on Linux but not on OSX. The easy -fix is to explicitly skip the OSX build using a [platform-specific -selector](http://conda.pydata.org/docs/building/meta-yaml.html#skipping-builds) -on a line in the `meta.yaml` that skips the build, like this: - -``` -build: - skip: True # [osx] -``` - -A better fix is to figure out what needs to be changed for a build to work on -OSX, and include those changes using platform-specific selectors. Such changes -could include tweaks to the build script, applying patches, or using -OS-specific installation methods. See [graphviz](recipes/graphviz), -[blast](recipes/blast), and UCSC utils like -[ucsc-pslcdnafilter](recipes/ucsc-pslcdnafilter) for examples of these methods. - -To set up a local OSX machine for building and testing OSX recipes, run -``scripts/travis-setup.sh``. Several commands in this script will prompt for -sudo privileges, but the script itself should be run as a regular user. This -script will set up a conda environment in ``/anaconda`` and install necessary -prerequisites. - -To test all OSX recipes (skipping those that define `skip: True #[osx]`) use -the following from the top-level dir: - -```bash -scripts/build-packages.py --repository . --env-matrix scripts/env_matrix.yml -``` - -### Managing multiple versions of a package - -If there is interest to keep multiple versions of a package or to explicitly -build an older version of a package, you can store those versions in -subdirectories of the corresponding recipe, e.g.: - -``` -java-jdk/ -├── 7.0.91 -│   ├── build.sh -│   └── meta.yaml -├── build.sh -└── meta.yaml -``` - -There should always be a primary in the root directory of a package that is -updated when new releases are made. - -## The bioconda build system -This repository is set up on [Travis CI](https://travis-ci.org) such that on -every pull request, the following steps are performed once within the Docker -CentOS 5 container and once within the OSX build environment: - -- find all recipes in the `recipes` dir -- filter out recipes that have already been uploaded to the bioconda channel -- parse the remaining recipes to recursively find dependencies -- topologically sort the recipes such that when they are built in order, - dependency packages are built first -- build and test each recipe -- add the recipe to the "local" channel so that subsequent recipes in this - build can use it as a dependency if needed - -If all recipes build and test without error, the pull request can be merged -with the master branch. Upon merging, Travis-CI will detect the merge and the -same steps will be performed again. In addition, at the end of the build, all -built packages will be uploaded to the bioconda channel. This means that as -soon as the Travis-CI tests pass on the master branch, the packages are now -publicly available to all users. - -### Dependencies - -There is currently no mechanism to define, in the `meta.yaml` file, that -a particular dependency should come from a particular channel. This means that -a recipe must have its dependencies in one of the following: - -- as-yet-unbuilt recipes in the repo included in the PR -- `bioconda` channel -- `r` channel -- default Anaconda channel - -Otherwise, you will have to write the recipes for those dependencies and -include them in the PR. `conda skeleton` is very useful for Python (PyPI), Perl -(CPAN), and R (CRAN) packages. Another option is to use `anaconda search -t -conda ` to look for other packages built by others. Inspecting -those recipes can give some clues into building a version of the dependency for -bioconda. - -### Python versions -By default, Python recipes (those that have `python` listed as a dependency) -must be successfully built and tested on Python 2.7, 3.4, and 3.5 in order to -pass. However, many Python packages are not fully compatible across all Python -versions. Use the [preprocessing -selectors](http://conda.pydata.org/docs/building/meta-yaml.html#preprocessing-selectors) -in the meta.yaml file along with the `build/skip` entry to indicate that -a recipe should be skipped. - -For example, a recipe that only runs on Python 2.7 should include the -following: - -```yaml -build: - skip: True # [not py27] -``` - -Or a package that only runs on Python 3.4 and 3.5: - -```yaml -build: - skip: True # [py27] -``` - -Alternatively, for straightforward compatibility fixes you can apply a [patch -in the -meta.yaml](http://conda.pydata.org/docs/building/meta-yaml.html#patches). - -### Creating Bioconductor recipes - -See [`scripts/bioconductor/README.md`](scripts/bioconductor/README.md) for -details on creating and updating Bioconductor recipes. - -### Creating and updating UCSC tools - -See [`scripts/ucsc/README.md`](scripts/ucsc/README.md) for details on creating -and updating recipes for UCSC programs. +Please visit the new docs at https://bioconda.github.io/contributing.html for details.