New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial pass at full documentation for Conda-based dependency development. #642

Merged
merged 1 commit into from Feb 23, 2017

Conversation

Projects
None yet
3 participants
@jmchilton
Copy link
Member

jmchilton commented Feb 22, 2017

This pull request:

  • Add a new "Advanced" topic for developing tools with Conda dependencies including the following 4 main sections:
    1. Using the planemo conda commands such conda_init, conda_install, conda_env, lint --conda_requirements, and test.
    2. Finding existing Conda packages - using the Anaconda site, conda search, or a new planemo conda_search command for searching best practice channels.
    3. A formal exercise based on the tools-iuc pear.xml tool - verify the tool is broken without a requirement tag, find the correct package name and version, lint the tool to verify the requirement is found, and run a test to verify it is fixed.
    4. A short stub of a section containing resources on building and contributing Conda recipes.
  • Adds a project template for downloading the completed intro seqtk_seq.xml example for testing out planemo conda commands.
  • Adds a project template for a Conda exercise based on the pear tool from tools-iuc.
  • Adds a planemo conda_search command for searching best practice channels from the command line.

I think this can serve as the template and example for the first third of the "Conda and Containers - A Developer Perspective" workshop at the GCC 2017.

@martenson

This comment has been minimized.

Copy link
Member

martenson commented Feb 22, 2017

👏

Exercise - Package a Tool
----------------------------------------------------------------

1. Package a tool for Bioconda.

This comment has been minimized.

@martenson

martenson Feb 22, 2017

Member

Is this something you want to have captured here or just link out to bioconda tutorials?

This comment has been minimized.

@jmchilton

jmchilton Feb 22, 2017

Author Member

I know I would get push back if I put it here - so this is probably not going to end up in the planemo docs. 😢 If it does - it might be a smaller section on galaxy-specific or planemo-specific tricks that already assumes the reader has gone through bioconda docs.

This comment has been minimized.

@martenson

martenson Feb 22, 2017

Member

Do you want it here?

This comment has been minimized.

@jmchilton

jmchilton Feb 22, 2017

Author Member

Perhaps - but that is besides the point. I'm sure the community would be better served by working on the bioconda documentation and tooling.

This comment has been minimized.

@jmchilton

jmchilton Feb 22, 2017

Author Member

Or do you mean this particular line - not the whole idea of documentation for building recipes? Sorry I'm being confusing.

This comment has been minimized.

@martenson

martenson Feb 22, 2017

Member

I am confused indeed.

This comment has been minimized.

@jmchilton

jmchilton Feb 22, 2017

Author Member

I changed the language here - I was using tool incorrectly.

Initial pass at full documentation for Conda-based dependency develop…
…ment.

- Add a new advanced topic for developing tools with Conda dependencies with 4 main sections:
  1. Using the planemo conda commands such conda_init, conda_install, conda_env, lint --conda_requirements, and test.
  2. Finding existing Conda packages - using the Anaconda site, conda search, or a new "planemo conda_search" command for searching best practice channels.
  3. A formal exercise based on the tools-iuc pear.xml tool.
  4. A short stub of a section containing resources on building and contributing Conda recipes.
- Add a project template for downloading the completed intro seqtk_seq.xml example for testing out planemo conda commands.
- Add a project template for a Conda exercise based on the pear tool from tools-iuc.
- Add a ``planemo conda_search`` command for searching best practice channels from the command line.

I think this can serve as the template and example for the first third of the "Conda and Containers - A Developer Perspective" workshop at the GCC 2017.

@jmchilton jmchilton force-pushed the jmchilton:conda_docs_diverge branch from 055202c to cf795a9 Feb 22, 2017

@mvdbeek
Copy link
Member

mvdbeek left a comment

That reads really well. Thanks a lot @jmchilton!

Linting tool /Users/john/workspace/planemo/docs/writing/seqtk_seq_v6.xml
...
Applying linter requirements_in_conda... CHECK
.. INFO: Requirement [seqtk@1.2] matches target in best practice Conda channel [bioconda].

This comment has been minimized.

@mvdbeek

mvdbeek Feb 23, 2017

Member

Could we drop the "best-practice" ? I guess this would be helpful also if you have uploaded things to your own channel.

This comment has been minimized.

@jmchilton

jmchilton Feb 23, 2017

Author Member

lint --conda_requirements checks the "best practice" channels - i.e. will these resolve with the default configuration of Galaxy. It does not check against channels you configure to target with say conda_install or test.

This comment has been minimized.

@jmchilton

jmchilton Feb 23, 2017

Author Member

I guess we should make a separate issue if you want to discuss this more - I feel like a conda_check command or something would be reasonable - but it isn't what I was trying to capture with lint. Does that make sense?

This comment has been minimized.

@mvdbeek

mvdbeek Feb 23, 2017

Member

Yeah, for the lint command it does make sense to limit the channels.
In any case you'd notice something is wrong once you do planemo conda_install, so I would say that a planemo conda_check is not of high priority.


::

$ . <(planemo conda_env seqtk_seq.xml)

This comment has been minimized.

@mvdbeek
(seqtk_seq) $ conda_env_deactivate
$

As shown above the ``conda_env_deactivate`` will be created in this environment and can

This comment has been minimized.

@mvdbeek

mvdbeek Feb 23, 2017

Member

Why not use deactivate that should be present in that conda environment? Less of a change for people that know conda outside of the galaxy/planemo universe.

This comment has been minimized.

@jmchilton

jmchilton Feb 23, 2017

Author Member
  • Conda's requires sourcing - unlike virtualenv's right? (source deactivate vs deactivate). As a personal taste I don't love that extra step.
  • Also I usually use planemo from within a virtualenv - what would happen if I replaced deactivate - I guess I could create an old_deactivate alias and restore it? I don't know how to do that in a cross shell compatible way. If someone wanted to work on that though - I'd be happy to merge the PR that did that.
``conda search`` command but configured to use the same channels and other options as
Planemo and Galaxy. The following Conda command would also work to search::

$ $HOME/miniconda3/bin/conda -c bioconda -c conda-forge -c iuc seqt

This comment has been minimized.

@mvdbeek

mvdbeek Feb 23, 2017

Member

Highest priority is iuc, then bioconda, then conda-forge. See https://github.com/galaxyproject/galaxy/blob/dev/config/galaxy.ini.sample#L212.
So that would be -c iuc -c bioconda -c conda-forge (I'm excluding defaults here, assuming this is always active). Can you confirm this @bgruening ?

When using the website to search though, you need to aware of what channel you are using. By
default, Planemo and Galaxy will search a few different Conda channels. While it is possible
to configure a local Planemo or Galaxy to target different channels - the current best practice
it to add tools to the existing channels.

This comment has been minimized.

@mvdbeek

mvdbeek Feb 23, 2017

Member

s/it/is/


This will download a tool for `PEAR - Paired-End reAd mergeR
<http://sco.h-its.org/exelixis/web/software/pear/>`__. This tool however has
``requirement`` tags and so will not work properly.

This comment has been minimized.

@mvdbeek

mvdbeek Feb 23, 2017

Member

This tool has no requirements tag ... right?

This comment has been minimized.

@jmchilton

jmchilton Feb 23, 2017

Author Member

Yeah - that is what I meant.

<tool id="iuc_pear" name="Pear" version="0.9.6.1">
<description>Paired-End read merger</description>
<stdio>
<exit_code range="1:" />

This comment has been minimized.

@mvdbeek

mvdbeek Feb 23, 2017

Member

I think that in the IUC we would probably advocate the use of <command detect_errors="exit_code"> or "aggressive"

This comment has been minimized.

@jmchilton

jmchilton Feb 23, 2017

Author Member

It is an IUC tool 😜 - I'll collect your advice and open a PR against tools-iuc.

<![CDATA[
pear
#if str( $library.type ) == "paired":
-f "$library.forward"

This comment has been minimized.

@mvdbeek

mvdbeek Feb 23, 2017

Member

And we're advocating single quotes for paths. But I guess it's not so straightforward to implement this in planemo, right?

This comment has been minimized.

@mvdbeek

mvdbeek Feb 23, 2017

Member

Also that has nothing to do with this PR, just mentioning it.

@jmchilton jmchilton merged commit cf795a9 into galaxyproject:master Feb 23, 2017

1 check failed

continuous-integration/travis-ci/pr The Travis CI build failed
Details

@jmchilton jmchilton added this to the GCC 2017 milestone Feb 24, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment