Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change Conda channel order #3237

Merged
merged 6 commits into from Dec 12, 2016

Conversation

@bgruening
Copy link
Member

commented Dec 1, 2016

After some discussions with conda-forge we agreed on this channel order for now. The r channel will go away at some point as we will try to get all packages from r into conda-forge.

I think this PR needs to be backported.

bgruening added some commits Dec 1, 2016

@lecorguille

This comment has been minimized.

Copy link
Member

commented Dec 1, 2016

👍

@galaxybot galaxybot added the triage label Dec 1, 2016

@galaxybot galaxybot added this to the 17.01 milestone Dec 1, 2016

@@ -203,7 +203,8 @@ paste.app_factory = galaxy.web.buildapp:app_factory
# Pass debug flag to conda commands.
#conda_debug = False
# conda channels to enable by default (http://conda.pydata.org/docs/custom-channels.html)
#conda_ensure_channels = conda-forge,r,bioconda,iuc
# the recommended channel order is the one from BioConda (https://github.com/bioconda/bioconda-recipes/blob/master/config.yml#L8)
#conda_ensure_channels = bioconda,r,defaults,conda-forge,iuc

This comment has been minimized.

Copy link
@nsoranzo

nsoranzo Dec 1, 2016

Member

Isn't iuc channel supposed to be used (also) to fix problems with packages in other channels? If we put it last, it's not very useful. It should probably be first, no?

This comment has been minimized.

Copy link
@bgruening

bgruening Dec 3, 2016

Author Member

This really depends on what we want to use this channel for. My Intension was to put in old recipes, that are outdated and we don't want to have in bioconda, or some ugly hacks (but for new recipes). Not replacing/overwriting recipes from other channels.

Up for discussion I suppose.

This comment has been minimized.

Copy link
@nsoranzo

nsoranzo Dec 6, 2016

Member

If iuc is only for old recipes, its position shouldn't matter, but if we are ever going to need to use it for some "ugly hack" it needs to be first.

Has any one else opinions here?

This comment has been minimized.

Copy link
@martenson

martenson Dec 6, 2016

Member

I am +1 on having a channel we directly control as the first entry here. Can we create a new one for this purposes?

This comment has been minimized.

Copy link
@jmchilton

jmchilton Dec 6, 2016

Member

I understand the desire to have like an IUC override channel for hacks. I don't think we need it yet though and it probably shouldn't hold up this PR. Can we create a low priority issue for that and continue with this correction?

This comment has been minimized.

Copy link
@nsoranzo

nsoranzo Dec 6, 2016

Member

@martenson Do you mean an additional one? I think one "Galaxy" channel should be enough and iuc is mostly unused.

This comment has been minimized.

Copy link
@nsoranzo

nsoranzo Dec 6, 2016

Member

@jmchilton @bgruening I don't think there is a rush to merge this or the need to backport it since channel order in Conda 3.19.3 matters only if 2 packages have the same version and build numbers. This changed in Conda 4.1.0, see http://conda.pydata.org/docs/channels.html

This comment has been minimized.

Copy link
@martenson

martenson Dec 6, 2016

Member

@nsoranzo Does putting iuc first have any unwanted effects?
@jmchilton this targets 17.01 - no rush, right?

This comment has been minimized.

Copy link
@nsoranzo

nsoranzo Dec 6, 2016

Member

@martenson iuc channel seems to have 3 packages, all in a version different from what's available in bioconda.

This comment has been minimized.

Copy link
@martenson

martenson Dec 8, 2016

Member

I honestly don't know. You guys decide.

@martenson

This comment has been minimized.

Copy link
Member

commented Dec 8, 2016

👍

bgruening added some commits Dec 12, 2016

@nsoranzo nsoranzo merged commit ad8d48c into dev Dec 12, 2016

5 checks passed

api test Build finished. 243 tests run, 0 skipped, 0 failed.
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details
framework test Build finished. 131 tests run, 0 skipped, 0 failed.
Details
toolshed test Build finished. 580 tests run, 0 skipped, 0 failed.
Details
@@ -31,7 +31,7 @@

DEFAULT_BASE_PATH_DIRECTORY = "_conda"
DEFAULT_CONDARC_OVERRIDE = "_condarc"
DEFAULT_ENSURE_CHANNELS = "conda-forge,r,bioconda,iuc"
DEFAULT_ENSURE_CHANNELS = "iuc,bioconda,r,defaults,conda-forge"

This comment has been minimized.

Copy link
@mvdbeek

mvdbeek Jan 3, 2017

Member

Sorry for being late to the game, but is it possible that this list should be reversed?
It appears that currently conda-forge takes the lowest priority.
Admittedly this is a bit confusing, but it appears that galaxyproject/tools-devteam#375 fails because bioconda takes the highest priority, while conda-forge should be highest. This also explains passing tests in bioconda but failing when we try to use them in galaxy.

This comment has been minimized.

Copy link
@bgruening

bgruening Jan 3, 2017

Author Member

We need to stick to the bioconda channel order and this was changed to give bioconda the opportunity to overwrite conda-forge. I don't agree with this move but it was decided this way and I think Galaxy should follow it. https://github.com/bioconda/bioconda-recipes/blob/master/config.yml#L11

This comment has been minimized.

Copy link
@mvdbeek

mvdbeek Jan 3, 2017

Member

We need to stick to the bioconda channel order

absolutely. but I don't think we do, given that for example for perl-gd the import tests pass (because the dependencies are pulled in from conda-forge, I presume). I don't completely understand the details yet, but to me it looks like galaxy is not respecting the channel order. Changing the order in planemo with --conda_channels has no effect whatsoever.

This comment has been minimized.

Copy link
@mvdbeek

mvdbeek Jan 3, 2017

Member

It's possible that there are two issues:

  • For whatever reason the bioconda build system doesn't do what it's supposed to (either false positive test on the import test, or it does actually have conda-forge at a higher priority ... difficult to tell with the limited test output)
  • We're unable to change channel priority with planemo conda_install --conda_channels ...
    So I guess @nsoranzo is right to pin perl-gd to a higher libgd build number

This comment has been minimized.

Copy link
@gregvonkuster

gregvonkuster Jan 3, 2017

Contributor

I've noticed similar issues in my development environment when changing the channel order to pull in recipes I'm developing and uploading here https://anaconda.org/gregvonkuster/, but not related to planemo or galaxy.

The issue I've encountered is that if a package/version is is available in one of "iuc,bioconda,r,defaults,condo-forge", and I add an altered recipe for the package using the same version to my anaconda channel and update my channels to be "gregvonkuster,iuc,bioconda,r,defaults,condo-forge", the altered recipe is not pulled from my channel. Instead, the package is pulled from its original channel using the unchanged recipe.

This is all done from the command line without using planemo or galaxy. It seems that conda somehow is not accounting for my channel order.

This comment has been minimized.

Copy link
@bgruening

bgruening Jan 3, 2017

Author Member

When adding channels using conda config --add they should be added in reverse (from low to high priority). How are you adding these channels @gregvonkuster?
@mvdbeek do you have updated your bioconda-utils installation, is everything up to date?

This comment has been minimized.

Copy link
@mvdbeek

mvdbeek Jan 3, 2017

Member

I haven't used bioconda utils for this ... everything is done by travis, so that's whatever travis is using.

This comment has been minimized.

Copy link
@lecorguille

lecorguille Jan 3, 2017

Member

Note that the condarc is hard code in Planemo and even if you specify all the channel to change the order, Planemo just add those that are missing

This comment has been minimized.

Copy link
@gregvonkuster

gregvonkuster Jan 3, 2017

Contributor

I'm running on Linux and using .condarc. I'm still running miniconda2, version 4.2.13. I have not tried the conda config --add channels approach, so I'll try it and see if it makes a difference. Thanks!

This comment has been minimized.

Copy link
@lecorguille

lecorguille Jan 3, 2017

Member

Edit: it's within Galaxy

@bgruening bgruening deleted the conda_channel branch Jan 3, 2017

@nsoranzo nsoranzo changed the title change conda channel order Change Conda channel order Jan 27, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.