Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow setting channels #24

Closed
moble opened this issue Feb 23, 2022 · 3 comments
Closed

Allow setting channels #24

moble opened this issue Feb 23, 2022 · 3 comments

Comments

@moble
Copy link

moble commented Feb 23, 2022

As mentioned in JuliaPy/PythonCall.jl#115, it can be necessary to specify the channel from which a particular package should be installed. But CondaPkg went against my expectations in three ways that seems to make that impossible.

  1. If using PythonCall.jl, it adds conda-forge to the channel list, even when it hasn't been added in my own CondaPkg.toml (or ~/.condarc). This seems to be because it picks up PythonCall/CondaPkg.toml, which has no channels listed, and therefore these lines add in conda-forge. It seems like that if statement should come after the for env loop.
  2. It sets --no-channel-priority, which makes it impossible to — for example — ensure that the numba channel takes priority over the conda-forge that was automatically added above. Any prioritization given in CondaPkg.toml should probably just be preserved, with channels added by the dependencies given lower priority.
  3. It doesn't allow setting the channel on a per-package basis. That seems to have been the original intent here, but it doesn't look like that channel part made it into the production code. This doesn't seem like a high priority, of course, since conda itself doesn't support that in environments.yml.

Speaking of environments.yml, is there any reason you chose to use a new spec for CondaPkg? It looks like CondaPkg.toml contains exactly the same information, but in a different format. It would be very convenient to just allow re-use of environments.yml files.

@cjdoris
Copy link
Collaborator

cjdoris commented Feb 24, 2022

All of this comes down to the fact that CondaPkg might find multiple CondaPkg.toml files (e.g. in your project plus some of its dependencies) and it needs to decide how to merge them. That, plus Conda only has two ways to specify channels: a list of channels to look in for all packages (via the --channel) argument, or a specific channel for a particular package (via the channel::package syntax). Currently we don't use the latter method. So to address your points:

  1. This is intentional, and documented, that not specifying a list of channels is equivalent to specifying only conda-forge. Someone who writes a CondaPkg.toml file must know in which channels to look.
  2. This is intentional, and comes from the fact that different CondaPkg.toml files might have a different view on which channels are more important. Since there's no way for them to agree, I just enforce that there is no specified order on channels.
  3. I do intend to allow specifying channels on a per-package basis, but haven't quite figured out how that would work.

Ultimately, I think CondaPkg will change to use the channel::package syntax entirely instead of the global --channel argument, because frankly it's pretty weird to have a list of channels which all might contain different packages of the same name (or the same package but built differently, like in your case, which is just a different package in my book).

The reason I don't use environments.yml is mainly that I don't want to be constrained to its syntax - I need to be able to parse it correctly, and that's easier if I define the format of the file myself (e.g. I don't need to worry about the full package spec format), plus I might need to add more fields not currently in environment.yml. Also TOML is standard in the Julia community. Also I need to parse these quickly at startup time and TOML was the only parser I found which didn't add to the package load time.

@cjdoris
Copy link
Collaborator

cjdoris commented Feb 24, 2022

See #25.

@moble
Copy link
Author

moble commented Feb 24, 2022

Whoa! I thought I knew a bit about conda, what with doing all sorts of conda-build fanciness, maintaining conda-forge packages, etc. But I had never come across the channel::package syntax. Even now, I'm having real trouble finding finding it in any documentation, other than some ambiguous ugliness, a subtle usage in the search help text, and the full spec in a docstring that apparently doesn't make it into the docs.

Thanks for explaining your reasoning. That all makes sense.

@moble moble closed this as completed Feb 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants