Point to other conda environment

[kescobo]

Might be related / duplicate of #2, but not sure.

Here's my situation: I have a package that wraps a bunch of command line utilities that are managed by conda. I was previously using Conda.jl to handle this from within julia, and I was able to shift to CondaPkg with very little effort. Nice!

But I expect that many of my users will already have conda environments set up (a number of the packages require databases and a bunch of other set-up to work) or will need to specify the path of the conda env externally (eg, those databases are large, they may not want them in their home folder, especially in a cluster environment). The way I do this at present is to just expect that the command line programs are in ENV["PATH"], and provide instructions for people to install things and set the $PATH if they need to. I also have a function install_deps() function that will do this for them.

I'd much rather not have to bother with this, and just have some CONDA_ENV variable or something that users can set, and if it's not there, use CondaPkg to set it up. Is that possible, at present?

[kescobo]

Is it just these lines: https://github.com/cjdoris/CondaPkg.jl/blob/main/src/env.jl#L24-L27 ?

It seems that if I've already got a conda environment activated from my shell, CondaPkg is smart enough to use that:

julia> CondaPkg.withenv() do
       run(`which python`)
end

# removed output....

/home/kevin/Software/miniconda3/bin/python
Process(`which python`, ProcessExited(0))

So that's cool! Maybe just add some guidance in the README to this effect would be useful?

[cjdoris]

No, CondaPkg ignores any other environment, but withenv uses your current PATH plus whatever the environment adds (just as if you had activated the environment from the command line).

If I understand correctly, you'd like the user to be able to choose between installing your dependencies themselves or letting CondaPkg do it for them?

Right now, CondaPkg dependencies are purely declarative, and not optional. This rules out shipping a CondaPkg.toml file with your package because they'd be installed for everyone. So I think the best you can do at the minute is give the user a function which adds the right CondaPkg dependencies to their current project.

Optional dependencies is at the back of my mind but I don't have any solid idea how it would work yet. Happy to discuss.

[kescobo]

Ah, ok, in that case perhaps I'll re-open this for now. I guess what I was seeing is the fact that activating a conda environment adds stuff to your PATH.

If I understand correctly, you'd like the user to be able to choose between installing your dependencies themselves or letting CondaPkg do it for them?

Yes, essentially. Eg, perhaps the simplest thing would be if ENV["CONDA_DEFAULT_ENV"] is set already, then just use that? Alternatively, maybe one could check for some new flag ENV["CONDAPKG_USE_DEFAULT_ENV"] = true or something?

I don't think it makes sense for you to support being able to stack environments or to manage some existing environment, but giving users the ability to use their own environment (and if it breaks, it's on them) might be nice.

[cjdoris]

Ok sure, there could certainly be an opt out mechanism. There is already a backends machinery to let you use a preinstalled Conda. I could add a Null backend which doesn't actually do anything, assuming the user already has a sufficient Conda environment already active.

[kescobo]

Oh, yeah that seems like a fine solution to me 👍

[cjdoris]

Ok I've made an issue specifically for the new backend.

Closing this now but feel free to reopen if you want to keep discussing.

[hhaensel]

This issue was very helpful mfor me! Thanks for adding the Null backend 🚀

I tend to use my Conda.jl installation for all my python work and for me it is very helpful to have the same environment in Python and Julia, particularly when I'm porting code. It took me a while to find the docs for using the same environment as PyCall.
Maybe the following two snippets could find their place in the installation instruction?

ENV["JULIA_PYTHONCALL_EXE"] = "@PyCall"
using PythonCall
@py import pandas as pd

ENV["JULIA_CONDAPKG_BACKEND"] = "Null"
ENV["JULIA_PYTHONCALL_EXE"] = Sys.which("python")
using PythonCall
@py import pandas as pd

[cjdoris]

Just added this: https://cjdoris.github.io/PythonCall.jl/dev/pythoncall/#pythoncall-config

[hhaensel]

Very clear now 😃. It will certainly help adopters coming from PyCall like me ...