Extend conda interface: store conda environments in project

[jan-janssen]

This pull request changes the location of the conda environment and stores it inside the project in a conda folder. In addition, it checks if an environment already exists and in that case just raises a warning rather than recreating the environment.

[srmnitc]

How would this play out when there is limited number of files possible in the file system? For example, on a HPC cluster. One might want to keep the project on a backed up space, but the conda environment there would be an overkill.

[jan-janssen]

How would this play out when there is limited number of files possible in the file system? For example, on a HPC cluster. One might want to keep the project on a backed up space, but the conda environment there would be an overkill.

I would exclude all directories which have conda in the name from the backup. It is similar to the way snakemake does it. The advantage is that it is more clear for the user to see which environment belongs to which project.

[srmnitc]

How would this play out when there is limited number of files possible in the file system? For example, on a HPC cluster. One might want to keep the project on a backed up space, but the conda environment there would be an overkill.

I would exclude all directories which have conda in the name from the backup. It is similar to the way snakemake does it. The advantage is that it is more clear for the user to see which environment belongs to which project.

Is this possible, for example on cmti. I feel a lot of users might simply not know it. I do see the advantages. I suggest we discuss this in one of the pyiron meetings.

[jan-janssen]

I would like to include this functionality in the 0.8 release of pyiron_base and as we have no pyiron meeting next week - given the easter holidays - it would be great to discuss this asynchronously.

[srmnitc]

I still have my reservations about storing the whole environment in the project. Mainly about the number of files created and backup as I mentioned. Additionally for example wont the pr.pack function now include the environment files itself? But maybe @pmrv or others can also take a look.

[jan-janssen]

Mainly about the number of files created and backup as I mentioned.

You can still create a central project and store all your environments there, it is just the default that is changing.

Additionally for example wont the pr.pack function now include the environment files itself?

This is correct. Still I guess in most cases people are anyway copying their jobs to an archive project as discussed in pyiron/FAQs#46

[pmrv]

Would it not be possible to make the conda path configurable here, that would side step @srmnitc concerns, which I think are valid.

[jan-janssen]

@pmrv and @srmnitc based on your suggestion, I kept the default option to create a new central conda environment. Still now the users can disable the global installation and create a conda environment only in a given project using the global_installation=False option:

pr.conda_environment.create(env_name="test", env_file="environment.yml", global_installation=False)

[srmnitc]

lgtm