Collapse conda_env into conda

[kenodegard]

Description

conda_env was previously a standalone project and was integrated into the main conda codebase ~6 years ago (#2950). No plans were made at the time for the overall code structure or what to do with the subcommand. It's high time to clean this up and finish collapsing conda_env into the conda codebase.

Goals of collapsing conda_env into conda:

1. simplify the code base
2. remove "duplicate" subcommands, many of these commands are similar but offer different features (this is confusing for users and developers), all unique features from conda env should be moved over into conda

   "Old" Command	"New" Command
   conda env config	conda config
   conda env create	conda create
   conda env export	🤔 introduce as conda export?
   conda env list	conda info --envs, conda envs
   conda env remove	conda remove already supports this
   conda env update	conda update
   conda env vars	🤔 is this still used/needed?

3. continue to make envs a first-class citizen of conda, as we've learned again and again, a large bulk of install/dependency issues arise from users installing everything into their base env, collapsing conda env commands into conda will help simplify the CLI and should help with educating users

To not break everything for everyone, the conda env subcommands should become aliases to the conda subcommands, and we can decide later whether there's any value in fully deprecating the env subcommand (I'm assuming probably not, conda env will just remain in a frozen/alias state).

Tasks

• POC
  • Investigating collapsing conda_env into conda #11873
  • Move conda_env cli into conda #12514
• Write CEP outlining CLI changes (if we aren't fully deprecating/deleting conda env subcommands, this isn't necessary)
• Decentralize conda.cli #13172
• Combine conda env create with conda create and make the former an alias #13015
  • Error from conda env create --name test #3859
  • support for command for creating env with conda env create --name <env_name> <python_version> without requirement of environment.yml file  #12479
  • Override Channels with conda env create #10498
  • Support creating environment from pyproject.toml #10633
  • conda create from yaml will not read env name from file #13676
  • Env Create Warning Message #13600
• Add conda export and alias conda env export Move conda env export to conda export #13577
• Make conda env list an alias of conda info --envs
• Combine conda env remove with conda remove and make the former an alias
  • Conda env remove removes environment directory and empty parent directories  #9688
  • conda env remove tries to remove parent path /tmp/, fails & crashes #10588
  • conda env remove does not run unlink transactions (pre_unlink, remove_menus, etc) #11092
  • cannot delete environment via conda #11651
  • ENH(remove): Add option to keep the (empty) environment despite removal of all packages #12485
  • "conda env remove" should not remove empty parent directories outside of a dedicated environment repository #13113
• Combine conda env update with conda update and make the former an alias
• Determine whether conda env vars is still in use and whether to deprecate or update
• Performance improvments: SPIKE 🔍 A faster CLI command to list all envs #11846

[jaimergp]

conda env remove | 🤔 combine with conda remove?

For completeness, this issue seems relevant: #11092

As in, yes please, combine it!

[dharhas]

continue to make envs a first-class citizen of conda, as we've learned again and again, a large bulk of install/dependency issues arise from users installing everything into their base env, collapsing conda env commands into conda will help simplify the CLI and should help with educating users

So a bit confused about this logic. Not sure how removing the env namespace actually helps improve best practices.

To me the issue was different behavior between similar commands. i.e. conda create and conda env create

One advantage about the env namespace is that it makes it clear whether you are running related in a env or not. I always found the -n flag problematic and opaque to new users.

To me what conda env list or conda env remove is a lot clearer than conda info --envs or conda remove --name

To be really heretical (or logical) an alternate proposal could be that conda create gets replaces by conda env create because then you have a nice dichotomy where plain conda commands apply to the current environment and ones in the env name space apply to a particular environment. The other option is to use --env instead of --name to refer to an environment.

Disclaimer, to some extent I'm bike shedding. I care more about having a single consistent way to do things that these particulars.

[jezdez]

continue to make envs a first-class citizen of conda, as we've learned again and again, a large bulk of install/dependency issues arise from users installing everything into their base env, collapsing conda env commands into conda will help simplify the CLI and should help with educating users

So a bit confused about this logic. Not sure how removing the env namespace actually helps improve best practices.

To me the issue was different behavior between similar commands. i.e. conda create and conda env create

This is also what we've identified, the difference in behavior stems from actual different, not-well aligned code bases and different opinions of what an environment is and how the user experience should be. It's related to the CLI being tightly coupled with a lot of backend UX focused features. I think the integration between the two were basically not fully done.

One advantage about the env namespace is that it makes it clear whether you are running related in a env or not. I always found the -n flag problematic and opaque to new users.

To me what conda env list or conda env remove is a lot clearer than conda info --envs or conda remove --name

env certainly is a CLI namespace for sure, but that is really just a facade for a pretty disorganized code to handle environments behind the scenes. conda and conda-env even have different code paths to do the same thing, but with subtle differences in behavior, something users have raised repeatedly as confusing.

I intend to focus conda's user experience on using environments explicitly as much as possible, so the core commands need to have a good understanding of how environments work. Something that to me is best served by rolling the conda-env functionality into the core commands. This is even more important given that the plugins API, currently in development, will allow additional higher-level functionality to be added as subcommands.

To be really heretical (or logical) an alternate proposal could be that conda create gets replaces by conda env create because then you have a nice dichotomy where plain conda commands apply to the current environment and ones in the env name space apply to a particular environment. The other option is to use --env instead of --name to refer to an environment.

Yeah, we considered that as well, but that would not match with the long-term goal to reduce and possibly even phasing out the base environments outright and having a strong core functionality that understands environments. That users currently need to understand the difference between an activated and explicitly provided environment and how they relate to the core subcommands and the conda-env subcommands seems more of a historical artifact than a good user experience.

Disclaimer, to some extent I'm bike shedding. I care more about having a single consistent way to do things that these particulars.

I fully agree that there should one way to refer to environments. And that because of UX issues with other core features of conda (e.g., env activation), it's required to having a strong understanding of environments in the core subcommands instead of separating it into an own subcommand namespace.

Generally speaking, it's in the interest of users to improve the affordances of the CLI, or in cases like this, remove the negative affordances that create confusion and mistrust with the tool – especially for a fundamental feature like environments.

[dharhas]

possibly even phasing out the base environments outright and having a strong core functionality that understands environments

I would strongly support ☝🏽

[kenodegard]

Another example of conda vs conda env confusion: #11651

[itcarroll]

A feature of conda env create that I would like to see brought into conda create during this "collapse": including the environment name in a file that is read by default. With conda env create you do not have to specify --name on the command line as you do with conda create. That's really useful for CI workflows that clone a repo with an environment.yml.

If it worked out that "collapsing" the way conda env create handles the default YAML file affected other commands, I would not mind 😃 . Just conda create && conda activate, with no --name after cloning would be grand.