mamba tries to install from default even so it's not a known channel #656

michaelaye · 2021-01-11T20:21:21Z

On this machine, I wiped my miniconda and wanted to try miniforge instead, which I understand its only difference is to use the conda-forge channel by default?
I wanted to immediately take advantage of mamba's awesome speed, so added it first before installing anything else.
But I am again met with weird behavior by mamba trying to install from defaults even so that is not an allowed source channel anymore:

$ conda config --show-sources                                                                (py38)
==> /Users/klay6683/miniforge3/.condarc <==
channels:
  - conda-forge

==> /Users/klay6683/.condarc <==
channel_priority: strict

Here's my attempt to install jupyterlab:

$ mamba install jupyterlab                                                                   (py38)

                  __    __    __    __
                 /  \  /  \  /  \  /  \
                /    \/    \/    \/    \
███████████████/  /██/  /██/  /██/  /████████████████████████
              /  / \   / \   / \   / \  \____
             /  /   \_/   \_/   \_/   \    o \__,
            / _/                       \_____/  `
            |/
        ███╗   ███╗ █████╗ ███╗   ███╗██████╗  █████╗
        ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗
        ██╔████╔██║███████║██╔████╔██║██████╔╝███████║
        ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║
        ██║ ╚═╝ ██║██║  ██║██║ ╚═╝ ██║██████╔╝██║  ██║
        ╚═╝     ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝╚═════╝ ╚═╝  ╚═╝

        mamba (0.7.6) supported by @QuantStack

        GitHub:  https://github.com/mamba-org/mamba
        Twitter: https://twitter.com/QuantStack

█████████████████████████████████████████████████████████████


Looking for: ['jupyterlab']

pkgs/main/osx-64         Using cache
pkgs/main/noarch         Using cache
pkgs/r/osx-64            Using cache
pkgs/r/noarch            Using cache
Transaction

  Prefix: /Users/klay6683/miniforge3/envs/py38

  Updating specs:

   - jupyterlab


  Package                Version  Build              Channel                Size
──────────────────────────────────────────────────────────────────────────────────
  Install:
──────────────────────────────────────────────────────────────────────────────────

  appnope                  0.1.2  py38hecd8cb5_1001  pkgs/main/osx-64      10 KB
  argon2-cffi             20.1.0  py38haf1e3a3_1     pkgs/main/osx-64      44 KB
  async_generator           1.10  py_0               pkgs/main/noarch      24 KB
  attrs                   20.3.0  pyhd3eb1b0_0       pkgs/main/noarch      43 KB
  backcall                 0.2.0  py_0               pkgs/main/noarch      15 KB
  bleach                   3.2.1  py_0               pkgs/main/noarch     112 KB
  decorator                4.4.2  py_0               pkgs/main/noarch      14 KB
  defusedxml               0.6.0  py_0               pkgs/main/noarch      23 KB
  entrypoints                0.3  py38_0             pkgs/main/osx-64      10 KB
  importlib-metadata       2.0.0  py_1               pkgs/main/noarch      35 KB
  importlib_metadata       2.0.0  1                  pkgs/main/noarch      11 KB
  ipykernel                5.3.4  py38h5ca1d4c_0     pkgs/main/osx-64     180 KB
  ipython                 7.19.0  py38h01d92e1_0     pkgs/main/osx-64     982 KB
  ipython_genutils         0.2.0  py38_0             pkgs/main/osx-64      39 KB
  jedi                    0.18.0  py38hecd8cb5_0     pkgs/main/osx-64     906 KB
  jinja2                  2.11.2  py_0               pkgs/main/noarch     103 KB
  json5                    0.9.5  py_0               pkgs/main/noarch      22 KB
  jsonschema               3.2.0  py38_1             pkgs/main/osx-64      91 KB
  jupyter_client           6.1.7  py_0               pkgs/main/noarch      77 KB
  jupyter_core             4.7.0  py38hecd8cb5_0     pkgs/main/osx-64      68 KB
  jupyterlab               2.2.6  py_0               pkgs/main/noarch       3 MB
  jupyterlab_pygments      0.1.2  py_0               pkgs/main/noarch       8 KB
  jupyterlab_server        1.2.0  py_0               pkgs/main/noarch      25 KB
  libsodium               1.0.18  h1de35cc_0         pkgs/main/osx-64     236 KB
  markupsafe               1.1.1  py38h1de35cc_1     pkgs/main/osx-64      21 KB
  mistune                  0.8.4  py38h1de35cc_1001  pkgs/main/osx-64      54 KB
  nbclient                 0.5.1  py_0               pkgs/main/noarch      58 KB
  nbconvert                6.0.7  py38_0             pkgs/main/osx-64     483 KB
  nbformat                 5.0.8  py_0               pkgs/main/noarch      88 KB
  nest-asyncio             1.4.3  pyhd3eb1b0_0       pkgs/main/noarch      11 KB
  notebook                 6.1.6  py38hecd8cb5_0     pkgs/main/osx-64       4 MB
  packaging                 20.8  pyhd3eb1b0_0       pkgs/main/noarch      36 KB
  pandoc                    2.11  h0dc7051_0         pkgs/main/osx-64      11 MB
  pandocfilters            1.4.3  py38hecd8cb5_1     pkgs/main/osx-64      14 KB
  parso                    0.7.0  py_0               pkgs/main/noarch      72 KB
  pexpect                  4.8.0  py38_1             pkgs/main/osx-64      77 KB
  pickleshare              0.7.5  py38_1001          pkgs/main/osx-64      13 KB
  prometheus_client        0.9.0  pyhd3eb1b0_0       pkgs/main/noarch      45 KB
  prompt-toolkit           3.0.8  py_0               pkgs/main/noarch     248 KB
  ptyprocess               0.7.0  pyhd3eb1b0_2       pkgs/main/noarch      17 KB
  pygments                 2.7.3  pyhd3eb1b0_0       pkgs/main/noarch     676 KB
  pyparsing                2.4.7  py_0               pkgs/main/noarch      65 KB
  pyrsistent              0.17.3  py38haf1e3a3_0     pkgs/main/osx-64      89 KB
  python-dateutil          2.8.1  py_0               pkgs/main/noarch     215 KB
  pyzmq                   20.0.0  py38h23ab428_1     pkgs/main/osx-64     405 KB
  send2trash               1.5.0  py38_0             pkgs/main/osx-64      17 KB
  terminado                0.9.2  py38hecd8cb5_0     pkgs/main/osx-64      25 KB
  testpath                 0.4.4  py_0               pkgs/main/noarch      82 KB
  tornado                    6.1  py38h9ed2024_0     pkgs/main/osx-64     585 KB
  traitlets                5.0.5  py_0               pkgs/main/noarch      83 KB
  wcwidth                  0.2.5  py_0               pkgs/main/noarch      29 KB
  webencodings             0.5.1  py38_1             pkgs/main/osx-64      20 KB
  zeromq                   4.3.3  hb1e8313_3         pkgs/main/osx-64     414 KB
  zipp                     3.4.0  pyhd3eb1b0_0       pkgs/main/noarch      15 KB

  Change:
──────────────────────────────────────────────────────────────────────────────────

  certifi              2020.12.5  py38h50d1736_1     installed
  certifi              2020.12.5  py38hecd8cb5_0     pkgs/main/osx-64     141 KB
  openssl                 1.1.1i  h35c211d_0         installed
  openssl                 1.1.1i  h9ed2024_0         pkgs/main/osx-64       2 MB

  Upgrade:
──────────────────────────────────────────────────────────────────────────────────

  ca-certificates      2020.12.5  h033912b_0         installed
  ca-certificates      2020.12.8  hecd8cb5_0         pkgs/main/osx-64     121 KB

  Summary:

  Install: 54 packages
  Change: 2 packages
  Upgrade: 1 packages

  Total download: 28 MB

──────────────────────────────────────────────────────────────────────────────────

Metadata

Item	Version
mamba	0.7.6
Python	3.8
OS	macOS 10.15.7 (Catalina)

The text was updated successfully, but these errors were encountered:

michaelaye · 2021-01-11T20:22:53Z

it seems like mamba ignores the .condarc file inside the miniforge folder that was put there by the miniforge installer?

michaelaye · 2021-01-11T20:42:10Z

yep, that seems to be the issue.
Executing the following 2 commands made it work as expected:

conda config --add channels conda-forge
# because previous adds defaults and conda-forge to `~/.condarc`:
conda config --remove channels defaults

wolfv · 2021-01-11T21:15:22Z

We're using the same config system as conda so we'd need to fix how conda overrides the condarc files

michaelaye · 2021-01-11T21:33:29Z

some more interested details in mamba's behavior, depending on content of `~/.condarc':

~/.condarc content:

channel_priority: strict

miniforge/.condarc:

channels: [conda-forge]

-> mamba works, but looks at defaults
-> conda works, but looks at conda-forge

changing only ~/.condarc to:-

channel_priority: strict
channels: []

-> mamba crashes
-> conda works, and looks at conda-forge

michaelaye · 2021-01-11T21:38:41Z

Isn't it a very clear and usual config file override hierarchy in conda:

$CONDA_PREFIX/.condarc (the environment config)
-> (overrules)
$CONDA_PREFIX_1 (the main installation folder, here: ~/miniforge3 )
->
$HOME/.condarc.

A very similar strategy is applied in git and vscode configs.

wolfv · 2021-03-16T11:03:31Z

I've tried to reproduce this following the instructions above, but it worked fine for me (even on OSX).
Again, we're not really touching any of the config file reading (we just use what conda uses).

michaelaye · 2021-03-16T21:08:26Z

I maintain that in the precise above state, conda and mamba perform differently:

$HOME/.condarc has NO entry for channels, (not an EMPTY channels entry, that would throw a conda error), but completely missing:

$ cat ~/.condarc                                                                                               (test_mamba) 
report_errors: true
channel_priority: strict

the system .condarc (which is the .condarc inside the distribution directory) has the channels entry conda-forge only:

$ cat ~/miniconda3/.condarc                                                                                    (test_mamba) 
channels:
  - conda-forge

mamba uses `defaults` channel:

$ mamba install nbformat                                                                                       (test_mamba) 

                  __    __    __    __
                 /  \  /  \  /  \  /  \
                /    \/    \/    \/    \
███████████████/  /██/  /██/  /██/  /████████████████████████
              /  / \   / \   / \   / \  \____
             /  /   \_/   \_/   \_/   \    o \__,
            / _/                       \_____/  `
            |/
        ███╗   ███╗ █████╗ ███╗   ███╗██████╗  █████╗
        ████╗ ████║██╔══██╗████╗ ████║██╔══██╗██╔══██╗
        ██╔████╔██║███████║██╔████╔██║██████╔╝███████║
        ██║╚██╔╝██║██╔══██║██║╚██╔╝██║██╔══██╗██╔══██║
        ██║ ╚═╝ ██║██║  ██║██║ ╚═╝ ██║██████╔╝██║  ██║
        ╚═╝     ╚═╝╚═╝  ╚═╝╚═╝     ╚═╝╚═════╝ ╚═╝  ╚═╝

        mamba (0.8.2) supported by @QuantStack

        GitHub:  https://github.com/mamba-org/mamba
        Twitter: https://twitter.com/QuantStack

█████████████████████████████████████████████████████████████


Looking for: ['nbformat']

pkgs/main/linux-64       [====================] (00m:00s) No change
pkgs/main/noarch         [====================] (00m:00s) No change
pkgs/r/noarch            [====================] (00m:00s) No change
pkgs/r/linux-64          [====================] (00m:00s) No change
Transaction

  Prefix: /home/maye/miniconda3/envs/test_mamba

  Updating specs:

   - nbformat


  Package               Version  Build           Channel                  Size
────────────────────────────────────────────────────────────────────────────────
  Install:
────────────────────────────────────────────────────────────────────────────────

  attrs                  20.3.0  pyhd3eb1b0_0    pkgs/main/noarch        43 KB
  importlib-metadata      3.7.3  py38h06a4308_1  pkgs/main/linux-64      31 KB
  importlib_metadata      3.7.3  hd3eb1b0_1      pkgs/main/noarch        11 KB
  ipython_genutils        0.2.0  py38_0          pkgs/main/linux-64      40 KB
  jsonschema              3.2.0  py38_1          pkgs/main/linux-64      93 KB
  jupyter_core            4.7.1  py38h06a4308_0  pkgs/main/linux-64      68 KB
  nbformat                5.1.2  pyhd3eb1b0_1    pkgs/main/noarch        68 KB
  pyrsistent             0.17.3  py38h7b6447c_0  pkgs/main/linux-64      90 KB
  traitlets               5.0.5  pyhd3eb1b0_0    pkgs/main/noarch        81 KB
  zipp                    3.4.1  pyhd3eb1b0_0    pkgs/main/noarch        15 KB

  Change:
────────────────────────────────────────────────────────────────────────────────

  certifi             2020.12.5  py38h578d9bd_1  installed                    
  certifi             2020.12.5  py38h06a4308_0  pkgs/main/linux-64     141 KB
  openssl                1.1.1j  h7f98852_0      installed                    
  openssl                1.1.1j  h27cfd23_0      pkgs/main/linux-64       3 MB

  Upgrade:
────────────────────────────────────────────────────────────────────────────────

  ca-certificates     2020.12.5  ha878542_0      installed                    
  ca-certificates     2021.1.19  h06a4308_1      pkgs/main/linux-64     118 KB

  Summary:

  Install: 10 packages
  Change: 2 packages
  Upgrade: 1 packages

  Total download: 3 MB

────────────────────────────────────────────────────────────────────────────────

Confirm changes: [Y/n] ^C

conda uses `conda-forge`:

$ conda install nbformat                                                                                       (test_mamba) 
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /home/maye/miniconda3/envs/test_mamba

  added / updated specs:
    - nbformat


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    importlib-metadata-3.7.3   |   py38h578d9bd_0          24 KB  conda-forge
    jsonschema-3.2.0           |     pyhd8ed1ab_3          45 KB  conda-forge
    zipp-3.4.1                 |     pyhd8ed1ab_0          11 KB  conda-forge
    ------------------------------------------------------------
                                           Total:          80 KB

The following NEW packages will be INSTALLED:

  attrs              conda-forge/noarch::attrs-20.3.0-pyhd3deb0d_0
  importlib-metadata conda-forge/linux-64::importlib-metadata-3.7.3-py38h578d9bd_0
  ipython_genutils   conda-forge/noarch::ipython_genutils-0.2.0-py_1
  jsonschema         conda-forge/noarch::jsonschema-3.2.0-pyhd8ed1ab_3
  jupyter_core       conda-forge/linux-64::jupyter_core-4.7.1-py38h578d9bd_0
  nbformat           conda-forge/noarch::nbformat-5.1.2-pyhd8ed1ab_1
  pyrsistent         conda-forge/linux-64::pyrsistent-0.17.3-py38h497a2fe_2
  traitlets          conda-forge/noarch::traitlets-5.0.5-py_0
  zipp               conda-forge/noarch::zipp-3.4.1-pyhd8ed1ab_0


Proceed ([y]/n)? n

Remedy

When adding back the conda-forge channel to $HOME/.condarc all is fine:

$ cat ~/.condarc                                                                                               (test_mamba) 
report_errors: true
channel_priority: strict
channels:
  - conda-forge

System info

conda: 4.9.2
Linux Pop!_OS 20.10 (Ubuntu groovy)

wolfv · 2021-03-16T21:31:25Z

can you give me the output of conda info, and maybe conda config --show-sources?

I would like to see the channels that are listed in conda info and the order of them.

wolfv · 2021-03-16T21:32:21Z

also, please the output of which mamba

did you install mamba in a non-base environment?

michaelaye · 2021-03-16T21:59:50Z

Sure, anything to solve this mistery!

$ conda info                                                                                                   (test_mamba) 

     active environment : test_mamba
    active env location : /home/maye/miniconda3/envs/test_mamba
            shell level : 3
       user config file : /home/maye/.condarc
 populated config files : /home/maye/miniconda3/.condarc
                          /home/maye/.condarc
          conda version : 4.9.2
    conda-build version : not installed
         python version : 3.8.6.final.0
       virtual packages : __cuda=11.2=0
                          __glibc=2.32=0
                          __unix=0=0
                          __archspec=1=x86_64
       base environment : /home/maye/miniconda3  (writable)
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /home/maye/miniconda3/pkgs
                          /home/maye/.conda/pkgs
       envs directories : /home/maye/miniconda3/envs
                          /home/maye/.conda/envs
               platform : linux-64
             user-agent : conda/4.9.2 requests/2.25.1 CPython/3.8.6 Linux/5.8.0-7642-generic pop/20.10 glibc/2.32
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False

The config sources are still the same as the beginning of the post, now in this test_mamba env and for a good old miniconda3 system, as the issue isn't caused by using miniforge apparently:

$ conda config --show-sources                                                                                  (test_mamba) 
==> /home/maye/miniconda3/.condarc <==
channels:
  - conda-forge

==> /home/maye/.condarc <==
channel_priority: strict
report_errors: True

And, yes, I install mamba per env, do you think that can make a difference? I will test that:

$ which mamba                                                                                                  (test_mamba) 
/home/maye/miniconda3/envs/test_mamba/bin/mamba

Using `-vv`

I did some further debugging using install with level 2 debugging and I simply see pretty early on the difference in which URL mamba is trying to access.
For good measure, I cleaned the caches in between using mamba clean --all:

Broken case:

DEBUG conda.gateways.logging:set_verbosity(231): verbosity set to 2
INFO    installed: reading repo file /tmp/tmpwbk8mdj2
INFO    loading from json /tmp/tmpwbk8mdj2
INFO    Checking dir "/home/maye/miniconda3/envs/test_mamba/pkgs"
INFO    "/home/maye/miniconda3/envs/test_mamba/pkgs/urls.txt" exists, checking if writable
INFO    "/home/maye/miniconda3/envs/test_mamba/pkgs/urls.txt" writable
INFO    No cache found https://repo.anaconda.com/pkgs/main/linux-64/repodata.json

Repaired case with the conda-forge entry in $HOME/.condarc:

DEBUG conda.gateways.logging:set_verbosity(231): verbosity set to 2
INFO    installed: reading repo file /tmp/tmp8u40vjr7
INFO    loading from json /tmp/tmp8u40vjr7
INFO    Checking dir "/home/maye/miniconda3/envs/test_mamba/pkgs"
INFO    "/home/maye/miniconda3/envs/test_mamba/pkgs/urls.txt" exists, checking if writable
INFO    "/home/maye/miniconda3/envs/test_mamba/pkgs/urls.txt" writable
INFO    No cache found https://conda.anaconda.org/conda-forge/linux-64/repodata.json

Note the different URLs at the end of both text blocks.

wolfv · 2021-03-16T22:13:06Z

mystery solved!

when you install mamba, it also pulls in conda. And that changes the root prefix of conda to the nested environment, where you have no configuration. However, the conda command itself doesn't change because it's "hardwired" to the root prefix.

If you want to check the config of the conda in the environment you can run $CONDA_PREFIX/bin/conda config ... etc., and that is the config that the mamba in that named env will see.

That's why we strongly recommend to only install mamba in the base environment. (You also don't install conda into each environment, right?)

michaelaye · 2021-03-16T22:34:13Z

Not in each, but all envs that are used to create conda packages need to have conda installed, IIUC?

EDIT: At least everytime I install the conda-build machinery, it pulls in a conda into the env.

I confirm that using mamba from condabin/mamba solves the issue.
But doesn't that asymmetry with conda irk you? ;)
Thanks for keeping listening!

wolfv · 2021-03-16T22:41:54Z

Yes, we're working on fixing that soonish.

wolfv · 2021-03-17T06:54:11Z

btw. there is no asymmetry. the trick of conda is that you do conda shell init and that install the conda command as a shell function (you can confirm that with which conda.

Therefore, if you type conda it will always refer to the conda of the base environment. Even if you install a newer or older conda in an environment, you won't "see" it.

Mamba doesn't do that, yet.

…re defaults channel (#34297) For some reason (potentially related to mamba-org/mamba#656 or maybe just the default conda behavior) it appears that specifying `-c conda-forge` is not sufficient as conda will still check the defaults channel. Lately it seems the defaults channel has introduced some incompatibility that leads to a deadlock or hang during solve. This PR disables defaults completely which seems to avoid the hang. * Closes: #34296 Authored-by: Weston Pace <weston.pace@gmail.com> Signed-off-by: Jacob Wujciak-Jens <jacob@wujciak.de>

…d ignore defaults channel (apache#34297) For some reason (potentially related to mamba-org/mamba#656 or maybe just the default conda behavior) it appears that specifying `-c conda-forge` is not sufficient as conda will still check the defaults channel. Lately it seems the defaults channel has introduced some incompatibility that leads to a deadlock or hang during solve. This PR disables defaults completely which seems to avoid the hang. * Closes: apache#34296 Authored-by: Weston Pace <weston.pace@gmail.com> Signed-off-by: Jacob Wujciak-Jens <jacob@wujciak.de>

wolfv closed this as completed Mar 16, 2021

michaelaye mentioned this issue Sep 29, 2021

Mamba should ONLY be installed into base DOI-USGS/ISIS3#4644

Closed

westonpace mentioned this issue Feb 22, 2023

GH-34296: [C++][CI] Force appveyor builds to use conda-forge and ignore defaults channel apache/arrow#34297

Merged

gapster mentioned this issue Oct 12, 2023

documentation inconsistancies #2907

Open

2 tasks

erik-mansson mentioned this issue Oct 12, 2023

LZO missing or not detected on 64-bit Windows when via conda-forge conda-forge/pytables-feedstock#93

Open

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

mamba tries to install from default even so it's not a known channel #656

mamba tries to install from default even so it's not a known channel #656

michaelaye commented Jan 11, 2021

michaelaye commented Jan 11, 2021

michaelaye commented Jan 11, 2021

wolfv commented Jan 11, 2021

michaelaye commented Jan 11, 2021 •

edited

Loading

michaelaye commented Jan 11, 2021 •

edited

Loading

wolfv commented Mar 16, 2021

michaelaye commented Mar 16, 2021 •

edited

Loading

wolfv commented Mar 16, 2021

wolfv commented Mar 16, 2021

michaelaye commented Mar 16, 2021

wolfv commented Mar 16, 2021

michaelaye commented Mar 16, 2021 •

edited

Loading

wolfv commented Mar 16, 2021

wolfv commented Mar 17, 2021

mamba tries to install from default even so it's not a known channel #656

mamba tries to install from default even so it's not a known channel #656

Comments

michaelaye commented Jan 11, 2021

Metadata

michaelaye commented Jan 11, 2021

michaelaye commented Jan 11, 2021

wolfv commented Jan 11, 2021

michaelaye commented Jan 11, 2021 • edited Loading

michaelaye commented Jan 11, 2021 • edited Loading

wolfv commented Mar 16, 2021

michaelaye commented Mar 16, 2021 • edited Loading

mamba uses defaults channel:

conda uses conda-forge:

Remedy

System info

wolfv commented Mar 16, 2021

wolfv commented Mar 16, 2021

michaelaye commented Mar 16, 2021

Using -vv

Broken case:

Repaired case with the conda-forge entry in $HOME/.condarc:

wolfv commented Mar 16, 2021

michaelaye commented Mar 16, 2021 • edited Loading

wolfv commented Mar 16, 2021

wolfv commented Mar 17, 2021

michaelaye commented Jan 11, 2021 •

edited

Loading

michaelaye commented Jan 11, 2021 •

edited

Loading

michaelaye commented Mar 16, 2021 •

edited

Loading

mamba uses `defaults` channel:

conda uses `conda-forge`:

Using `-vv`

michaelaye commented Mar 16, 2021 •

edited

Loading