conda: command not found for windows-job

[JohannesWiesner]

I am using a matrix to create a windows and linux job that solve an input .yml file and should output a solved .yml file / spec-file.txt using:

conda env export -p "${MAMBA_ROOT_PREFIX}/envs/${{ matrix.os }}" | grep -v "^prefix: " > ./environments/csp_${{ matrix.os }}_solved.yml
conda list --explicit -p "${MAMBA_ROOT_PREFIX}/envs/${{ matrix.os }}" > ./environments/csp_${{ matrix.os }}_solved.txt

The linux job works fine but for the windows job I get:

conda: command not found

It seems that the shell has not been initialized correctly here? Here's the link to the full code. And here is the output when setting up setup-micromamba@v1 for the Windows-job:

Run mamba-org/setup-micromamba@v1
Install micromamba
Adding conda-forge to condarc channels ...
Initialize micromamba for bash.
Initialize micromamba for powershell.
Init powershell profile at 'C:\Users\runneradmin\Documents\WindowsPowerShell\profile.ps1'
Adding (or replacing) the following in your "C:\\Users\\runneradmin\\Documents\\WindowsPowerShell\\profile.ps1" file

#region mamba initialize
# !! Contents within this block are managed by 'mamba shell init' !!
$Env:MAMBA_ROOT_PREFIX = "C:\Users\runneradmin\micromamba"
$Env:MAMBA_EXE = "C:\Users\runneradmin\micromamba-bin\micromamba.exe"
(& $Env:MAMBA_EXE 'shell' 'hook' -s 'powershell' -p $Env:MAMBA_ROOT_PREFIX) | Out-String | Invoke-Expression
#endregion

Init pwsh profile at 'C:\Users\runneradmin\Documents\PowerShell\profile.ps1'
Adding (or replacing) the following in your "C:\\Users\\runneradmin\\Documents\\PowerShell\\profile.ps1" file

#region mamba initialize
# !! Contents within this block are managed by 'mamba shell init' !!
$Env:MAMBA_ROOT_PREFIX = "C:\Users\runneradmin\micromamba"
$Env:MAMBA_EXE = "C:\Users\runneradmin\micromamba-bin\micromamba.exe"
(& $Env:MAMBA_EXE 'shell' 'hook' -s 'powershell' -p $Env:MAMBA_ROOT_PREFIX) | Out-String | Invoke-Expression
#endregion

Windows long-path support already enabled.

Install environment `windows-latest`
Adding environment windows-latest to auto-activate bash ...
Adding environment windows-latest to auto-activate powershell ...
Skipping micromamba run shell on Windows.
Set environment variables.
micromamba info
  C:\Users\runneradmin\micromamba-bin\micromamba.exe info -r C:\Users\runneradmin\micromamba -n windows-latest
  
                                             __
            __  ______ ___  ____ _____ ___  / /_  ____ _
           / / / / __ `__ \/ __ `/ __ `__ \/ __ \/ __ `/
          / /_/ / / / / / / /_/ / / / / / / /_/ / /_/ /
         / .___/_/ /_/ /_/\__,_/_/ /_/ /_/_.___/\__,_/
        /_/
  
  
              environment : windows-latest
             env location : C:\Users\runneradmin\micromamba\envs\windows-latest
        user config files : C:\Users\runneradmin\.mambarc
   populated config files : C:\Users\runneradmin\micromamba-bin\.condarc
         libmamba version : 1.4.4
       micromamba version : 1.4.4
             curl version : libcurl/8.1.2-DEV Schannel zlib/1.2.13
       libarchive version : libarchive 3.6.2 zlib/1.2.13 liblzma/5.4.3 bz2lib/1.0.8 liblz4/1.9.4 libzstd/1.5.5
         virtual packages : __win=0=0
                            __archspec=1=x86_64
                 channels : https://conda.anaconda.org/conda-forge/win-64
                            https://conda.anaconda.org/conda-forge/noarch
         base environment : C:\Users\runneradmin\micromamba
                 platform : win-64
micromamba list

[pavelzw]

If you compare the runner-images of ubuntu-latest and windows-latest, you can see the following:

ubuntu-latest

Miniconda 23.3.1

windows-latest

Miniconda 23.3.1 (pre-installed on the image but not added to PATH)

So we can see that conda is on path on linux but not on windows.
Since setup-micromamba doesn't do anything with conda, conda is still not in PATH in the windows runner.

You could fix this like that:

- run: echo $CONDA >> $GITHUB_PATH
  shell: bash

Or use the absolute path for the conda executable

# probably C:\Miniconda\conda.exe (?)
- run: C:\Miniconda\conda.exe env export -p [...]

[pavelzw]

Alright I just checked what the conda installation on windows-latest looks like

# ls /c/Miniconda/
_conda.exe                                     api-ms-win-core-string-l1-1-0.dll      api-ms-win-crt-utility-l1-1-0.dll  python.exe
api-ms-win-core-console-l1-1-0.dll             api-ms-win-core-synch-l1-1-0.dll       concrt140.dll                      python.pdb
api-ms-win-core-datetime-l1-1-0.dll            api-ms-win-core-synch-l1-2-0.dll       condabin                           python3.dll
api-ms-win-core-debug-l1-1-0.dll               api-ms-win-core-sysinfo-l1-1-0.dll     conda-meta                         python310.dll
api-ms-win-core-errorhandling-l1-1-0.dll       api-ms-win-core-timezone-l1-1-0.dll    cwp.py                             python310.pdb
api-ms-win-core-file-l1-1-0.dll                api-ms-win-core-util-l1-1-0.dll        DLLs                               pythonw.exe
api-ms-win-core-file-l1-2-0.dll                api-ms-win-crt-conio-l1-1-0.dll        envs                               pythonw.pdb
api-ms-win-core-file-l2-1-0.dll                api-ms-win-crt-convert-l1-1-0.dll      etc                                Scripts
api-ms-win-core-handle-l1-1-0.dll              api-ms-win-crt-environment-l1-1-0.dll  include                            share
api-ms-win-core-heap-l1-1-0.dll                api-ms-win-crt-filesystem-l1-1-0.dll   Lib                                shell
api-ms-win-core-interlocked-l1-1-0.dll         api-ms-win-crt-heap-l1-1-0.dll         Library                            Tools
api-ms-win-core-libraryloader-l1-1-0.dll       api-ms-win-crt-locale-l1-1-0.dll       libs                               ucrtbase.dll
api-ms-win-core-localization-l1-2-0.dll        api-ms-win-crt-math-l1-1-0.dll         LICENSE_PYTHON.txt                 Uninstall-Miniconda3.exe
api-ms-win-core-memory-l1-1-0.dll              api-ms-win-crt-multibyte-l1-1-0.dll    Menu                               vccorlib140.dll
api-ms-win-core-namedpipe-l1-1-0.dll           api-ms-win-crt-private-l1-1-0.dll      msvcp140.dll                       vcomp140.dll
api-ms-win-core-processenvironment-l1-1-0.dll  api-ms-win-crt-process-l1-1-0.dll      msvcp140_1.dll                     vcruntime140.dll
api-ms-win-core-processthreads-l1-1-0.dll      api-ms-win-crt-runtime-l1-1-0.dll      msvcp140_2.dll                     vcruntime140_1.dll
api-ms-win-core-processthreads-l1-1-1.dll      api-ms-win-crt-stdio-l1-1-0.dll        msvcp140_codecvt_ids.dll           zlib.dll
api-ms-win-core-profile-l1-1-0.dll             api-ms-win-crt-string-l1-1-0.dll       pkgs
api-ms-win-core-rtlsupport-l1-1-0.dll          api-ms-win-crt-time-l1-1-0.dll         pre_uninstall.bat

So you should use C:\Miniconda\_conda.exe env export ... (when using cmd.exe or powershell) or alternatively /c/Miniconda/_conda.exe env export ... (when using bash)...

[jonashaag]

It looks like you can also just use Micromamba instead of Conda.

[pavelzw]

I think the conda env export part (especially with pip dependencies) is not directly transferable to micromamba.

[JohannesWiesner]

So this works fine (generating a .yml file):

/c/Users/runneradmin/micromamba-bin/micromamba.exe env export -p "${MAMBA_ROOT_PREFIX}/envs/${{ runner.os }}" | grep -v "^prefix: " > ./environments/csp_${{ runner.os }}_solved.yml

but when executing this (generate an explicit spec-file):

/c/Users/runneradmin/micromamba-bin/micromamba.exe list --explicit -p "${MAMBA_ROOT_PREFIX}/envs/${{ runner.os }}" > ./environments/csp_${{ runner.os }}_solved.txt`

I got this:

The following argument was not expected: --explicit

which makes sense because micromamba list seems to be implemented in micromamba but not micromamba list --explicit. If you want to get an explicit list you still have to rely on conda list --expcicit as described in the docs or run:

`/c/Users/runneradmin/micromamba-bin/micromamba.exe env export -p "${MAMBA_ROOT_PREFIX}/envs/${{ runner.os }}" --explicit > ./environments/csp_${{ runner.os }}_solved.txt`

[pavelzw]

Alright it seems that the docs are out of date. They should point to micromamba env export --explicit for explicit files.

[pavelzw]

But then I would say the easiest solution would be

micromamba env export -n "${{ matrix.os }}" > ./environments/csp_${{ matrix.os }}_solved.yml
micromamba env export --explicit -n "${{ matrix.os }}" > ./environments/csp_${{ matrix.os }}_solved.txt

If you rely on pip dependencies being added to the export, then you need to call conda (which can be done on windows runners with /c/Miniconda/_conda.exe env export ...)

[JohannesWiesner]

If you rely on pip dependencies being added to the export, then you need to call conda (which can be done on windows runners with /c/Miniconda/_conda.exe env export ...)

Just to be clear here: /c/Miniconda/ will automatically be installed when using mamba-org/setup-micromamba@v1?

[jonashaag]

No, only Micromamba will be installed, no Conda

[pavelzw]

/c/Miniconda/ is already shipped when using windows-latest.

[JohannesWiesner]

Okay, I completely had to switch back to conda commands for both the Linux and Windows jobs. This is how my code looks now (see full code here):

if [ "$RUNNER_OS" == "Windows" ]; then
  /c/Miniconda/_conda.exe env export -p "${MAMBA_ROOT_PREFIX}/envs/${{ matrix.os }}" | grep -v "^prefix: " > ./environments/csp_${{ matrix.os }}_solved.yml
  /c/Miniconda/_conda.exe list -p "${MAMBA_ROOT_PREFIX}/envs/${{ matrix.os }}" --explicit > ./environments/csp_${{ matrix.os }}_solved.txt
elif [ "$RUNNER_OS" == "Linux" ]; then
  conda env export -p "${MAMBA_ROOT_PREFIX}/envs/${{ matrix.os }}" | grep -v "^prefix: " > ./environments/csp_${{ matrix.os }}_solved.yml
  conda list -p "${MAMBA_ROOT_PREFIX}/envs/${{ matrix.os }}" --explicit > ./environments/csp_${{ matrix.os }}_solved.txt
fi

For the linux-runner, everything works fine but for the windows-runner, I get:

Run if [ "$RUNNER_OS" == "Windows" ]; then
  if [ "$RUNNER_OS" == "Windows" ]; then
    /c/Miniconda/_conda.exe env export -p "${MAMBA_ROOT_PREFIX}/envs/windows-latest" | grep -v "^prefix: " > ./environments/csp_windows-latest_solved.yml
    /c/Miniconda/_conda.exe list -p "${MAMBA_ROOT_PREFIX}/envs/windows-latest" --explicit > ./environments/csp_windows-latest_solved.txt
  elif [ "$RUNNER_OS" == "Linux" ]; then
    conda env export -p "${MAMBA_ROOT_PREFIX}/envs/windows-latest" | grep -v "^prefix: " > ./environments/csp_windows-latest_solved.yml
    conda list -p "${MAMBA_ROOT_PREFIX}/envs/windows-latest" --explicit > ./environments/csp_windows-latest_solved.txt
  fi
  shell: C:\Program Files\Git\usr\bin\bash.EXE -el {0}
  env:
    pythonLocation: C:\hostedtoolcache\windows\Python\3.7.9\x64
    PKG_CONFIG_PATH: C:\hostedtoolcache\windows\Python\3.7.9\x64/lib/pkgconfig
    Python_ROOT_DIR: C:\hostedtoolcache\windows\Python\3.7.9\x64
    Python[2](https://github.com/JohannesWiesner/tcy/actions/runs/5322826699/jobs/9639798139#step:7:2)_ROOT_DIR: C:\hostedtoolcache\windows\Python\[3](https://github.com/JohannesWiesner/tcy/actions/runs/5322826699/jobs/9639798139#step:7:3).7.9\x6[4](https://github.com/JohannesWiesner/tcy/actions/runs/5322826699/jobs/9639798139#step:7:4)
    Python3_ROOT_DIR: C:\hostedtoolcache\windows\Python\3.7.9\x[6](https://github.com/JohannesWiesner/tcy/actions/runs/5322826699/jobs/9639798139#step:7:6)4
    MAMBA_ROOT_PREFIX: C:\Users\runneradmin\micromamba
    MAMBA_EXE: C:\Users\runneradmin\micromamba-bin\micromamba.exe
    CONDARC: C:\Users\runneradmin\micromamba-bin\.condarc

CommandNotFoundError: No command 'conda env'.

A few take-aways:

1.) Generating a spec-file (as documented here): The command conda env export --explicit (in contrast to micromamba env export --explicit) does not exist. For conda the equivalent to micromamba env export --explicit is conda list --explicit. I am aware that this file will not include pip packages which is addressed in this issue and which is out of scope of setup-micromamba@v1. A possible workaround is to create a separate requirements.txt file as mentioned in this comment.

2.) Generating a .yml file (which can serve as an alternative to a spec-file.txt + requirements.txt and which will include pip-packages. The drawback here is, that it's not an exact recipe): In order to get a file that also includes pip-packages you have to use conda env export -p "${MAMBA_ROOT_PREFIX}/envs/name_of_my_environment", whereas micromamba env export -p "${MAMBA_ROOT_PREFIX}/envs/name_of_my_environment" will not include pip-packages as mentioned here

[pavelzw]

Btw you can also use this syntax in GitHub actions

- run: |
    conda env export -p "${MAMBA_ROOT_PREFIX}/envs/${{ matrix.os }}" | grep -v "^prefix: " > ./environments/csp_${{ matrix.os }}_solved.yml
    conda list -p "${MAMBA_ROOT_PREFIX}/envs/${{ matrix.os }}" --explicit > ./environments/csp_${{ matrix.os }}_solved.txt
  if: runner.os == 'Linux'
- run: |
    /c/Miniconda/_conda.exe env export -p "${MAMBA_ROOT_PREFIX}/envs/${{ matrix.os }}" | grep -v "^prefix: " > ./environments/csp_${{ matrix.os }}_solved.yml
    /c/Miniconda/_conda.exe list -p "${MAMBA_ROOT_PREFIX}/envs/${{ matrix.os }}" --explicit > ./environments/csp_${{ matrix.os }}_solved.txt
  if: runner.os == 'Windows'

CommandNotFoundError: No command 'conda env'.

My guess would be that conda needs to be initialized for it to work... Did you try running conda init, see here.

[JohannesWiesner]

Okay, I solved it.

My guess would be that conda needs to be initialized for it to work... Did you try running conda init, see here.

I followed the steps from this medium post and added the following step to my workflow to initialize conda for Git Bash:

- name: initialize conda for Git Bash (will be skipped for Linux)
  run: |
    if [ "$RUNNER_OS" == "Windows" ]; then
      echo ". /c/Miniconda/etc/profile.d/conda.sh" >> ~/.bashrc
    fi

Interestingly, this step has to be done before running mamba-org/setup-micromamba@v1 otherwise I ended up with the same error (CommandNotFoundError: No command 'conda env'), I don't know why the order is important here.

With this approach, it's now also not necessary to use /c/Miniconda/_conda.exe since conda is now a valid command. Feel free to close this issue.

[pavelzw]

Interestingly, this step has to be done before running mamba-org/setup-micromamba@v1

The ~/.bashrc only gets sourced when using the interactive shell mode (i.e. bash -i). This is the default on Linux hence it is described in the Medium post.

On macOS and Windows on the other hand, the ~/.bash_profile gets sourced when starting bash. You can (and do in your code) simulate this behavior in CI using bash -l.

Since the bashrc doesn't get sourced in CI (since it is not an interactive shell and you didn't specify bash -i), the command didn't do anything

The pip install ... step in your CI fixed your setup

I would suggest that you replace the bashrc part with bash_profile

- name: initialize conda for Git Bash (will be skipped for Linux)
  run: |
    if [ "$RUNNER_OS" == "Windows" ]; then
      echo ". /c/Miniconda/etc/profile.d/conda.sh" >> ~/.bash_profile
    fi

Then it should work no matter if you put it in front of the setup-micromamba step or behind it.