PackagesNotFoundError

[benjamin-cash]

Hi @christinaholtNOAA @maddenp-noaa

I'm still working my way through the instructions to install uwtools, and I have made it past the build error from yesterday but have run into a new issue. When I try to create the uwtools environment I am getting the following error.

/work2/02441/bcash/frontera/workflow-tools
(uwtools_build) login2.frontera(1134)$ conda create -y -n uwtools --override-channels -c local uwtools
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed

PackagesNotFoundError: The following packages are missing from the target environment:

• uwtools

Do I need to give it a full path to uwtools or something along those lines?

[maddenp-noaa]

Hi @benjamin-cash -- Thanks for working through this with us. The feedback is very valuable.

Here's a transcript (with most of the output elided) of me running through the install recipe:

~ % mkdir uw-test
~ % cd uw-test
~/uw-test % wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-aarch64.sh
...
/uw-test % bash Miniforge3-Linux-aarch64.sh -bfp ./conda
...
~/uw-test % rm Miniforge3-Linux-aarch64.sh
~/uw-test % source ./conda/etc/profile.d/conda.sh
~/uw-test % conda activate
(base) ~/uw-test % conda install -y conda-build conda-verify
...
(base) ~/uw-test % git clone https://github.com/ufs-community/workflow-tools.git # not in the instructions, but I want to test with a fresh clone
...
(base) ~/uw-test % cd workflow-tools/
(base) ~/uw-test/workflow-tools % conda build recipe
...
(base) ~/uw-test/workflow-tools % conda create -y -n uwtools -c local uwtools
...
(base) ~/uw-test/workflow-tools % conda activate uwtools
(uwtools) ~/uw-test/workflow-tools % uw --help
usage: uw [-h] MODE ...

Unified Workflow Tools

Optional arguments:
  -h, --help
        Show help and exit

Positional arguments:
  MODE
    config
        Handle configs
    forecast
        Configure and run forecasts
    template
        Handle templates

Unfortunately (?) I didn't run into the same issue you did. I note, though, that you've added --override-channels to your conda create line, which tells conda to only take packages from channels you provide on the command line. That's a good idea if your [ana|mini]conda isn't one distributed by the conda-forge team, but then you would need to include a -c conda-forge switch so that you can get dependency packages from conda-forge. If you were taking advice from the first paragraph of the User Installation section, you might have seen --override-channels but missed the -c conda-forge bit. Could you try adding the latter to your conda create command?

By the way, after the conda build recipe command -- if the build is successful (you could share the output of that command if you have doubts) -- you should be able to locate the locally-built uwtools packages like so:

(base) ~ % conda search -c local uwtools
Loading channels: done
# Name                       Version           Build  Channel             
uwtools                        1.0.0    pyh233a6f4_0  home/maddenp/uw-test/conda/conda-bld
uwtools                        1.0.0    pyh28a5fc4_0  home/maddenp/uw-test/conda/conda-bld
uwtools                        1.0.0    pyh83875e1_0  home/maddenp/uw-test/conda/conda-bld

If conda search shows the uwtools packages for you, then the PackagesNotFoundError seems a bit misleading, as probably only uwtools was found and its dependency packages (jinja2, f90nml, etc.) would be the un-findable packages due to --override-channels and no -c conda-forge.

Let's see if adding -c conda-forge helps you.

Slightly longer term, we hope to be building and publishing these packages via our CI process so that a user could create a uwtools conda environment with a single command, perhaps

% conda create -y -n uwtools -c ufs-community uwtools

It's a hopefully temporary hack to have users build the packages locally and then install them.

[benjamin-cash]

Hi @maddenp-noaa - I went to my local cluster and started from scratch with a new miniforge install and was able to get through all of the steps without issue. So it is some kind of interaction between my existing conda installation and environments and the install from local step. Tomorrow I will go back onto Frontera and try from a clean miniforge install there as well and see if that solves the problem.

[maddenp-noaa]

@benjamin-cash We were able to get our public conda channel set up today, so you might like to try it out:

% conda create -y -n uwtools -c ufs-community uwtools
% conda activate uwtools
% uw --help

We still have a little work to do to get our CI configured to stage new packages when we release each version of workflow-tools, but the currently staged packages represent what's on our develop branch at the moment.

[benjamin-cash]

Hi @maddenp-noaa - This worked like a charm! Which is a real relief, because while I was able to get uwtools built and added to a new environment on my local cluster I was running into all sorts of headaches on Frontera. Too many different versions of python, miniconda, etc running around there I think. But my script to generate namelist files using the templater tool is running now, so I will call this a win. :)