FatesPFTIndexSwapper IndexError

[glemieux]

Attempting to run the tool from the elm-fates container is generating a python error code:

Creating Variable:  fates_canopy_closure_thresh
Traceback (most recent call last):
  File "/E3SM/components/elm/src/external_models/fates/tools/./FatesPFTIndexSwapper.py", line 296, in <module>
    main(sys.argv)
  File "/E3SM/components/elm/src/external_models/fates/tools/./FatesPFTIndexSwapper.py", line 207, in main
    out_var.assignValue(float(fp_in.variables.get(key).data))
  File "/usr/local/lib/python3.9/dist-packages/scipy/io/_netcdf.py", line 943, in assignValue
    self.data[:] = value
IndexError: too many indices for array: array is 0-dimensional, but 1 were indexed
sys:1: RuntimeWarning: Cannot close a netcdf_file opened with mmap=True, when netcdf_variables or arrays referring to its data still exist. All data arrays obtained from such files refer directly to data on disk, and must be copied before the file can be cleanly closed. (See netcdf_file docstring for more information on mmap.)

I suspect this is likely an issue with python dependency versioning. The dockerfile simply runs pip install on the necessary dependencies per this line: https://github.com/NGEET/tutorial-containers/blob/18f32cecfb07f1772245a86fc172aea53a239459/docker/baseos/gcc1120/Dockerfile#L170

[glemieux]

I was able to replicate this issue via a new conda environment with the following versions:

• python: 3.9.3
• netCDF4: 1.7.1 (using library 4.9.2)
• numpy: 2.0.2
• scipy: 1.13.1

Testing this with an older conda environement I have results in the tool successfully completing:

• python: 3.7.3
• netCDF4: 1.5.1.2 (using library 4.6.2)
• numpy: 1.12.6
• scipy: 1.5.3

[glemieux]

@ckoven noted this is probably due to the switch to numpy 2.0, which apparently introduced breaking changes.

[glemieux]

This appears to be an issue with scipy. The maximum version that appears to work is 1.11.4. To discover this, I built minimum conda environments only installing scipy at a specific version and incrementing the version for each build. The numpy version conda solved for is 1.26.4. Note that this version was also solved for with the next newer version of scipy that failed.