desisurvey broken with astropy 4.2

[sbailey]

Multiple desisurvey unit tests fail with astropy 4.2 due to "ModuleNotFoundError: No module named 'astropy._erfa.core'" due to "import astropy._erfa.core" in utils.py.

Whatever solution is found, it should also be backwards compatible with 4.0.1.post1 (what we currently use at NERSC) and whatever version of astropy is in the ICS environment for operations (it's not in the default desiobserver environment and I don't see a eups package to load, so I'm not sure what that config is).

======================================================================
ERROR: desisurvey.test.test_utils (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: desisurvey.test.test_utils
Traceback (most recent call last):
  File "/global/common/software/desi/cori/desiconda/20210326-1.5.0-spec/conda/lib/python3.9/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/global/common/software/desi/cori/desiconda/20210326-1.5.0-spec/conda/lib/python3.9/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/global/common/software/desi/users/sjbailey/desisurvey/py/desisurvey/test/test_utils.py", line 18, in <module>
    from desisurvey.test.base import Tester
  File "/global/common/software/desi/users/sjbailey/desisurvey/py/desisurvey/test/base.py", line 12, in <module>
    import desisurvey.ephem
  File "/global/common/software/desi/users/sjbailey/desisurvey/py/desisurvey/ephem.py", line 22, in <module>
    import desisurvey.utils
  File "/global/common/software/desi/users/sjbailey/desisurvey/py/desisurvey/utils.py", line 18, in <module>
    import astropy._erfa.core
ModuleNotFoundError: No module named 'astropy._erfa.core'

Mentioning @dkirkby @weaverba137 as original erfa workaround gurus. This is a blocking factor for upgrading astropy in our next desiconda release.

[sbailey]

earlier versions of astropy are unavailable in conda for python 3.9, thus requiring us to stick to python 3.8 until this issue is fixed

[sbailey]

@weaverba137 could you take a look at this astropy erfa issue and suggest a workaround that works with astropy 4.2 and 4.0.x? This is a blocking factor for upgrading to astropy 4.2 for the next software release. Although that isn't strictly required, it would be preferable to catch up with that instead of staying on 4.0.x and we'll need to address this at some point. In the meantime, maintaining compatibility with 4.0.x is important to support the KPNO operations environment which uses the the next tile selector in desisurvey.

[weaverba137]

I'll take a look, but I'm wiped out for the day. I don't think this is a hard problem. I believe there is now a pyerfa module, no longer part of astropy itself, that has to be imported instead. There shouldn't be any API changes, just an import change.

[weaverba137]

For reference: https://docs.astropy.org/en/stable/whatsnew/4.2.html#removed-erfa-module

I'll start testing this momentarily.

[weaverba137]

@dkirkby, what does astropy._erfa.core actually do in desisurvey.utils? As far as I can tell, it is imported but not used! If it is completely unused, then this could be a particularly easy fix.

[dkirkby]

@weaverba137 This is the commit where I added this import: 32fdde1

It was needed for the definition of astropy._erfa.core.ErfaWarning used in freeze_iers but that code now lives in desiutils.iers.

[weaverba137]

Aha, so this may need to be fixed in desiutil then. Good to know.

[weaverba137]

In fact, desiutil already handles this correctly: https://github.com/desihub/desiutil/blob/9dd77fe24f620d9282d4211fa881763c307ed890/py/desiutil/iers.py#L126

So I think the solution is just to remove the unused import statement.