Compatibility with unreleased ASDF stack for JWST s2d data

[pllim]

This will hopefully address spacetelescope/jdaviz#2446 . Minimally reproducible example that crashes without this patch using unreleased ASDF stack:

from astropy.utils.data import download_file
from specutils import Spectrum1D
fn = download_file('https://stsci.box.com/shared/static/exnkul627fcuhy5akf2gswytud5tazmw.fits', cache=True)
sp = Spectrum1D.read(fn)

🐱

[codecov]

Codecov Report

Merging #1079 (b6b0f44) into main (1674a27) will increase coverage by 0.00%.
The diff coverage is 50.00%.

@@           Coverage Diff           @@
##             main    #1079   +/-   ##
=======================================
  Coverage   70.72%   70.73%           
=======================================
  Files          64       64           
  Lines        4485     4483    -2     
=======================================
- Hits         3172     3171    -1     
+ Misses       1313     1312    -1     

Files Changed	Coverage Δ
specutils/io/default_loaders/jwst_reader.py	57.75% <50.00%> (+0.05%)	⬆️

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[braingram]

Looks good to me except I'm not sure the astropy dev testing change is working.
I see here it installed astropy 5.3.3:
https://github.com/astropy/specutils/actions/runs/6161782664/job/16721804170?pr=1079#step:6:35

[pllim]

Good catch...

[pllim]

But astropy 5.3.3 is also pulled into the devdeps job in unrelated PR (https://github.com/astropy/specutils/actions/runs/6147346407/job/16678644002). So something is pinning it but not caused by the changes here.

[braingram]

It also doesn't appear to be grabbing numpy 2.0:
https://github.com/astropy/specutils/actions/runs/6161782664/job/16721804170?pr=1079#step:6:59

:-|

[pllim]

BTW numpy is fine, it is grabbing RC instead of dev but that is a known issue.

[pllim]

For my own sanity, I moved the other fixes out to separate PRs. Now this PR only deals with the parser.

• DOC: Fix RTD #1080
• Debug devdeps #1081

[pllim]

Wut?

INTERNALERROR>   File "specutils/utils/wcs_utils.py", line 7, in <module>
INTERNALERROR>     from gwcs import WCS as GWCS
INTERNALERROR>   File "gwcs/__init__.py", line 72, in <module>
INTERNALERROR>     from .wcs import *   # noqa
INTERNALERROR>     ^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "gwcs/wcs.py", line 20, in <module>
INTERNALERROR>     from . import coordinate_frames as cf
INTERNALERROR>   File "gwcs/coordinate_frames.py", line 17, in <module>
INTERNALERROR>     from astropy.coordinates import StokesCoord
INTERNALERROR> ImportError: cannot import name 'StokesCoord' from 'astropy.coordinates' (astropy/coordinates/__init__.py)

[pllim]

@braingram , what is this? Is this really necessary in specutils?

specutils/conftest.py

Lines 9 to 14 in 85d07a9

	entry_points = []
	for entry_point in pkg_resources.iter_entry_points('pytest11'):
	entry_points.append(entry_point.name)
	if "asdf_schema_tester" not in entry_points:
	pytest_plugins = ['asdf.tests.schema_tester']

[braingram]

@braingram , what is this? Is this really necessary in specutils?

specutils/conftest.py

Lines 9 to 14 in 85d07a9

	entry_points = []
	for entry_point in pkg_resources.iter_entry_points('pytest11'):
	entry_points.append(entry_point.name)
	if "asdf_schema_tester" not in entry_points:
	pytest_plugins = ['asdf.tests.schema_tester']

I don't think that's required for any of the asdf schema testing. The testing should be enabled by:

specutils/setup.cfg

Lines 63 to 64 in 85d07a9

	asdf_schema_root = specutils/io/asdf/schemas
	asdf_schema_tests_enabled = true

See: https://asdf.readthedocs.io/en/latest/asdf/extending/schemas.html#testing-custom-schemas

Testing locally by removing the lines I still see the schemas tests. The CI does currently test the schemas, see:
https://github.com/astropy/specutils/actions/runs/6162339183/job/16723997336?pr=1079#step:6:155
I would expect that removing the lines shouldn't disable the spectrum1d-1.0.0.yaml etc tests.

[braingram]

I might have missed this on my first look but:

specutils/specutils/io/default_loaders/jwst_reader.py

Lines 570 to 573 in 85d07a9

	with asdf_in_fits.open(filename) as af:
	wcslist = [af.tree["meta"]["wcs"]]
	with fits.open(filename, memmap=False) as hdulist:

needs a similar fix as the one applied in this PR

[pllim]

Good catch, @braingram . I pushed new changes. Does it look good now?

[braingram]

Looks great! Thanks for fixing this.

[pllim]

I don't understand the GWCS error on devdeps job though. It seems unrelated but it is also blocking the test against asdf dev here.

[braingram]

My suspicion is it has something to do with astropy-iers-data. The logs show attempts to install several astropy versions:
https://github.com/astropy/specutils/actions/runs/6163642724/job/16727823617?pr=1079#step:6:48
before 'succeeding' with ...dev94... which is the newest version that doesn't list astropy-iers-data as a dependency. Why pip is doing this is a mystery to me.

[pllim]

Oh, right... That would theoretically be fixed by #1081... So I guess I will merge this and worry about the devdeps in that PR. 🤪

Thanks!

[pllim]

asdf 3.0 is released, so we also need to release this... 🐱

[rosteen]

I'll release today, thanks for the ping.

[pllim]

Thank YOU!