TST: 400+ doctest failures because numpy 2.0.dev changed string representation

[pllim]

Example log:

https://github.com/astropy/astropy/actions/runs/5671453922/job/15368606312

Too many to post here but this might give you an idea:

Numpy: 2.0.0.dev0+629.g511f77d12
...
      at [0, 2]:
  -     a> 2
  -     b> 3
  +     a> np.int64(2)
  +      ?          ^
  +     b> np.int64(3)
  +      ?          ^
      ...and at 6 more indices.

Caused by:

• ENH: Update scalar representations as per NEP 51 numpy/numpy#22449

Also related maybe:

• TST: Run some doctest with numpy>=1.25 #14553

I think @mhvk has a cunning plan? 🙏

[pllim]

I think any fix for this would be too painful to backport so we might also want to do a numpy<2 pin on 5.3.x

[pllim]

Thanks to seberg, we can probably use --doctest-plus-generate-diff now to easily generate the new expected doctest outputs. It was implemented in scientific-python/pytest-doctestplus#227 and should be in https://github.com/scientific-python/pytest-doctestplus/releases/tag/v1.1.0 release.

[mhvk]

Clearly, I should have waited for such a nice automatic fix, but as it is we do have #15065 waiting in the wings to solve this once numpy 2.0 is out - though then we will need to run doctests on numpy 2.0 only.

[pllim]

though then we will need to run doctests on numpy 2.0 only

That is correct. It is too painful to run doctest on multiple numpy versions given all the changes. And there is little point is upgrading to 1.26 and then upgrade again. So I say we just jump from numpy<1.25 to numpy>=2 for doctest.

[neutrinoceros]

I think this should be triaged into 6.1.1
90% of the solution is already isolated in #16416, and we can probably extract backportable solutions for the other 10% from #15065. I'm game to take this on.

[pllim]

I really don't think it is that critical to have a backport branch to use numpy 2.0 for doctest. I still think it is fine to milestone this for v7.0 unless I have missed a situation where a backport is unavoidable?

[neutrinoceros]

oops, sorry I somehow didn't realize this issue was specificly about doctests. I agree doctests aren't critical. I'll open another issue for regular tests.

EDIT:

• numpy 2 compat: tests failing with default print options #16423