DOC: 2.0 release highlights and compat notes changes

[rgommers]

This is a draft of the narrative section of the 2.0.0 release notes, with highlights and main messages. I think it needs to be significantly longer than that of a normal feature release, and give the reader an overview of what's in this release as well as links to other key documentation like the migration guide and the page about packaging for downstream libraries.

For the highlights I chose six top-level categories and then 2-4 items for each of those categories. Which are:

• New features
• Performance improvements
• Python API improvements
• C API improvements
• Improved behavior
• Documentation

A lot of work has gone into this release, so I'm sure this isn't 100% complete or representative yet. A few questions to promt feedback on this draft:

• Are the above six categories the right ones?
• Are they in the right order? (note: I had it differently first, but noticed that starting with new features and performance is more concrete for the average user, and hence probably more appealing)
• What is missing? And what did I include that you think should not be there?

This PR also takes care of some leftover tasks for release notes and migration guide.

[rgommers]

Let me ping a few people for specific categories:

• Documentation: @Mukulikaa, @rossbar, @melissawm
• Static typing: @BvB93 (no major changes I think, but want to make sure)
• Performance: @seiko2plus, @r-devulap, @Mousius
• Platform support: @andyfaff (no major changes here either I think; musllinux went into 1.26.x)
• Any user-facing ufunc/gufunc features worth calling out that I missed? @seberg @mhvk

[r-devulap]

Performance section looks good to me. Just some minor edits, if it seems appropriate.

[mattip]

here is the rendered document, if that is easier for people to read.

[lysnikolaou]

Not sure if this is ready for review yet, but here's a first round.

[mattip]

Please sneak this snippet in, it was left out of #25914:

Removed ambiguity when broadcasting in ``np.solve``
---------------------------------------------------
The broadcasting rules for ``np.solve(a, b)`` were ambiguous when ``b`` had 1
fewer dimensions than ``a``. This has been resolved in a backward-incompatible
way and is now compliant with the Array API. The old behaviour can be
reconstructed by using ``np.solve(a, b[..., None])[..., 0]``.

(`gh-25914 <https://github.com/numpy/numpy/pull/25914>`__) 

cc @asmeurer

[seberg]

Not sure how, so fine. One thing that I wonder about is that e.g. NEP 50 and the Python removals is what users will actually care about most in terms of compatibility and in terms of being the largest changes and improvements in many ways.

I would be happy to iterate, thus the thought of adding a note.

[rgommers]

Please sneak this snippet in, it was left out of #25914:

Sure, putting that under Compatibility notes

[rgommers]

Thanks, accepted all suggestions that are resolved.

[mattip]

As expected, #25943 causes pandas import to fail, which fails the docs build.

[BvB93]

Let me ping a few people for specific categories:
* Static typing: @BvB93 (no major changes I think, but want to make sure)

Can confirm, yes; just regular maintenance to reflect the runtime changes.

[charris]

close/reopen

[mattip]

Close/reopen to get latest dev wheels

[mattip]

Maybe we should skip the doctest tests for now, until a pandas fix for the dtype is merged and new nightly wheels are uploaded?

[mattip]

Instead of skipping doctests, we could move them later in the CI steps, which AFAICT would still upload artifacts, see #25962

[rgommers]

I had the same thought. We use next to nothing from pandas, so it's a little annoying that our doc builds are blocked on it. However, if we break pandas we usually also break matplotlib, so in those cases it won't help.

[mattip]

so in those cases it won't help.

Right, in general it won't help. But for this PR specifically it will: #25962, which was merged to HEAD, does upload artifacts. So rebasing this off main will clear the merge conflict and allow seeing the rendered doc (as well as checking for new warnings).

[rgommers]

Thanks! Will rebase after the next update - which should make this merge-able (EDIT: not just yet).

[mattip]

The rendered doc is here. There some warnings it would be nice to clean up if possible

doc/source/release/2.0.0-notes.rst:937: WARNING: py:obj reference target not found: numpy.linalg.vecdot
doc/source/release/2.0.0-notes.rst:1350: WARNING: py:obj reference target not found: numpy.fft._helper
doc/source/release/2.0.0-notes.rst:1359: WARNING: py:obj reference target not found: numpy.linalg._linalg
doc/source/release/notes-towncrier.rst:7: WARNING: py:obj reference target not found: numpy.polynomial.Polynomial

[rgommers]

The rendered doc is here. There some warnings it would be nice to clean up if possible

We really have to get to zero warnings and turn on the "error on warning" behavior in CI. I'd say that at least 75% of all PRs that add release notes are introducing new issues.

[rgommers]

I think this is good to go now. It deserves another large copy-edit of the full notes, but that can be done in a few weeks time. This should be good enough for 2.0.0b1.

[mattip]

here are the rendered docs.

[mattip]

I am not sure how to review this, so I will just put it in as-is. Anyway it will not successfully be displayed as the devdocs until the pandas doctest passes and the build is un-broken. It is evident a lot of work went into this, thanks to all who contributed. There are some formatting glitches: some items are lists and some are topics, it would be nice to try to straighten that out for the next version.

[mattip]

Thanks @rgommers

[seberg]

Thanks!

the pandas doctest passes and the build is un-broken

Would it be easy to work around? I am slightly worried that the next in line is matplotlib and that might not be quite as quick.

[rgommers]

I am slightly worried that the next in line is matplotlib and that might not be quite as quick.

Is that in progress / do they need help? I haven't had problems yet with local doc builds, so I think the damage should be limited.

I'm babysitting the SciPy nightlies now, they will hopefully be up within the next hour.

[mattip]

Would it be easy to work around?

I think a PR to skip the doctest tests and then another to revert it would work, we would merge the second one once it passes.

[seberg]

Is that in progress / do they need help? I haven't had problems yet with local doc builds, so I think the damage should be limited.

OK, cool, for me it is just an unknown for whether they might hit pybind11 issues mainly (I doubt it from my first look, but hard to be sure).

Let's just see, maybe this is all very quick, if not we can still rearrange the upload.

[charris]

I'm not worried about the failing doc tests, just ignore them. They don't block the wheel releases and the 2.0.0 release is about a month away, which is when we need to generate the docs.