Unpin build-system requirements, but impose an upper-bound

[DMRobertson]

Fixes #14079, properly. Reverts #14080.

A summary of the problem:

1. Synapse defines an extra called url_preview with an underscore.
2. Synapse looks up its own packaging metadata at runtime to check if the dependencies needed for url_preview are installed.
3. Poetry-core 1.3.0 now normalises extra names when writing metadata, per PEP 685.

• This means the extra will be written to metadata as url-preview with a hyphen.
• So the lookup in point 2 will fail if Synapse was built/installed by poetry-core 1.3.0.

Fix ourselves to be forward compatible by changing the spelling of url_preview to url-preview. Call this out in the upgrade notes.

Remark. I considered adding compatibility code to check_dependencies.py which could cope with denormalised extra names, both in source code and in the the on-disk metadata. But hat would only be helpful for someone with stale metadata: for instance, someone who has git pulled but not poetry install --all-extrased. On reflection, I deemed this not important enough to work around for two reasons:

• In light of the rust changes, admins installing from source will have to poetry install (or similar) frequently anyway, or else Synapse will refuse to start because of an outdated Rust library.
• I doubt many people are installing the url-preview extra in isolation: I think most people just use all.

[richvdh]

are either of these things that people are likely to be doing every upgrade?

I'm not entirely sure this needs calling out in the upgrade notes.

[DMRobertson]

As noted in the description: if you're installing from source, you need to be doing pip install or poetry install to pick up the latest version of the Rust source. So reinstalls are likely.

I think installations that specifically name url_previews are unlikely.

(Also a factor: if you install an extra with poetry install --extras EXTRA_NAME once, you have to keep supplying that EXTRA_NAME in future poetry install calls, or else it will remove unused dependencies.)

[richvdh]

right, fair.

I think installations that specifically name url_previews are unlikely.

Yeah. On this occasion, I feel like this is unlikely to be useful to anyone. We do need to find some sort of balance in these upgrade notes: if they describe every single change, then people won't read them at all. IMHO this is on the "not worth including" side of the balance (though it might be worth an explicit shoutout in the changelog). YMMV though.

[DMRobertson]

I mainly want to have something here to point people to if they encounter this problem on upgrade. Can remove it if you think it's too niche.

[DMRobertson]

Ahh, sorry. I missed your earlier comment.

I'll omit this from the notes and add a "BTW" in the changelog.

[DMRobertson]

This is a722cf5.

[richvdh]

perhaps we should cap these to known-good versions, to avoid #13849 situations?

[DMRobertson]

No strong opinions here. I'd be happy to cap to poetry-core>=1.0.0,<=1.3.0, but I don't have any context on what's going on with setuptools_rust and how quickly-moving it is. E.g. I expect we'll want to take advantage of PyO3/setuptools-rust#295 sooner rather than later.

[ahesford]

Please don't do <=1.3.0, version 1.3.1 is already out. If you're going to constrain the upper end of the version, at least make it <1.4.

[DMRobertson]

Discussed among the team. We'll constrain to <= 1.3.1 and will raise that upper bound to 1.3.2/1.4.0/another version if a) someone wants to use that version, and b) we can check that it's safe.

[DMRobertson]

This is 045cecd.

[ahesford]

I think making a too-conservative upper bound conserves frustration by replacing "Synapse is broken" issues with "Requirements need updating" issues. Limiting to the current minor version might thread the needle by minimizing the likelihood of breakage while still allowing upgrades. (Of course, given how some Python projects seem to bump patch versions with ever merge and minor versions every third hour of every fourth day, maybe the relaxation isn't all that significant.)