Practice around supported Python and Sphinx versions

[choldgraf]

Description

Two of the big dependencies for our stack are Python and Sphinx. We should decide on a policy to follow for "officially" supporting Python and Sphinx versions in our test suites.

Python

In #306 @chrisjsewell suggested that we use the Python end-of-life dates to decide which versions of Python to support.

I suggest that we do the following:

• Use the EOL dates to drop support for Python versions
• Test against the last 3 minor Python releases (so from today, it'd be 3.10, 3.9, 3.8)
• Document this practice

Sphinx

IN #306 we had some discussion and this seemed to land on officially supporting the last 2 major versions of Sphinx.

Tasks

• Any objections / edits to the practice described above?
• Document this in the contributing guide

[chrisjsewell]

Test against the last 3 minor Python releases (so from today, it'd be 3.10, 3.9, 3.8)

If you don't test against 3.7, then you are not supporting it: it'll be most likely that new code changes would break support for the oldest version of python than any other (using newer syntax features that are not supported or require a back port package).
At a minimum we should test against the oldest version supported and newest, i.e. from the end of the year 3.7 and 3.10

[choldgraf]

I guess the main concern is having tons of jobs in our CI/CD as new versions are released.

Questions

• What if we tested the oldest supported version, and the latest two released versions?
• What about Sphinx and major versions of Python? Do we test all supported versions of sphinx each Python version that we also test? It wouldn't surprise me if there are some versions of Sphinx that don't work with some versions of python, but I don't have data to back that up

[choldgraf]

(I also don't feel that strongly that we need to only test 3 python versions, if others don't think it's a big deal...it just seemed like a reasonable number to balance n_jobs and coverage)

[chrisjsewell]

Another thing I would suggest is that the (documented) aim should be for the the support to come within maybe 6 months of the release of new versions, e.g. as soon as sphinx v4 came out there were people asking for its support, which is understandable, but obviously depending on the number of breaking changes it's unlikely going to be feasible to roll out support for all JB dependencies within days

[mmcky]

I tend to agree we should test against all supported python versions.

@choldgraf perhaps we should force the tests to pass for python=3.7,3.8, and 3.9 and then add python=3.10 as a separate individual test on a single platform until python=3.10 is more mainstream (i.e. adopted in anaconda etc.).