Preparing for 3.0

[minrk]

I believe the updated scopes work is ready to release with #3877, which means starting the release process. The first decision in preparing this release is: do we call it 2.4 or 3.0?

Todo list:

• Review/merge remaining PRs
• Review docs for new/changed features
• changelog PR
• Decide 2.4 vs 3.0 (details below)

@manics re: dropping Python 3.6 support:

Are you releasing this in 2.4.0 or 3.0.0?

The next release has relatively little user-facing changes (mostly custom scopes - a pretty advanced feature), and should have no backward-compatible changes. But there are reasons to call it 3.0 anyway:

• It includes a schema change for the roles->scopes migration, which means db upgrade.
• Dropping a Python version support could be considered a backward-incompatble change, but with tooling (pip, conda, etc.) this doesn't have any real issues as older Python users simply won't get the new version as an installation candidate.

I think we can decide that for JupyterHub:

• Schema change (excluding wholly new tables which have no migration operation) only happens in major versions
• We can similarly decide that we only drop Python versions in major versions.

Making either of these choices would make the next release 3.0.

Neither of these are strictly required by semantic versioning, and we can always decide on a case by case basis at release time, as we're doing here.

A github compare is not useful because of the switch to isort in pre-commit.

To me, the biggest reason in favor of 2.x is that it will feel like a minor release, and anything that breaks is a bug and not an intentional change.

And the biggest reason in favor of 3.0 is the db upgrade step, and deciding we won't do that except in major revisions, since it adds an upgrade step for non-sqlite users, and makes rolling back more difficult (for everyone).

[manics]

I think 3.0 makes sense given the DB change.

From the point of view of maintaining our distributions (Z2JH, TLJH) do you think any JupyterHub upgrade that requires a manual step by an administrator (in this case jupyterhub upgrade-db) could count as a major change?

[consideRatio]

I lean towards 3.0 due to the need to do a db-upgrade =/

I consider the following:

1. Going for 2.4 would be disruptive for people that needs to make a db-upgrade and bumped to the new minor version even though they have a 2.* pin for example.
2. Going for 3.0 would make it obvious for people to look in the changelog for breaking changes.
3. Going for 3.0 could make people think they need to update jupyterhub in the user environment as well, while they could still use 2.3+ or similar - right?

Initially I thought dropping python versions didn't merit a breaking change based on what you described @minrk, but then I realized that there are jupyterhub in a server and jupyterhub in the user environment, and if you have different python versions they could get version mismatched because the latest available python version will be different if they are using different python versions. That made me think that maybe we should drop support of python versions as part of major versions anyhow after all - I'm not sure.

But, schema upgrades I think should happen only as part of major version upgrades.

[manics]

Dropping a Python version may not be technically breaking, but I find it quite annoying when dependabot makes a minor bump and my old CI tests start failing. I'd even support dropping a non-EOL Python if it fitted in with a major JupyterHub change and would avoid having to drop it in a later minor release.

[consideRatio]

I'd even support dropping a non-EOL Python if it fitted in with a major JupyterHub change and would avoid having to drop it in a later minor release.

I know that the entire dask ecosystem currently now only support py38-py310, to me, that sounds reasonable as well. If we drop python only as part of major releases, and want to avoid going for major releases just because of it, dropping them a bit earlier makes sense i think. Also, I think staying somewhat more modern is very reasonable as well in my mind.

Is dropping both py3.6+py3.7 an option? For reference, python 3.8 has been out since Oct. 14, 2019, almost three years, and Python 3.6 reached EOL 6 months and 3 weeks ago (23 Dec 2021). I'm a strong +1 at least on dropping support for Python versions before their EOL if we drop support for them as part of breaking versions.

[manics]

My preferred option is to commit to supporting 3.7 until the next major JupyterHub release even if it goes EOL, but if that's too much work then I'd rather drop 3.7 now.

[minrk]

OK, let's go with 3.0!

I feel better about the db-upgrade rule than the Python one, but I think we can adopt both to be extra conservative. We can also review that again when it comes time to drop 3.7.

3.6 went EOL in December, 3.7 is estimated to come in a year (2023-06). I don't anticipate continued support for 3.7 at least through EOL being a problem.

Given how limited JupyterHub's Python needs are (we really have no problems solved by dropping 3.6, let alone 3.7), I don't feel good about following more aggressive drop-support schedules used by packages that have larger challenges in Python version support. I think the best time to drop support for a Python version in a package like ours:

1. first release after EOL (2021-12 for 3.6, ~2023-06 for 3.7),
2. consider Python versions for at least two Ubuntu LTS releases (in March, that would have meant 18.04 which comes with 3.6, while 20.04 has 3.8)
3. earlier only when supporting an old Python version causes a real maintenance challenge (e.g. we want to adopt a feature that doesn't have an easy backport - this can happen, but never has for us)

Part of the reason for that is that dropping support for a not-yet-EOL Python puts more pressure on us to maintain backport releases, especially for security issues. If a collection of versions is reasonably supported, users have a reasonable expectation of fixes. We can't reasonably say 'please upgrade to latest' is the solution if that includes upgrading Python itself if their Python is still supported.

I think it's generally not a good practice to remove features or support until/unless their continued maintenance is a source of problems. In JupyterHub, supporting 3.6 had a small amount of friction, while dropping 3.7 gets us nothing that I'm aware of.

It would be sensible, for instance, for kubespawner to have a more aggressive baseline version because it's always run in a container and ~always in z2jh. JupyterHub itself has to be the most conservative component in terms of broad support, since it's the only component that has to work everywhere.

I'll note that requiring that the release that drops 3.7 be at least 4.0 doesn't mean it has to be a big release - if dropping 3.7 is the only major-increment-triggering change a year from now, that's fine, too! Just because it's major doesn't mean it has to be big.