Remove upper bound for jupyterlab in kedro-starters and default template

[SajidAlamQB]

Following #2269 setuptools was pinned with an upper bound <67.0.0 as a temporary solution to avoid CI failures. We should investigate why the latest version of setuptools is breaking our CI.

Furthermore, in the same PR jupyterlab was also pinned with an upper bound <3.6.0 as it required jupyterlab_server~=2.19 but our requirements.txt contains:
jupyterlab_server>=2.11.1, <2.16.0 # 2.16.0 requires importlib_metedata >= 4.8.3 which conflicts with flake8 requirement

[astrojuanlu]

flake8 is pinned to <5.0, which means ~october 2021 (almost 2.5 years ago), see #1594

flake8 5.0 still pins importlib_metadata <4.3 on Python 3.8: https://github.com/PyCQA/flake8/blob/5.0.4/setup.cfg#L45

and then flake8 6.0 requires Python >=3.8.1 https://github.com/PyCQA/flake8/blob/b9a7794c4f425ef8419081e6993f99419cc667ea/setup.cfg#L40-L41

So I guess the only solution is to jump to flake8 6.x. But we won't be able to do that until later this year #2158

[astrojuanlu]

To note: JupyterLab 4.0 will be out soon, with major improvements. Pending user-friendly changelog jupyterlab/jupyterlab#14060

Maybe instead of waiting until we drop Python 3.7, we could do some selective requirements pinning depending on the Python version? i.e. we pin flake8<5.0 but only for Python 3.7.

[astrojuanlu]

One more question - trying to understand the pinning policy. Do we somehow pin in our CI the same as the requirements that are shipped in the default template?

kedro/kedro/templates/project/{{ cookiecutter.repo_name }}/src/requirements.txt

Lines 6 to 8 in 7f44733

	jupyter~=1.0
	jupyterlab_server>=2.11.1, <2.16.0
	jupyterlab~=3.0, <3.6.0

[astrojuanlu]

JupyterLab 4.0 is available https://pypi.org/project/jupyterlab/4.0.0/

[astrojuanlu]

See more context in #2700 (closed in favour of this one)

[noklam]

Updated title to reflect the requirements, kedro doesn't have dependency on jupyter, it is only requirements in the starters.

[astrojuanlu]

#3072 lifted some of the upper bounds in framework

[astrojuanlu]

This is how the dependencies of the default template look like now:

kedro/kedro/templates/project/{{ cookiecutter.repo_name }}/requirements.txt

Lines 1 to 9 in bf536d4

	ipython>=7.31.1, <8.0; python_version < '3.8'
	ipython~=8.10; python_version >= '3.8'
	jupyter~=1.0
	jupyterlab_server>=2.11.1, <2.16.0
	jupyterlab~=3.0, <3.6.0
	kedro~={{ cookiecutter.kedro_version }}
	kedro-telemetry~=0.2.0
	# Pin problematic traitlets release - https://github.com/jupyter/notebook/issues/7048
	traitlets<5.10.0

I think ideally we should

• Remove the requirements for python_version < '3.8' now that Python 3.7 got dropped in develop
• (Possibly) Remove ipython~=8.10 entirely (it's a dependency of jupyterlab anyway through ipykernel)
• Remove jupyter~=1.0 (this was never a real package)
• Remove jupyterlab_server (unclear why we're pinning this low-level package)
• Replace jupyterlab~=3.0, <3.6.0 with notebook~=7.0.0 or even notebook>=7.0 (this is a breaking change that upgrades users to the latest Jupyter Notebook, based on JupyterLab components https://blog.jupyter.org/announcing-jupyter-notebook-7-8d6d66126dcf)
• Remove traitlets<5.10.0 (that was a hotfix that has been addressed)

[astrojuanlu]

Follow-up in kedro-starters at kedro-org/kedro-starters#200