⬆️ Maintenance upgrades tests and tools libraries

[pcrespov]

What do these changes do?

Updates tests and tooling libraries before updating services. Since it was a while since last update, there are many changes.
Nonetheless, i call your attention on the major updates, namely:

• pytest 8: https://docs.pytest.org/en/stable/changelog.html#breaking-changes
• dask-family (only testing)
• black (tooling)
• cryptograph (only testing)
• docker (only testing)
• httpcore (only testing)
• watchdog (only testing)

Changes in code

• ♻️ ruffed all old setup.py
• ⬆️ api/test: removed deprecated openapi_spec_validator.validate_spec
• 🎨 bad tests in test_httpx_utils.py that hard-coded version
• 🗑️ pytest-lazyfixture fails with pytest 8 (SEE Raise error with pytest==8.0.0 TvoroG/pytest-lazy-fixture#65): Modified the tests code to use parametrized fixtures instead of lazy-fixtures and removed the dependency.
• 🎨 uses pytest-asyncio (constraint to <0.22 due to Breaking change in 0.23.* pytest-dev/pytest-asyncio#706).
• 🗑️ Removes pytest-aiohttp where is not needed. NOTE: it is necessary until aiohttp.pytest_plugin.loop fixture retired!
• aiohttp<3.9: Changes in 3.9 needs to be addressed on a separate PR
• ♻️ migration scripts in tests to get it to work after the update!

In detail

#	name	before	after	upgrade	count	packages
1	aiohttp	3.8.5, 3.8.6, 3.9.1	3.8.6	🔥 downgrade	7	aws-library🧪 dask-sidecar🧪 dask-task-models-library🧪 environment-setup🧪 models-library🧪 osparc-gateway-server🧪 postgres-database🧪
2	arrow	1.3.0	🗑️ removed		3	environment-setup🧪 service-library🧪🧪
3	dnspython	2.4.2	🗑️ removed		1	service-library🧪
4	email-validator	2.1.0.post1	🗑️ removed		1	service-library🧪
5	idna	3.6, 3.4	3.4	🔥 downgrade	4	dask-task-models-library🧪 models-library🧪 osparc-gateway-server🧪 postgres-database🧪
6	jsonschema	4.19.2	🗑️ removed		2	environment-setup🧪 service-library🧪
7	jsonschema-specifications	2023.7.1	🗑️ removed		2	environment-setup🧪 service-library🧪
8	markdown-it-py	3.0.0	🗑️ removed		3	environment-setup🧪 service-library🧪🧪
9	mdurl	0.1.2	🗑️ removed		3	environment-setup🧪 service-library🧪🧪
10	orjson	3.9.10	🗑️ removed		3	environment-setup🧪 service-library🧪🧪
11	pydantic	1.10.13	🗑️ removed		1	service-library🧪
12	pygments	2.16.1	🗑️ removed		3	environment-setup🧪 service-library🧪🧪
13	pytest-aiohttp	1.0.5	🗑️ removed		10	aws-library🧪 dask-sidecar🧪 dask-task-models-library🧪 director-v2🧪 environment-setup🧪 models-library🧪 postgres-database🧪 simcore-sdk🧪 swarm-deploy🧪 tests🧪
14	pytest-asyncio	0.21.1, 0.23.5, 0.23.2	0.21.1	🔥 downgrade	6	autoscaling🧪 aws-library🧪 dask-task-models-library🧪 notifications-library🧪 osparc-gateway-server🧪 public-api🧪
15	pytest-lazy-fixture	0.6.3	🗑️ removed		1	simcore-sdk🧪
16	referencing	0.29.3	🗑️ removed		2	environment-setup🧪 service-library🧪
17	rich	13.6.0	🗑️ removed		3	environment-setup🧪 service-library🧪🧪
18	rpds-py	0.10.6	🗑️ removed		2	environment-setup🧪 service-library🧪
19	typer	0.9.0	🗑️ removed		3	environment-setup🧪 service-library🧪🧪
20	types-python-dateutil	2.8.19.14	🗑️ removed		3	environment-setup🧪 service-library🧪🧪
21	yarl	1.9.4, 1.9.2	1.9.2	🔥 downgrade	7	aws-library🧪 dask-sidecar🧪 dask-task-models-library🧪 environment-setup🧪 models-library🧪 osparc-gateway-server🧪 postgres-database🧪

Legend:

• ⬆️ base dependency (only services because packages are floating)
• 🧪 test dependency
• 🔧 tool dependency

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 87.4%. Comparing base (cafbf96) to head (3d70cf0).
Report is 42 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff            @@
##           master   #5346      +/-   ##
=========================================
+ Coverage    84.5%   87.4%    +2.8%     
=========================================
  Files          10    1355    +1345     
  Lines         214   55945   +55731     
  Branches       25    1249    +1224     
=========================================
+ Hits          181   48905   +48724     
- Misses         23    6769    +6746     
- Partials       10     271     +261     

Flag	Coverage Δ
integrationtests	64.0% <ø> (?)
unittests	85.4% <ø> (+0.8%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

see 1338 files with indirect coverage changes

[bisgaard-itis]

Thanks a lot for the effort

[sanderegg]

great! thanks!

[matusdrobuliak66]

thanks 💯

[GitHK]

please ask me for a review when you mange to get the tests green again

[GitHK]

Thanks for all this. I see we are getting behind on upgrades and need to address all this

[sonarcloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
0.4% Duplication on New Code

See analysis details on SonarCloud