⚗️ web-server refactoring backlog

[pcrespov]

Refactoring to be done in webserver

• OBaseException pydantic-exception based base exception for all repo -> code+message. Umbrella
• remove middleware for openapi and envelope. The latter turn into a decorator instead
• Remove trafaret dependency by pydantic-settings @pcrespov
• Replace openapi-core by pydantic (evaluate https://github.com/Maillol/aiohttp-pydantic, https://pypi.org/project/aiohttp-asgi/)
• Remove openapi-core and use pydantic
• socket.io extract in separate service (or attach a broker to sync all replicas)
• Remove constraints to python-socketio
• Remove constraints to aiohttp
• OAS checked in tests and not upon startup. Loading OAS takes very long time in rest.py app module
• Refactor design of webserver/projects plugin
• Remove / entrypoint from web-server @GitHK

webserver/projects plugin

• review request/response in project CRUD at the OAS
  • reduce size of project items when listing, i.e. lists do not need to be that detailed
  • review read/write/nullable constraints
  • use hierarchichal to access subresources
  • standard naming of path and query parameters (e.g. ONLY project_uuid, or filtering query params)
• pydantic models for all interaces and contexts (see issues in Fix issues reported by validate-pg-project diagnose #2552)
• repository pattern: ProjectRepo class to interface the db and uses pydantic models to export/parse data
• reduce coupling with other plugins by defining a clear api. module
• profile plugin to speed up

python-socketio

From 5.0.0, https://github.com/miguelgrinberg/python-socketio/blob/main/CHANGES.md
test_resource_manager.py::test_websocket_resource_management fails because
socket_id saved in redis does not correspond to client's sio (see traceback in comments below).

plugins

• logging settings to change levels of noisy logger?
• plugin enhancements

[pcrespov]

• check manually if cookie is received in browser

[pcrespov]

python-socketio~=4.6.1

st_access_study_anonymously` fails with aiohttp 3.7.X #2386

Upgrading aiohttp caused a test to fail. Needs further investigation before completing the upgrade.

test_access_to_studies fails since client.session.cookiejar is empty when study is redirected.

aiohttp                   3.7.3
aiohttp-jinja2            1.2.0
aiohttp-security          0.4.0
aiohttp-session           2.9.0
aiohttp-swagger           1.0.15

    
>       expected_prj_id = await assert_redirected_to_study(resp, client.session)

tests/unit/with_dbs/fast/test_access_to_studies.py:266: ...
        # Expects auth cookie for current user
>       assert "osparc.WEBAPI_SESSION" in [c.key for c in session.cookie_jar]
E       AssertionError: assert 'osparc.WEBAPI_SESSION' in []

tests/unit/with_dbs/fast/test_access_to_studies.py:187: AssertionError

aiohttp                   3.7.2
aiohttp-jinja2            1.2.0
aiohttp-security          0.4.0
aiohttp-session           2.9.0
aiohttp-swagger           1.0.15

        # Expects auth cookie for current user
>       assert "osparc.WEBAPI_SESSION" in [c.key for c in session.cookie_jar]
E       AssertionError: assert 'osparc.WEBAPI_SESSION' in []

tests/unit/with_dbs/fast/test_access_to_studies.py:187: AssertionError

aiohttp                   3.7.1
aiohttp-jinja2            1.2.0
aiohttp-security          0.4.0
aiohttp-session           2.9.0
aiohttp-swagger           1.0.15

        # Expects auth cookie for current user
>       assert "osparc.WEBAPI_SESSION" in [c.key for c in session.cookie_jar]
E       AssertionError: assert 'osparc.WEBAPI_SESSION' in []

tests/unit/with_dbs/fast/test_access_to_studies.py:187: AssertionError

aiohttp                   3.7.0
aiohttp-jinja2            1.2.0
aiohttp-security          0.4.0
aiohttp-session           2.9.0
aiohttp-swagger           1.0.15

        # Expects auth cookie for current user
>       assert "osparc.WEBAPI_SESSION" in [c.key for c in session.cookie_jar]
E       AssertionError: assert 'osparc.WEBAPI_SESSION' in []

next downgrade is 3.6.3

aiohttp                   3.6.3
aiohttp-jinja2            1.2.0
aiohttp-security          0.4.0
aiohttp-session           2.9.0
aiohttp-swagger           1.0.15

and test passes ...