Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨Personalized resource limits: add project nodes table to store nodes resources 🗃️ #4362

Conversation

sanderegg
Copy link
Member

@sanderegg sanderegg commented Jun 13, 2023

What do these changes do?

  • creates a new projects_nodes table and projects_to_projects_nodes table
  • projects_nodes contains currently only node requirements as a JSONB dictionary
  • conceptually will allow migrating the projects.workbench completely to the projects_nodes which has a number of advantages:
    • faster access of nodes per DB row (no need to get the whole workbench JSON each time)
    • faster patching: patch 1 node without having to lock the entire workbench while doing so
    • faster search
    • will allow sharing of nodes among projects (meta, re-use of nodes in separate projects)
  • following disadvantages:
    • need to handle both projects and projects_nodes/projects_to_nodes at the moment
    • copy of projects must be adapted
    • computations must be adapted

Bonus:

  • postgres-database testing now runs every test once with the sqlalchemy models directly, once with the alembic migration to detect migration discrepencies
  • reduced sql2.0 warnings

Related issue/s

How to test

DevOps Checklist

@sanderegg sanderegg added this to the Watermelon milestone Jun 13, 2023
@sanderegg sanderegg self-assigned this Jun 13, 2023
@sanderegg sanderegg added the a:database associated to postgres service and postgres-database package label Jun 13, 2023
@sanderegg sanderegg changed the title ✨Personalized resource limits/add project nodes table ✨Personalized resource limits: add project nodes table to store nodes resources Jun 13, 2023
@codecov
Copy link

codecov bot commented Jun 13, 2023

Codecov Report

Merging #4362 (3000ead) into master (337b8e7) will decrease coverage by 2.3%.
The diff coverage is 98.2%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master   #4362      +/-   ##
=========================================
- Coverage    85.6%   83.4%    -2.3%     
=========================================
  Files         463     844     +381     
  Lines       22348   37224   +14876     
  Branches      554     461      -93     
=========================================
+ Hits        19138   31054   +11916     
- Misses       3092    6054    +2962     
+ Partials      118     116       -2     
Flag Coverage Δ
integrationtests 68.1% <ø> (?)
unittests 81.4% <98.2%> (-4.3%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...s-database/src/simcore_postgres_database/errors.py 100.0% <ø> (ø)
...e/src/simcore_postgres_database/utils_aiopg_orm.py 84.5% <ø> (ø)
...tabase/src/simcore_postgres_database/_protocols.py 60.0% <50.0%> (ø)
...simcore_postgres_database/models/projects_nodes.py 100.0% <100.0%> (ø)
...gres_database/models/projects_to_projects_nodes.py 100.0% <100.0%> (ø)
.../simcore_postgres_database/utils_projects_nodes.py 100.0% <100.0%> (ø)

... and 655 files with indirect coverage changes

@sanderegg sanderegg force-pushed the personalized-resource-limits/add-project-nodes-table branch 2 times, most recently from 301db71 to 092c9f7 Compare June 15, 2023 14:44
@sanderegg sanderegg marked this pull request as ready for review June 15, 2023 16:55
@sanderegg sanderegg changed the title ✨Personalized resource limits: add project nodes table to store nodes resources ✨Personalized resource limits: add project nodes table to store nodes resources 🗃️ Jun 15, 2023
Copy link
Contributor

@matusdrobuliak66 matusdrobuliak66 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good 👍 nice that we started to normalize tables

Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic!
just some thoughts to discuss tomorrow on-site

Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some initial comments. I would like to take about this with you in person.

@sanderegg sanderegg requested a review from GitHK June 16, 2023 12:52
@sanderegg sanderegg force-pushed the personalized-resource-limits/add-project-nodes-table branch from 874346f to 91e9357 Compare June 16, 2023 12:55
@sanderegg sanderegg force-pushed the personalized-resource-limits/add-project-nodes-table branch from 91e9357 to a9909b7 Compare June 16, 2023 14:28
@sanderegg sanderegg force-pushed the personalized-resource-limits/add-project-nodes-table branch from a9909b7 to 3000ead Compare June 16, 2023 15:09
@sonarcloud
Copy link

sonarcloud bot commented Jun 16, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@codeclimate
Copy link

codeclimate bot commented Jun 16, 2023

Code Climate has analyzed commit 3000ead and detected 0 issues on this pull request.

View more on Code Climate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:database associated to postgres service and postgres-database package
Projects
None yet
Development

Successfully merging this pull request may close these issues.

update_modified_timestamp_producure is not sql2 compatible
4 participants