Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Personalized resources: Allow override of computational service needed resources (⚠️ devops) #3989

Conversation

sanderegg
Copy link
Member

@sanderegg sanderegg commented Mar 17, 2023

What do these changes do?

This PR will bring the following:

Details:

replaces #3971

  • remove mpi flag as resource (catalog API /resources endpoint will not return MPI anymore)
  • director-v2 does not pass MPI as a task requirements anymore
  • director-v2 uses catalog to get the computational service resources instead of director-v0, thus allowing for resource override through table services_specifications
  • dask-sidecar boot mode, remove MPI and MPI advertisement
  • dask-sidecar computes VRAM on node using nvidia-smi, and registers
  • pass boot-mode from catalog to director-v2
  • pass boot-mode from director-v2 to dask-sidecar

Related issue/s

fixes ITISFoundation/osparc-issues#618

How to test

Checklist

@sanderegg sanderegg added a:catalog catalog service a:director-v2 issue related with the director-v2 service a:dask-service Any of the dask services: dask-scheduler/sidecar or worker labels Mar 17, 2023
@sanderegg sanderegg added this to the Mithril milestone Mar 17, 2023
@sanderegg sanderegg self-assigned this Mar 17, 2023
@codecov
Copy link

codecov bot commented Mar 17, 2023

Codecov Report

Merging #3989 (52348da) into master (4e7ecc4) will decrease coverage by 14.1%.
The diff coverage is 80.0%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master   #3989      +/-   ##
=========================================
- Coverage    83.0%   68.9%   -14.1%     
=========================================
  Files         944     579     -365     
  Lines       40778   23817   -16961     
  Branches      848     539     -309     
=========================================
- Hits        33862   16421   -17441     
- Misses       6706    7271     +565     
+ Partials      210     125      -85     
Flag Coverage Δ
integrationtests ?
unittests 68.9% <80.0%> (-13.2%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...core_service_api_server/api/routes/solvers_jobs.py 45.2% <0.0%> (ø)
.../simcore_service_api_server/modules/director_v2.py 53.5% <ø> (ø)
...service_webserver/director_v2_core_computations.py 61.6% <ø> (-33.7%) ⬇️
...vice_webserver/exporter/formatters/formatter_v1.py 27.6% <0.0%> (ø)
...rvice_webserver/projects/projects_handlers_crud.py 92.8% <ø> (ø)
..._service_webserver/studies_dispatcher/_projects.py 34.0% <0.0%> (-56.0%) ⬇️
...e_service_catalog/api/routes/services_resources.py 86.4% <83.3%> (-0.2%) ⬇️
...s-library/src/models_library/services_resources.py 90.7% <90.0%> (-0.4%) ⬇️
...simcore_service_webserver/projects/projects_api.py 78.9% <100.0%> (-2.4%) ⬇️

... and 497 files with indirect coverage changes

@sanderegg sanderegg force-pushed the personalized_resources/override_comp_services branch from ef791f1 to f5eb1c2 Compare March 17, 2023 16:15
@sanderegg sanderegg force-pushed the personalized_resources/override_comp_services branch 2 times, most recently from 3ea4c7f to 485e70c Compare March 20, 2023 13:10
@sanderegg sanderegg marked this pull request as ready for review March 20, 2023 15:45
@sanderegg sanderegg force-pushed the personalized_resources/override_comp_services branch from c3e0e1e to 1cdf4d8 Compare March 20, 2023 15:46
@sanderegg sanderegg changed the title ✨ Personalized resources: Allow override of computational service needed resources ✨ Personalized resources: Allow override of computational service needed resources (⚠️ devops) Mar 20, 2023
Copy link
Member

@mguidon mguidon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. This will be very useful

Copy link
Contributor

@GitHK GitHK left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 looks good to me

@sanderegg sanderegg force-pushed the personalized_resources/override_comp_services branch from f93dfdb to 2c4df34 Compare March 21, 2023 08:28
@sanderegg sanderegg force-pushed the personalized_resources/override_comp_services branch from 8dc8be7 to 3be0dbd Compare March 21, 2023 12:48
@codeclimate
Copy link

codeclimate bot commented Mar 21, 2023

Code Climate has analyzed commit 52348da and detected 1 issue on this pull request.

Here's the issue category breakdown:

Category Count
Security 1

View more on Code Climate.

@sonarcloud
Copy link

sonarcloud bot commented Mar 21, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@sanderegg sanderegg merged commit 95ef4e3 into ITISFoundation:master Mar 21, 2023
@sanderegg sanderegg deleted the personalized_resources/override_comp_services branch March 21, 2023 15:39
mrnicegyu11 pushed a commit to ITISFoundation/osparc-ops-environments that referenced this pull request Mar 22, 2023
@pcrespov pcrespov mentioned this pull request Mar 23, 2023
31 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:catalog catalog service a:dask-service Any of the dask services: dask-scheduler/sidecar or worker a:director-v2 issue related with the director-v2 service
Projects
None yet
Development

Successfully merging this pull request may close these issues.

personalizable resource limits remove the MPI flag for computational services
4 participants