Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 agent service followup fixes #3513

Merged
merged 21 commits into from
Nov 11, 2022

Conversation

GitHK
Copy link
Contributor

@GitHK GitHK commented Nov 4, 2022

What do these changes do?

This is a followup to #3465

  • agent now properly mounts the docker volumes on the machine
  • fixes an issue with docker-compose config where the cpu is not accepted.

Bonus: editing non service files will trigger CI jobs

Related issue/s

How to test

Checklist

@GitHK GitHK marked this pull request as ready for review November 4, 2022 14:11
@GitHK GitHK self-assigned this Nov 4, 2022
@GitHK GitHK added the bug buggy, it does not work as expected label Nov 4, 2022
@GitHK GitHK added this to the Katherine Switzer milestone Nov 4, 2022
Makefile Outdated
@@ -242,7 +242,7 @@ CPU_COUNT = $(shell cat /proc/cpuinfo | grep processor | wc -l )
# -> filestash config at $(TMP_PATH_TO_FILESTASH_CONFIG)
@$(shell \
export TMP_PATH_TO_FILESTASH_CONFIG="${TMP_PATH_TO_FILESTASH_CONFIG}" && \
docker-compose --env-file .env --file services/docker-compose-ops.yml --log-level=DEBUG config > $@ \
docker-compose --env-file .env --file services/docker-compose-ops.yml --log-level=DEBUG config | sed -E "s/cpus: ([0-9\\.]+)/cpus: '\\1'/" > $@ \
Copy link
Member

@pcrespov pcrespov Nov 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I understand you intend to replace the cpus: field to enforce single quotes ' right? If so the way you do it has quite some problems: Check your regex against these example https://regex101.com/r/a99Y5z/1

Moreover, if the problem is docker-compose ... please check the version and the specs for it. Typically it is a problem with incompatibilities between the version of docker-compose (i.e. the executable that runs config) and the version of the compose specs file.

I would prefer solving this by aligning versions than patching the docker-compose in the makefile with sed. Notice how messy would it be if we have to do this with other fields. ..

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As pointed out by the post above, it's an issue with docker-compose that is not getting fixed. Since It is currently frozen at version 1.29.2. We have not upgraded to version 2.x.x which I'm not sure that if fixes this issue.

Copy link
Member

@pcrespov pcrespov Nov 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, so then I propose that you add a comment in this change so that we can follow up when we do the upgrade of docker-compose.
IMO patching the docker-compose this way should only be done in exceptional circumstances (this could be one of them!).

Regarding the regex... It realize now that I do not understand how really sed works because

echo cpus:    "4.3" | sed -E "s/cpus: ([0-9\\.]+)/cpus: '\\1'/"
cpus: '4.3'

echo "cpus:    '4.3'" | sed -E "s/cpus: ([0-9\\.]+)/cpus: '\\1'/"
cpus:    '4.3'

despite the fact that the regex clearly misses spaces \w and the quotes as shown in https://regex101.com/r/a99Y5z/1

@sonarcloud
Copy link

sonarcloud bot commented Nov 4, 2022

Please retry analysis of this Pull-Request directly on SonarCloud.

@GitHK GitHK requested a review from pcrespov November 4, 2022 14:32
Copy link
Member

@mrnicegyu11 mrnicegyu11 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please capture the env-vars noted in #3465 in the docker-compose.yml

Copy link
Member

@pcrespov pcrespov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add a comment in this change with info about the issue so that we can follow up when we do the upgrade of docker-compose
thx

Makefile Outdated Show resolved Hide resolved
@GitHK GitHK changed the title 🐛 agent service fixes 🐛 agent service followup fixes Nov 7, 2022
@GitHK GitHK enabled auto-merge (squash) November 8, 2022 14:43
@GitHK GitHK disabled auto-merge November 8, 2022 14:44
@GitHK GitHK enabled auto-merge (squash) November 9, 2022 10:23
@codecov
Copy link

codecov bot commented Nov 10, 2022

Codecov Report

Merging #3513 (541cfed) into master (d0fba0b) will decrease coverage by 0.2%.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff            @@
##           master   #3513     +/-   ##
========================================
- Coverage    82.6%   82.4%   -0.3%     
========================================
  Files         852     852             
  Lines       35978   35978             
  Branches      759     759             
========================================
- Hits        29752   29679     -73     
- Misses       6029    6102     +73     
  Partials      197     197             
Flag Coverage Δ
integrationtests 52.2% <ø> (-10.0%) ⬇️
unittests 80.5% <ø> (+<0.1%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
...imcore_service_webserver/exporter/export_import.py 34.1% <0.0%> (-63.5%) ⬇️
...vice_webserver/exporter/formatters/formatter_v2.py 31.9% <0.0%> (-58.6%) ⬇️
...vice_webserver/exporter/formatters/formatter_v1.py 27.6% <0.0%> (-55.9%) ⬇️
...ore_service_webserver/exporter/request_handlers.py 41.3% <0.0%> (-54.7%) ⬇️
...c/simcore_service_webserver/users_to_groups_api.py 46.1% <0.0%> (-53.9%) ⬇️
...core_service_webserver/exporter/file_downloader.py 48.0% <0.0%> (-52.0%) ⬇️
...re_service_webserver/exporter/formatters/models.py 55.1% <0.0%> (-42.4%) ⬇️
...rvice_webserver/exporter/formatters/base_models.py 48.9% <0.0%> (-40.9%) ⬇️
...src/simcore_service_webserver/exporter/settings.py 66.6% <0.0%> (-33.4%) ⬇️
...mcore_service_webserver/garbage_collector_utils.py 52.5% <0.0%> (-30.8%) ⬇️
... and 29 more

@sonarcloud
Copy link

sonarcloud bot commented Nov 10, 2022

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.2% 0.2% Duplication

@GitHK GitHK merged commit 2f6248e into ITISFoundation:master Nov 11, 2022
@GitHK GitHK deleted the pr-osparc-agent-missing branch November 11, 2022 08:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug buggy, it does not work as expected
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants