🐛 adds tests for S3TransferError; refactoring flaky CI test; better logging for long running task errors

[GitHK]

What do these changes do?

• Adds tests for _check_for_aws_http_errors.
• adjusts timeout detection interval in tests/fastapi/long_running_tasks/test_long_running_tasks_context_manager.py::test_task_result_times_out (lesson learned: when dealing with timeouts always use an order of magnitude of difference in checks)
• enhances logging when long running fastapi tasks finished with error

Related issue/s

• addresses tests/fastapi/long_running_tasks/test_long_running_tasks_context_manager.py::test_task_result_times_out in list of flaky tests in CI #3421
• will help figuring out S3TransferErrors S3TransferError 400 BadRequest with reason RequestTimeout #3531

How to test

Checklist

• Unit tests for the changes exist

[codecov]

Codecov Report

Merging #3525 (d874f7d) into master (31ac97f) will decrease coverage by 12.0%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master   #3525      +/-   ##
=========================================
- Coverage    80.5%   68.4%   -12.1%     
=========================================
  Files         852     503     -349     
  Lines       35991   22028   -13963     
  Branches      757     133     -624     
=========================================
- Hits        28985   15078   -13907     
- Misses       6822    6901      +79     
+ Partials      184      49     -135     

Flag	Coverage Δ
integrationtests	66.4% <ø> (-12.9%)	⬇️
unittests	77.3% <ø> (-2.2%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...-v2/src/simcore_service_director_v2/cli/_client.py	0.0% <0.0%> (-100.0%)	⬇️
...v2/src/simcore_service_director_v2/cli/__init__.py	0.0% <0.0%> (-100.0%)	⬇️
...service_webserver/projects/_project_models_rest.py	0.0% <0.0%> (-100.0%)	⬇️
...service_director_v2/cli/_close_and_save_service.py	0.0% <0.0%> (-100.0%)	⬇️
...es/web/server/src/simcore_service_webserver/log.py	0.0% <0.0%> (-94.8%)	⬇️
...erver/src/simcore_service_webserver/rest_models.py	0.0% <0.0%> (-92.7%)	⬇️
...or-v2/src/simcore_service_director_v2/cli/_core.py	0.0% <0.0%> (-87.5%)	⬇️
...simcore_service_webserver/director/director_api.py	0.0% <0.0%> (-76.0%)	⬇️
...ce_director_v2/modules/db/repositories/clusters.py	23.0% <0.0%> (-71.7%)	⬇️
...e_service_webserver/login/handlers_confirmation.py	27.2% <0.0%> (-69.1%)	⬇️
... and 551 more

[pcrespov]

Can you also please add a more descriptive title for the PR

[sanderegg]

pair reviewd

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.2% Duplication