Pre-release master -> staging_switzer1 (Oct. 2022)

[pcrespov]

In preparation for pre-release Staging switzer1, let's review these commits together and

1. turn them into changelog message
2. define how to test them (note: see how to test section in associated PR)

---

Changelog follow structure defined in https://keepachangelog.com/en/1.0.0/

Added

• ✨ Dy-services: adds cli command to monitor study status (#3401)
• ✨ S4L: Adds 'start s4l*' buttons in dashboard (#3399)
• ✨ Service deprecation FE: Deprecated services are not started (#3386)
• ✨ Service deprecation: define strict limits legacy dynamic services (#3347)
• ✨ API: api keys with expiration (#3381)
• ✨ TIP: FE support email on expired message, Enter key linked to validate button and blink Output in App Mode (#3400)
• ✨ TIP: trial invitations (#3380)(#3352)
• ✨ TIP: trial invitations UI: (#3349) (#3397) (#3398)
• ✨ Copy: Duplicate&ToTemplate in frontend (#3370)
• ✨ Copy: Study from Template, log messages and progress (#3354)
• ✨ Copy: Do not auto-cancel study copy if the user disconnects (#3345)
• ✨ UI App Mode editor improvements (#3427)
• ✨ UI App Mode: Allow uploading file to File Picker (#3383)
• ✨ UI App mode: get ready for larger studies (#3359)
• ✨ Autoscaling: new service (#3346)(#3364)

Changed

• ✨ Sharing: Show projects with inaccessible services (⚠️ devops) (#3368)
• ✨ OOIL: updates o2sparc-integration-libray (#3425)(#3428)(#3433)
• ✨ TIP: safer product setup (#3395)
• ♻️ TIP: Expired confirmation tokens are logged and INVITATION tokens do not expire (#3440)
• ✨ Service deprecation: Prevent deprecated services to start (#3350)
• ♻️ Dy-services: Visualise body in stacktraces from S3 HTTP errors (#3409)
• ♻️ Dy-services: wait for resources before failing (#3388)
• ♻️ Dy-services: Adjusting timeout for docker_node_id recovery (#3382)
• ✨Dy-services: Retry upon AWS 500s (#3353), error when uploading file (#3403)
• ✨ UI/UX: Show study name when deleting (#3344)

Deprecated

Removed

Fixed

• 🐛 Dy-services: Avoids service shutdown due to swarm networking errors (#3394)
• 🐛 Dy-services: dy-volume removal service fixes and improvements (#3431)
• 🐛 Dy-services: fixes auth pop-up when container dies (🚨 ⚠️ devops) (#3384)
• 🐛 Dy-services: fix issue with archiving utils (#3320)
• 🐛 API: Fixes Retry-After header in global_rate_limit_route (#3379)
• 🐛 TIP: Fix/confirmation codes (#3356)
  🐛 UI: fix unkown label in tree (#3355)

Security / Maintenance

• ♻️ lgtm+CodeQL fixes for Q&A (#3363)
• ♻️ tests: Reduce test flakyness (#3332)
• ♻️ tests: Minor changes on testing on overall repo (#3348)
• ♻️ e2e: add X-Simcore-User-Agent to requests, s4l-e2e and p2e (#3392) (#3410) (#3411) (#3408) (#3404) (#3415) (#3432)
• 🐛 e2e: fixes CORS issue in puppeteer while testing with X-Simcore-User-Agent header (#3417)
• 🔨 tools: Make project migration script more versatile (#3178)
• 🔨 CI: auto-cancel job with same name (#3434)
• 🔨 CI: Fixes gh-actions scripts for release workflow (#3414)
• 🔨 CI: Fix file path to code climate coverage
• 🔨 CI build times: leverage docker buildx github registry caching and transmit builds to int/sys tests (#3369)
• 🔨 CI: Improve CI reliability + add healthcheck in migration service (#3380)
• 🔨 CI: Improve production build times (#3367)
• 🔨 CI: Fix typos in hotfix staging (#3358)
• ⬆️ CI: set buildx to 0.8.2 (#3333)
• ⬆️🔒️ vulnerability upgrade for ujson, upgrade fastapi+starlette (#3112)
• ⬆️🔒️ fixes security vulnerabilities in mako (#3376)
• ⬆️ Bump actions/setup-node from 3.4.1 to 3.5.0 (#3405)
• ⬆️ Bump codecov/codecov-action from 3.1.0 to 3.1.1 (#3389)
• ⬆️ Bump actions/checkout from 2 to 3 (#3351)

Legend

✨ New feature
🐛 Fixes bugs
♻️ Refactors code
⬆️ Upgrades dependencies
🔒️ Fixes security issues
🔨 Adds or updates development scripts or CI.
🚨 important change. REQUIRES app-level testing before releasing to production
📌 can be cherry-picked to production

[mrnicegyu11]

• DevOps changes look good to me, except for:
• @Surfict 🐛 First attempt to get rid of authentication pop-up when container dies (🚨 ⚠️ devops) osparc-simcore#3384 these devops changes relate to the maintenance page, I think you touched it yesterday on master. Please comment w.r.t. this 🙂
• The autoscaling service will not be started and run on staging, correct? It is only a part of the codebase, not a running service? @pcrespov

[Surfict]

• @Surfict 🐛 First attempt to get rid of authentication pop-up when container dies (🚨 ⚠️ devops) #3384 these devops changes relate to the maintenance page, I think you touched it yesterday on master. Please comment w.r.t. this 🙂

I tested the maintenance page yesterday with theses change in master and it works as it should. We can go forward

[pcrespov]

• The autoscaling service will not be started and run on staging, correct? It is only a part of the codebase, not a running service? @pcrespov

@mrnicegyu11 no, it will actually start but will not do anything highly memory/cpu consuming (check in master deploy to see details)

[mrnicegyu11]

This looks good to me, please give DevOps the final ok for this release to staging @pcrespov . Probably no more PRs should be merged into master from now on. Will @elisabettai continue preparing changelogs or is this a task for devops?

[pcrespov]

@mrnicegyu11

1. we do not need to block master since we are not constraint to release from HEAD
2. if you look at the top of this issue, these are the changelogs that will go in staging. As we have reported daily, we have been monitoring, repharsing and classifying these since some weeks.
3. I will give some precise instructions for the release by the end of the day. I propose @Surfict and me to do that on Monday.

[mrnicegyu11]

Hi Pedro, thanks a lot for the answers. I truly forgot we are able to specify the gitsha from which to release, so no blocks on master.
If I may, I would like to clarify one of my questions: Will the final preparation of the changelog (and posting it in the release notes) be something DevOps do as in past, or has this responsibility been fully switched to the application scientists (i.e. @elisabettai ?) Or is this not decided yet, which is also fine? Sorry about these stupid questions, just wondering...

[pcrespov]

Hi Pedro, thanks a lot for the answers. I truly forgot we are able to specify the gitsha from which to release, so no blocks on master. If I may, I would like to clarify one of my questions: Will the final preparation of the changelog (and posting it in the release notes) be something DevOps do as in past, or has this responsibility been fully switched to the application scientists (i.e. @elisabettai ?) Or is this not decided yet, which is also fine? Sorry about these stupid questions, just wondering...

This is the second time we try this. I would suggest to finish with this iteration and then evaluate the outcome so we can take a more definitive decision.

[mrnicegyu11]

re: devops changes for ITISFoundation/osparc-simcore#3384 : https://github.com/ITISFoundation/osparc-simcore/pull/3384 is done on staging dalco and aws staging (and master)

[pcrespov]

• @Surfict and @GitHK will release on Oct.17, Monday around 15h
• Release master 576c686ecd5f0921480af930e530de8da9379d5a to Staging_????

[Surfict]

Release to staging just completed. We had to update the products table in both staging aws and staging dalco as an entry in the manual_url was not with https but with http, which prevented the webserver to start correctly (symptoms : impossible to log in)

Release notes: https://github.com/ITISFoundation/osparc-simcore/releases/tag/staging_switzer1