🚀 Release v1.52.0

[matusdrobuliak66]

Release version

1.52.0

Commit SHA

49ee3cdc8bacd6d557f0aff68b1da2c0b6eb947a

Previous pre-release

https://github.com/ITISFoundation/osparc-simcore/releases/tag/staging_JellyBeans2

Did the commit CI suceeded?

• The commit CI succeeded.

Motivation

• Regular release

Changes

Staging Mithril5

Release Issue: #4054

• ♻️ Frontend: Notifications cleanup #4004 by @GitHK
• ✨ Gateway-server: add ENV variable to change how many CPUs/RAM is advertised by each sidecar #4019 by @sanderegg
• ✨Add simcore_user_agent and product_name to user service labels #3990 by @GitHK
• ♻️Dask gateway: add missing variables + change deprecated calls #4022 by @sanderegg
• 🎨Frontend: Update services from Dashboard UX #4011 by @odeimaiz
• 🐛 Fix: Download Study Data from Firefox #4026 by @odeimaiz
• 🐛 Sometime docker timestamps are not parsed correctly #4003 by @sanderegg
• ✨🗃️ Implements services listing for nih-portal (1/2) (⚠️ devops) #4024 by @pcrespov
• 🐛Dask: Handle when gateway is offline #4033 by @sanderegg
• ✨ Is683/view link can open a project with a single service (2/2) #4038 by @pcrespov
• 🎨 Frontend: Update services UX II and change Boot mode in the UI #4031 by @odeimaiz
• 🐛 filetypes in webserver API are case insensitive #4041 by @pcrespov
• 🔨 Maintenance/upgrade python version to 3.10 #4010 by @matusdrobuliak66
• 🐛 Frontend: minors 30.03 RM #4045 by @odeimaiz
• 🐛🗃️ Dask gateway: clusters cannot be deleted after usage #4036 by @sanderegg
• 🐛 fixes broken node progress [📌 v1.51.2]  #4049 by @GitHK
• ♻️ e2e: improve error reporting #4048 by @sanderegg

Staging Mithril6

Release Issue: #4058

• ♻️ Maintenance/small cleaning after python upgrade #4047 by @matusdrobuliak66
• ✨ Frontend: Shared with filter in Dashboard #4050 by @odeimaiz
• 🐛 updates last modified timestamp when the project is opened #4029 by @matusdrobuliak66
• ♻️ Dask: maintenance, refactor, upgrade #4051 by @sanderegg
• ♻️ Api server: improve job creation speed #4053 by @sanderegg
• 🐛 Dask: fix issue with disappearing workers #4057 by @sanderegg
• 🐛 Fix GC taking too much time to stop services #4040 by @GitHK
• ♻️ Refactors service listing queries and 🗃️ drops service_latest cache table #4056 by @pcrespov

Staging JellyBeans1

Release Issue: #4115

• 🐛 Dask gateway: Fix raised exception when service or task is not found #4060 by @sanderegg
• ♻️ Maintenance/removing-old-json-schemas 🚨 #3966 by @matusdrobuliak66
• 🎨 Frontend: more Icons in Dashboard's toolbar #4061 by @odeimaiz
• 🎨 Frontend: Improvements to tagging functionality #4052 by @odeimaiz
• 🔨 Maintenance: missing updates in schemas #4066 by @pcrespov
• 🐛 coroutine not awaited in director-v2 #4070 by @GitHK
• ♻️⬆️ Maintenance/removing pydantic constraints #4065 by @matusdrobuliak66
• 🐛 fix e2e: ti-plan test #4078 by @odeimaiz
• 📝 enhancing release workflow + adding hotfix template #4083 by @matusdrobuliak66
• ⬆️ Maintenance: database libraries and 📝 pep-0257 for doc #4062 by @pcrespov
• ♻️ director-v2 uses one dy-sidecar API client per sidecar #4085 by @GitHK
• 🐛 added missing header to legacy dynamic service containers #4088 by @GitHK
• ✨ catalog service: New CLI with settings command and ♻️ cleanup meta (⚠️ devops) #4093 by @pcrespov
• 🐛 Frontend: Fix Service's More options view #4096 by @odeimaiz
• ✨Frontend: Landing page PoC #4077 by @odeimaiz
• ♻️ Add Devops notes to PR template #4025 by @mrnicegyu11
• 🗑️ Removes deprecated codecov #4098 by @pcrespov
• ♻️🔨 catalog: fixes mypy issues and add type-check step in CI #4095 by @pcrespov

Staging JellyBeans2

Release Issue: #4135

• ♻️🔨 api-server: fixes mypy issues and add type-check step in CI #4094 by @pcrespov
• ✨Maintenance: enable mypy in storage #4086 by @sanderegg
• ♻️ Ensure log level is passed to the frontend #4090 by @sanderegg
• ✨ Metrics: add simcore_user_agent in service_started/service_stopped metric #4092 by @sanderegg
• 🐛 Fix s4l-lite e2e test #4101 by @odeimaiz
• 🗑️ Removes coveralls for test coverage #4099 by @pcrespov
• ♻️ webserver: fixes mypy issues in version_control plugin  #4102 by @pcrespov
• ✨Webserver: Allow to disable auto-start of dynamic services #4103 by @sanderegg
• ✨ Frontend: Panddy PoC #4091 by @odeimaiz
• ⬆️Maintenance: update mypy to 1.2.0 #4108 by @sanderegg
• ✨ Frontend: Disable Services Auto Start #4109 by @odeimaiz
• ♻️ webserver: fixes mypy issues in meta_modeling plugin  #4113 by @pcrespov
• 🔨♻️ Fixes mypy issue after upgrade #4118 by @pcrespov
• 🔒️Maintenance: Fix Regex DOSes #4121 by @sanderegg
• ♻️ introducing structured logging (part 1) (⚠️ devops) #4114 by @matusdrobuliak66
• 🎨 Frontend: Landing Page PoC II #4119 by @odeimaiz
• Bump codecov/codecov-action from 3.1.1 to 3.1.2 #4130 by @dependabot
• ♻️ SonarCloud issues in director-v2 and dynamic-sidecar #4125 by @GitHK
• ♻️ SonarCloud webserver.exporter #4126 by @GitHK
• ♻️Clusters: make non dev feature, disabled by default (⚠️) #4129 by @sanderegg

Devops check 👷

• Add changes (if any) and release tags in https://github.com/ITISFoundation/osparc-ops-environments/tags
• Add changes (if any) and release tags in osparc-ops-deployment-configuration (osparc.io & speag.com & tip)
• Ops changes
• Rebooting each machine
• Testing the maintenance page

Tests assessment: e2e testing check 🧪

• aws staging - good
• dalco staging - good (tiplan failing)

Test assessment: targeted-testing 🔍️

No response

Test assessment: user-testing 🧐

No response

Summary 📝

• Prepare release link

make release-prod version=1.52.0 git_sha=49ee3cdc8bacd6d557f0aff68b1da2c0b6eb947a

• Draft release changelog
• Announce maintenance ( ** ANNOUNCE AT LEAST 24 HOURS BEFORE ** )
• redis {"start": "2023-04-24T06:00:00.000Z", "end": "2023-04-24T10:00:00.000Z", "reason": "Release <v1.52.0>"}
  • aws
  • dalco
  • tip
• status page (https://manage.statuspage.io/)
  • osparc
  • s4l
• mattermost channels
  • maintenance
  • power users

Releasing 🚀

• Maintenance page up.
• Release by publishing draft
• Check release CI
• Check deployed
  • aws deploy
  • dalco deploy
  • tip deploy
• Delete announcement
• Check e2e runs
• Announce

:tada: https://github.com/ITISFoundation/osparc-simcore/releases/tag/v1.52.0

[matusdrobuliak66]

@Surfict @mrnicegyu11 I would like to announce the maintenance for users. Please can we decide when and for how long the maintenance page should be up? For example, can we do it Monday morning 8 am? Thanks.

[mrnicegyu11]

Timing is fine for me personally, this is pending approval from @Surfict .
We will reboot each node individually one at-a-time, which will likely take 2-4 hours across all 3 prod deployments. Potentially, we will do some tests with the maintenance page if @Surfict managed to create their setup.

[pcrespov]

@matusdrobuliak66 would it make sense to release first only the work in Mithril and then the rest? Not a must but this way we keep the rule that we have at least one release per sprint .

[matusdrobuliak66]

Timing is fine for me personally, this is pending approval from @Surfict . We will reboot each node individually one at-a-time, which will likely take 2-4 hours across all 3 prod deployments. Potentially, we will do some tests with the maintenance page if @Surfict managed to create their setup.

Ok after discussion we agreed to do it exceptionally at Monday morning 8 am as we will announce the maintenance for 4 hours.

[mrnicegyu11]

In checked the devops warnings, all good.

Actionable items:

• Set env var STUDIES_DEFAULT_SERVICE_THUMBNAIL="" (overwrite default with emptystring as agreed with @odeimaiz )

[pcrespov]

Actionable items:

• Set env var STUDIES_DEFAULT_SERVICE_THUMBNAIL="" (overwrite default with emptystring as agreed with @odeimaiz )

@mrnicegyu11 if you look carefully at the jsonscheam of STUDIES_DEFAULT_SERVICE_THUMBNAIL you will notice that if you setup that to an empty string, the validation will fail! Either write a proper URL or leave the code default. :-)

Typically the assets should go in https://github.com/ITISFoundation/osparc-assets/tree/main/assets

I see e.g. that @elisabettai added a default thumbnail there

STUDIES_DEFAULT_SERVICE_THUMBNAIL=https://raw.githubusercontent.com/ITISFoundation/osparc-assets/60701745000f61261705f8f54c390715797f69f8/assets/default-thumbnail-cookiecutter-osparc-service.png

[mrnicegyu11]

We need to set

AUTOSCALING_EC2_INSTANCES_CUSTOM_BOOT_SCRIPTS='["mkfs --type xfs -f -n ftype=1 /dev/nvme1n1","mkdir --parents /mnt/docker","mount /dev/nvme1n1 /mnt/docker","jq '\''.\"data-root\"=\"/mnt/docker/data\",.\"exec-root\"=\"/mnt/docker/exec\"'\'' /etc/docker/daemon.json  > /tmp/daemon.json","mv /tmp/daemon.json /etc/docker/daemon.json","jq '\''. += { \"data-root\": \"/docker\", \"log-driver\": \"gelf\", \"log-opts\": {\"gelf-address\": \"udp://0.0.0.0:12201\"}, \"default-address-pools\" : [{\"base\" : \"172.17.0.0/12\",\"size\" : 20},{\"base\" : \"192.168.0.0/16\",\"size\" : 24}] }'\'' /etc/docker/daemon.json > tmp.json && mv tmp.json /etc/docker/daemon.json","systemctl restart docker"]'

To make logging in autoscaled nodes work

[GitHK]

Logout seemed to work good on Dalco staging and prod, all services were closed.
On AWS production 4 sidecars were still running normally. They shut down via the director-v2's cli without any issue. Prior to the maintenance there were 8-10 sidecars running normally.

[Surfict]

• Maintenance page tested with success in aws production
• Portainer cron jobs deployed on all stacks with ansible