Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Pre-release master -> staging_switzer_4 #759

Closed
6 tasks done
pcrespov opened this issue Nov 1, 2022 · 2 comments
Closed
6 tasks done

Pre-release master -> staging_switzer_4 #759

pcrespov opened this issue Nov 1, 2022 · 2 comments
Assignees
@pcrespov
Labels
release Preparation for pre-release/release
Milestone
Katherine Switzer

Comments

@pcrespov
Copy link
Member

pcrespov commented Nov 1, 2022 •
edited

In preparation for pre-release. Here an initial (incomplete) list of tasks to prepare before pre-releasing:

  • Draft changelog from commits list (see docs/releasing-workflow-instructions.md)
  • Check important changes 🚨
  • Devops check (⚠️ devops)
  • e2e testing check
  • Pre-release summary
  • Pre-release assessment

Check important changes 🚨

Motivation: end of switzer sprint. Highlight of important changes are

  • new s3 layout
  • new public-api and I/O ports exposed in APIs
  • new s4l-light product
  • upgraded postgres database

See specific cases marked with 🚨 in changelog below

Devops check (⚠️ devops)

  • setup s4l-light product, enable s4l-light url
  • ensure TLS option for s4l-light product on ops traefik
  • Machine maintenance: Switch on kernel flag for cgroup swap memory limits, reboot
  • New postgres PANIC https://git.speag.com/oSparc/osparc-infra/-/issues/12. 🚨
  • Changes in rabbit requires all dy-sidecars down (#3502) 🚨

e2e testing check

  • Fri Nov.4 : still unstable. Fixes to s4l-light failure. TIP failing. Parallel failing: @GitHK will cleanup sidecars. @mrnicegyu11 will complete postgres upgrade.
  • Mon Nov.7: skipped stand-up
  • Tue Nov.8: off due to problems with CI
  • Wed Nov.9: off due to demos
  • Thr Nov.10: ...
  • Fri Nov.11: ... (release?)
  • Mon Nov.14: e2e unstable. Needs action
  • Tue Nov.15: parallel tests only
  • Wed Nov.16: Agent issue fixed and still checking on parallel tests (related project lock when closing)

Commits (in order)

  • 🔨📝 Maintenance/fix registry tests (#3553)
  • Revert "Revise docker networks (#3543)" (#3556)
  • Revise docker networks (#3543)
  • ⚗️🐛 Add a test for reproducing potential 400 issue with upload to AWS (#3538)
  • maintenance template for maintenance issues [skip ci]
  • 🐛 CI: reduce test flakyness (#3542)
  • 🐛 agent service followup fixes (#3513)
  • ♻️🔨 Is3517/refactor service io and diagnostics tool concept (#3537)
  • 🐛 Fix s4l-lite test (#3539)
  • ✨ S4L Lite: 11.08 meeting feedback (⚠️ devops) (✨ S4L Lite: 11.08 meeting feedback (⚠️ devops) osparc-simcore#3534)
  • 🐛 adds tests for S3TransferError; refactoring flaky CI test; better logging for long running task errors (#3525)
  • ✨ Is355/optimizer projects ports (#3504)
  • ♻️ Fixes flaky test_update_profile and cleanup tests (#3528)
  • ✅ Fixing webserver 02 unit test (#3532)
  • 🐛 Fixes CI test issues introduced by faulty #3524 (#3527)
  • 🐛 CI integration tests not run when they should (#3529)
  • ⬆️ Upgrade postgres to 14.5 alpine⚠️ devops (⬆️ Upgrade postgres to 14.5 alpine⚠️ devops osparc-simcore#3500) 🚨
  • ⬆️ Maintenance/upgrades tests+tooling requirements (#3524)
  • ✨ Sort files and NodeTreeItem menu's bgColor (#3523)
  • ⬆️ 🔨 Workaround to avoid test failures due to pytest-sugar (#3514)
  • ✨ Deprecated and Retired (#3512)
  • ✨♻️ Differentiate between TLS and STARTTLS in web-mailserver (⚠️ devops) (✨♻️ Differentiate between TLS and STARTTLS in web-mailserver (⚠️ devops)  osparc-simcore#2965)
  • 🐛 Bugfix: s4l-light maximize button (#3511)
  • ✨adding agent service with dyv volumes removal (#3465)
  • 🐛 Project copy failing when pennsieve token is active (#3509) [ 📌 v1.38.3 ]
  • ✨ Ensure memory swap for computational services is same as memory (⚠️ devops) (✨ Ensure memory swap for computational services is same as memory (⚠️ devops) osparc-simcore#3510) 🚨
  • 🐛✨ Frontend: S4L Light product II (#3508)
  • ✨ Frontend: S4L Light product (#3503)
  • ✨ Allow Starting dynamic services when idle OR failed (#3501) 🚨
  • 🐛 Fix/sanitize old data for usergroups.thumbnail (#3498) 🚨
  • ⬆️ 🔒️ Maintenance/week-44: upgrades pytest, aiohttp, jupyter-core and pillow (#3497)
  • ✨ Add instructions to App Mode steps (#3491)
  • ⬆️ Use latest rabbit MQ service (#3496)
  • ✨ Allow selective start/stop of dynamic services (⚠️ devops) (✨ Allow selective start/stop of dynamic services (⚠️ devops) osparc-simcore#3449) 🚨
  • 🔨♻️ ⬆️ Adds CI python-linting job in python 3.11 (#3489)
  • 🐛✨ e2e: open outputs folder in some cases (#3495)
  • ⬆️ upgrade aio-pika to latest version 8.2.4 (#3492)
  • ✨ Is686/api port schemas public api: api-server 0.4.1 (#3485) 🚨
  • ✨ Is3418/validation with ooil test my/osparc/service (#3479)
  • ✨ Is686/list_service_ports in catalog service API: catalog 0.4.0 (#3484)
  • 🐛✨ Ensure adding/deleting node is thread safe (#3490)
  • 🐛 Storage: crash when not a file in the project (#3483) [📌 v1.38.1]
  • 🔨 CI ensure CI using the correct ENVs in master (#3482)
  • ✨ FE: TIP Follow up I (#3466)
  • 🐛 CI ensure built images are used for testing (#3481)
  • 🐛 Ensure uploaded outputs always have a unique S3 object name (#3462) 🚨
  • CI maintenance: changes detection need checkout on master branch
  • skip changes job if on a push
  • ♻️ CI: only run tests jobs on path changes in pull requests (#3429)
  • 🐛 fix/low-CPU-load healthcheck for migration service (#3477) [📌 staging_switzer_3]
  • 🔨 Maintenance/typecheck steps in CI (#3475)
  • 🔨 Fix/CI build&deploy jobs with integration-library image (#3474) [📌 staging_switzer_3]
  • 🐛 Bugfix#2: datcore-adapter stops calling into pennsieve after too many requests are done? (#3473) [📌 staging_switzer_3]
  • ✨ ooil executable in a docker image (#3458)
  • 🐛Bugfix/644/listing makes pennsieve client fail (#3464)
  • removing dynamic_sidecar_network from dy-sidecar (#3467)
  • ✨ Tooltip on node links (#3441)
  • ⬆️ Update datcore-adapter requirements (#3463)

Draft Changelog

## Added / Changed / Removed
- ✨ O2IL:  ooil executable in a docker image (#3458)
- ✨ O2IL: Is3418/validation with ``ooil test my/osparc/service`` (#3479)
- ✨ I/O: Is686/api port schemas public api: api-server 0.4.1 (#3485) 🚨
- ✨ I/O: Is686/list_service_ports in catalog service API: catalog 0.4.0 (#3484)
- ♻️  I/O: Is3517/refactor service io and 🔨diagnostics tool concept (#3537)
- ✨ Optimizer: Is355/optimizer projects ports (#3504)
- ✨ Services deprecation: Deprecated and Retired (#3512)
- ✨ Computation: Ensure memory swap for computational services is same as memory (⚠️ devops) (#3510) 🚨
- ✨♻️ Email: Differentiate between TLS and STARTTLS in web-mailserver  (⚠️ devops)  (#2965)
- ✨ UI App: Tooltip on node links (#3441)
- ✨ UI: Sort files and NodeTreeItem menu's bgColor (#3523)
- ✨ UI S4L Lite: 11.08 meeting feedback (⚠️ devops) (#3534)
- ✨ UI:S4L-lite product (#3503)(#3508)(#3511)
- ✨ UI TIP: TIP Follow up I (#3466)
- ✨ UI App: Add instructions to App Mode steps (#3491)
- ✨ Dy-Services: Adding agent service with dyv volumes removal (#3465)(#3513)
- ✨ Dy-Services: Allow Starting dynamic services when idle OR failed (#3501) 🚨
- ✨ Dy-Services: Allow selective start/stop of dynamic services (⚠️ devops) (#3449) 🚨
- 🗑️ Dy-Services: removing dynamic_sidecar_network from dy-sidecar (#3467)

## Fixed
- 🐛 DatCore: datcore-adapter stops calling into pennsieve after too many requests are done? (#3473) [📌 ``staging_switzer_3``]
- 🐛 DataCore listing makes pennsieve client fail (#3464)
- 🐛 fix/low-CPU-load healthcheck for migration service (#3477) [📌 ``staging_switzer_3``]
- 🐛 Study: Ensure adding/deleting node is thread safe (#3490)
- 🐛 Sharing: Fix/sanitize old data for usergroups.thumbnail (#3498) 🚨
- 🐛 Storage: Ensure uploaded outputs always have a unique S3 object name (#3462) 🚨
- 🐛 Storage: crash when not a file in the project (#3483)  [📌 ``v1.38.1``]
- ⚗️🐛 Dy-services: Add a test for reproducing potential 400 issue with upload to AWS (#3538)
- 🐛 Dy-services: Fix s4l-lite test (#3539)
- 🐛 Dy-services: adds tests for S3TransferError; refactoring flaky CI test; better logging for long running task errors (#3525)
- 🐛 Copy: Project copy failing when pennsieve token is active (#3509)  [ 📌 ``v1.38.3`` ]

## Security / Maintenance

- 🔨CI: Ensure CI uses the correct ENVs in master (#3482)
- ♻️ CI: Only run tests jobs on path changes in pull requests (#3429)
- 🔨CI: Maintenance/typecheck steps in CI (#3475)
- 🔨CI: Fix/CI build&deploy jobs with integration-library image (#3474) [📌 ``staging_switzer_3``]
- 🔨CI Ensure built images are used for testing (#3481)
- 🔨CI: Fixes CI test issues introduced by faulty #3524 (#3527)
- 🔨CI: integration tests not run when they should (#3529)
- 🔨CI: Adds CI ``python-linting`` job in python 3.11 (#3489)
- 🔨test: Maintenance/fix registry tests (#3553)
- 🔨test: Reduce test flakyness (#3542)
- ♻️ test: Fixes flaky test_update_profile and cleanup tests (#3528)
- ♻️ test: Fixing webserver 02 unit test (#3532)
- 🔨 e2e: open outputs folder in some cases (#3495)
- ♻️ Revise docker networks (#3543) and reverted (#3556)
- 🔨 Github template for maintenance issues
- ⬆️ Update datcore-adapter requirements (#3463)
- ⬆️ Upgrade aio-pika to latest version 8.2.4 (#3492)
- ⬆️ Upgrade postgres to 14.5 alpine⚠️ devops (#3500) 🚨
- ⬆️ Upgrades tests+tooling requirements (#3524)
- ⬆️ 🔨 Workaround to avoid test failures due to pytest-sugar (#3514)
- ⬆️ 🔒️ Upgrades pytest, aiohttp, jupyter-core and pillow (#3497)
- ⬆️ Use latest rabbit MQ service (#3496)

**Legend**

- ✨ New feature
- 🐛 Fixes bugs
- ♻️ Refactors code
- ⬆️ Upgrades dependencies
- 🔒️ Fixes security issues
- 🔨 Adds or updates development scripts or CI.
- 🚨 Important change. REQUIRES target testing before releasing to production. Steps to test appended as ``[TODO:  ... ]``
- 📌 can be cherry-picked to production or staging
- ✅ Target/User tests done

Pre-release summary

After this release, we need to 🚑️ hotfix right away staging by cherry-picking some commits for the new agent (@pcrespov ) CANCELLED

Pre-release assessment
@pcrespov pcrespov self-assigned this Nov 1, 2022
@pcrespov pcrespov changed the title Pre-release master -> staging_switzer_3 Pre-release master -> staging_switzer_4 Nov 1, 2022
@pcrespov pcrespov mentioned this issue Nov 3, 2022
Closed
@pcrespov pcrespov added this to the Katherine Switzer milestone Nov 4, 2022
@pcrespov pcrespov added the release Preparation for pre-release/release label Nov 17, 2022
@Surfict
Copy link
Collaborator

Surfict commented Nov 18, 2022

The release went fine and was done 17/11/2022 around 13:00 and went ok. From Ops, we had to do the chance concerning the new product after the release.

We had some trouble with the migration of PGSQL, Dustin will add more informations.

@mrnicegyu11
Copy link
Member

The migration simcore service and all other pg clients have to be stopped / paused when dropping the old simcoredb and importing a new db .

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pcrespov pcrespov

Labels
release Preparation for pre-release/release
Projects
None yet
Milestone
Katherine Switzer
Development

No branches or pull requests
3 participants
@Surfict @mrnicegyu11 @pcrespov