v0.22.0
Armada v0.22.0
For more info, head over to the docs page at https://armadaproject.io
NOTICE
This release completes the move of job metadata from the jobs -> job_metadata table in the scheduler database.
This is not a breaking change but as part of the upgrade a forceful migration of data will happen moving groups and submit_message data from the jobs -> jobs_metadata table.
This will block all writes to the scheduler db during the migration, so if you have a large dataset this could cause a production outage during this migration.
- For timings on approximate outage time is shown in this PR #4984
To avoid outage, it is recommended to upgrade to a version between v0.21.1 and v0.21.7 and follow this migration:
- Update scheduler configuration to set jobMetadataMigrationPhase to dualWrite
- Update scheduler ingester configuration to set jobMetadataMigrationPhase to cutover
- Wait for all rows in your database to be updated
- This will only happen due to natural job churn, so you'll need to wait for all jobs in the system prior to the config updates to finish
- If waiting on all jobs to finish is impractical, the outage time is determined by the number of rows left to migrate and should be relatively quick even for relatively large datasets (see #4984 for timings)
- Upgrade to v0.22.0+
More detail:
#4906
https://github.com/armadaproject/armada/releases#release-v0.21.1
WARNING
jobMetadataMigrationPhase should not be set to cutover in the scheduler ingester before the jobMetadataMigrationPhase is set to dualWrite in the scheduler.
- Doing so will mean that the metadata will be written to the job_metadata table but the scheduler will only be looking at the jobs table
jobMetadataMigrationPhase should not be set to cutover in the scheduler before all jobs in the database have an entry in the job_metadata table
- Doing so will mean it will fail to load the podspec to send to the executor and therefore be unable to run those jobs
Armada CLI
armadactl controls the Armada batch job queueing system and is used for interacting with the system.
The CLI can be downloaded for a specific OS & Architecture from the Assets section below.
NOTE: The OSX binary is packaged as an universal binary and should work on both Intel and Apple Silicon based Macs.
Docker images
Armada Bundle
docker pull gresearch/armada:0.22.0docker pull gresearch/armada:latest
Armada Lookout Bundle
docker pull gresearch/armada-lookout-bundle:0.22.0docker pull gresearch/armada-lookout-bundle:latest
Armada Full Bundle
docker pull gresearch/armada-full-bundle:0.22.0docker pull gresearch/armada-full-bundle:latest
Armada Server
docker pull gresearch/armada-server:0.22.0docker pull gresearch/armada-server:latest
Armada Executor
docker pull gresearch/armada-executor:0.22.0docker pull gresearch/armada-executor:latest
Armada Lookout
docker pull gresearch/armada-lookout:0.22.0docker pull gresearch/armada-lookout:latest
Armada Lookout Ingester
docker pull gresearch/armada-lookout-ingester:0.22.0docker pull gresearch/armada-lookout-ingester:latest
Armada Event Ingester
docker pull gresearch/armada-event-ingester:0.22.0docker pull gresearch/armada-event-ingester:latest
Armada Scheduler
docker pull gresearch/armada-scheduler:0.22.0docker pull gresearch/armada-scheduler:latest
Armada Scheduler Ingester
docker pull gresearch/armada-scheduler-ingester:0.22.0docker pull gresearch/armada-scheduler-ingester:latest
Armada Binoculars
docker pull gresearch/armada-binoculars:0.22.0docker pull gresearch/armada-binoculars:latest
armadactl
docker pull gresearch/armadactl:0.22.0docker pull gresearch/armadactl:latest
Changelog
Other work
- 7e5aec7: Drop legacy submit_message & groups columns from jobs (#4984) (@tgucks)
- 9eff451: Fix RangeError when null dates passed to DateTimePicker (#4997) (@mauriceyap)
- f6ab3a9: Fix cross-partition duplicate job rows in Lookout ingester (experimental hot/cold) (#4996) (@mauriceyap)
- d0a0344: Hot cold local dev (#4899) (@dslear)
Full Changelog: v0.21.7...v0.22.0
