Add option to override Celery worker images#63514
Add option to override Celery worker images#63514sg-c0de wants to merge 1 commit intoapache:mainfrom
Conversation
|
Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors' Guide (https://github.com/apache/airflow/blob/main/contributing-docs/README.rst)
|
jscheffl
left a comment
There was a problem hiding this comment.
Thanks, that option makes sense to me. Can you also please add some tests?
|
@sg-c0de This PR has a few issues that need to be addressed before it can be reviewed — please see our Pull Request quality criteria. Issues found:
What to do next:
There is no rush — take your time and work at your own pace. We appreciate your contribution and are happy to wait for updates. If you have questions, feel free to ask on the Airflow Slack. |
Summary
Since we have the option to configure multiple Celery worker sets, it is useful to be able to use different images for specific sets (for example, to include CUDA drivers). This adds the option to override the image for all Celery workers globally or on a per-set basis using
Values.workers.celery.imageandValues.workers.celery.sets[].image.Changes
chart/templates/_helpers.yaml: Add
airflow_worker_image,airflow_worker_image_pull_policy, andairflow_worker_image_for_migrationshelpers, and patch thefullOverwritehelper to support image overrides.chart/templates/workers/worker-deployment.yaml: Update
imageandimagePullPolicyreferences to use the new helper templates.chart/values.yaml: Add new values to support the image overrides.