Add SSL_MUTUAL_TLS config option to Celery provider for one-way TLS support#64767
Merged
potiuk merged 1 commit intoapache:mainfrom Apr 13, 2026
Merged
Conversation
dandanseo123
requested review from
dheerajturaga and
hussein-awala
as code owners
potiuk
added
the
ready for maintainer review
dandanseo123
force-pushed
the
fix/celery-ssl-skip-empty-key-cert
branch
3 times, most recently
from
9608390 to
6508b71
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds a new Celery TLS configuration flag to support one-way TLS (server verification only) while preserving the existing mutual TLS default behavior, plus documentation and tests.
Changes:
- Introduces
[celery] SSL_MUTUAL_TLS(defaultTrue) to toggle mutual vs one-way TLS. - Updates Celery broker SSL dict generation for AMQP/Redis based on
SSL_MUTUAL_TLS, with a clearer error for missing client cert/key. - Adds unit tests and updates provider config documentation for the new option and requirements.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| providers/celery/src/airflow/providers/celery/executors/default_celery.py | Build broker_use_ssl conditionally based on SSL_MUTUAL_TLS, add clearer validation/warnings. |
| providers/celery/tests/unit/celery/executors/test_celery_executor.py | Adds unit tests for Redis mutual TLS, AMQP missing key/cert error, and AMQP/Redis one-way TLS behavior. |
| providers/celery/src/airflow/providers/celery/get_provider_info.py | Documents ssl_mutual_tls and clarifies when ssl_key/ssl_cert are required. |
| providers/celery/provider.yaml | Adds ssl_mutual_tls config entry and clarifies ssl_key/ssl_cert requirements. |
providers/celery/src/airflow/providers/celery/executors/default_celery.py
Show resolved
Hide resolved
providers/celery/src/airflow/providers/celery/executors/default_celery.py
Show resolved
Hide resolved
providers/celery/src/airflow/providers/celery/executors/default_celery.py
Show resolved
Hide resolved
dandanseo123
force-pushed
the
fix/celery-ssl-skip-empty-key-cert
branch
2 times, most recently
from
955c422 to
654d419
Compare
dandanseo123
requested review from
amoghrajesh,
ashb,
bugraoz93,
gopidesupavan,
jason810496,
jscheffl and
potiuk
as code owners
dandanseo123
force-pushed
the
fix/celery-ssl-skip-empty-key-cert
branch
from
654d419 to
e50c1c9
Compare
jscheffl
reviewed
Apr 11, 2026
providers/celery/src/airflow/providers/celery/executors/default_celery.py
Outdated
Show resolved
Hide resolved
jscheffl
approved these changes
Apr 11, 2026
dandanseo123
force-pushed
the
fix/celery-ssl-skip-empty-key-cert
branch
from
e50c1c9 to
43a93a9
Compare
jscheffl
approved these changes
Apr 11, 2026
Contributor
jscheffl
left a comment
jscheffl
left a comment
There was a problem hiding this comment.
Oh, even cooler! As next relese cut is anyway in ~1.5 weeks I leave the PR open for another pair of eyes, but if no other maintainer on it I think it is good to merge.
potiuk
approved these changes
Apr 13, 2026
Member
|
Nice! And good feedback! |
dandanseo123
pushed a commit
to dandanseo123/airflow
that referenced
this pull request
Apr 13, 2026
TestExecutorConf tests used conf.read_string() to set config values like ssl_active=true without cleanup, leaking state into subsequent tests. This caused CeleryExecutor SSL validation (added in apache#64767) to raise ValueError in unrelated test_executor_loader tests.
1 task
Contributor
Author
1 task
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
SSL_MUTUAL_TLSconfig option (default:True) to the[celery]section.When
False,SSL_KEYandSSL_CERTare skipped frombroker_use_ssl, enabling one-way TLS (server verification only viaSSL_CACERT). This is needed for setups using AMQP/Redis over TLS with token or password-based auth instead of client certificates.When
True(default), existing behavior is unchanged —SSL_KEYandSSL_CERTare required. A clear error is now raised if they are missing, replacing the previous cryptic[Errno 2]crash from py-amqp attempting to load empty certificate paths.Supports both AMQP (
amqps://) and Redis (rediss://) brokers.related: #39210
follow-up: #64392
Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code following the guidelines