Under which category would you file this issue?
Providers
Apache Airflow version
main (development)
What happened and how to reproduce it?
This is a follow-up to #69822.
Removing the Vertex AI Feature Store system Dag's expensive parse-time imports allowed the generic system Dag import and serialization checks to proceed far enough to expose a separate problem in five Google Kubernetes Engine system Dags.
The GKE examples set initial_node_count directly on the cluster payload. The Google provider warns that this cluster-level field is deprecated and recommends configuring the initial node count on a node pool instead. Consequently, checks that treat provider deprecation warnings as errors fail while importing these example Dags.
Steps to reproduce:
-
Check out main with the Google provider development dependencies installed.
-
Run the affected example Dag import tests in Breeze while treating the warning as an error:
breeze run pytest airflow-core/tests/unit/always/test_example_dags.py \
-k 'example_kubernetes_engine' \
-W 'error:The body field.*is deprecated:UserWarning' \
-q --disable-warnings
-
Observe that importing the GKE system Dags fails because their cluster payloads use the
deprecated top-level initial_node_count field.
The affected examples are:
example_kubernetes_engine.pyexample_kubernetes_engine_async.pyexample_kubernetes_engine_job.pyexample_kubernetes_engine_kueue.pyexample_kubernetes_engine_resource.py
What you think should happen instead?
The GKE system Dags should define an explicit node pool and set initial_node_count on that node pool. They should import and serialize without emitting the cluster-level field deprecation warning.
Operating System
No response
Deployment
None
Apache Airflow Provider(s)
No response
Versions of Apache Airflow Providers
No response
Official Helm Chart version
Not Applicable
Kubernetes Version
No response
Helm Chart configuration
No response
Docker Image customizations
No response
Anything else?
No response
Are you willing to submit PR?
Code of Conduct
Under which category would you file this issue?
Providers
Apache Airflow version
main(development)What happened and how to reproduce it?
This is a follow-up to #69822.
Removing the Vertex AI Feature Store system Dag's expensive parse-time imports allowed the generic system Dag import and serialization checks to proceed far enough to expose a separate problem in five Google Kubernetes Engine system Dags.
The GKE examples set
initial_node_countdirectly on the cluster payload. The Google provider warns that this cluster-level field is deprecated and recommends configuring the initial node count on a node pool instead. Consequently, checks that treat provider deprecation warnings as errors fail while importing these example Dags.Steps to reproduce:
Check out
mainwith the Google provider development dependencies installed.Run the affected example Dag import tests in Breeze while treating the warning as an error:
breeze run pytest airflow-core/tests/unit/always/test_example_dags.py \ -k 'example_kubernetes_engine' \ -W 'error:The body field.*is deprecated:UserWarning' \ -q --disable-warningsObserve that importing the GKE system Dags fails because their cluster payloads use the
deprecated top-level
initial_node_countfield.The affected examples are:
example_kubernetes_engine.pyexample_kubernetes_engine_async.pyexample_kubernetes_engine_job.pyexample_kubernetes_engine_kueue.pyexample_kubernetes_engine_resource.pyWhat you think should happen instead?
The GKE system Dags should define an explicit node pool and set
initial_node_counton that node pool. They should import and serialize without emitting the cluster-level field deprecation warning.Operating System
No response
Deployment
None
Apache Airflow Provider(s)
No response
Versions of Apache Airflow Providers
No response
Official Helm Chart version
Not Applicable
Kubernetes Version
No response
Helm Chart configuration
No response
Docker Image customizations
No response
Anything else?
No response
Are you willing to submit PR?
Code of Conduct