Under which category would you file this issue?
Providers
Apache Airflow version
3.3.0
What happened and how to reproduce it?
The change introduced by #66613 (for #39002) makes DatabricksRunNowOperator always include DAG-level params as job_parameters. Since apache-airflow-providers-databricks 7.16.0, users cannot opt out of passing those parameters, which creates compatibility issues for Databricks jobs whose entry point does not expect or accept them.
What you think should happen instead?
DatabricksRunNowOperator should provide a way to opt out of forwarding DAG-level params as job_parameters. Automatically injecting self.params changes the job payload and can break existing jobs that do not expect those parameters. The default behavior should either remain unchanged for backward compatibility or provide an explicit flag controlling parameter propagation.
Operating System
No response
Deployment
Official Apache Airflow Helm Chart
Apache Airflow Provider(s)
databricks
Versions of Apache Airflow Providers
apache-airflow-providers-databricks 7.16.0
Official Helm Chart version
1.21.0
Kubernetes Version
No response
Helm Chart configuration
No response
Docker Image customizations
No response
Anything else?
No response
Are you willing to submit PR?
Code of Conduct
Under which category would you file this issue?
Providers
Apache Airflow version
3.3.0
What happened and how to reproduce it?
The change introduced by #66613 (for #39002) makes DatabricksRunNowOperator always include DAG-level params as job_parameters. Since apache-airflow-providers-databricks 7.16.0, users cannot opt out of passing those parameters, which creates compatibility issues for Databricks jobs whose entry point does not expect or accept them.
What you think should happen instead?
DatabricksRunNowOperator should provide a way to opt out of forwarding DAG-level params as job_parameters. Automatically injecting self.params changes the job payload and can break existing jobs that do not expect those parameters. The default behavior should either remain unchanged for backward compatibility or provide an explicit flag controlling parameter propagation.
Operating System
No response
Deployment
Official Apache Airflow Helm Chart
Apache Airflow Provider(s)
databricks
Versions of Apache Airflow Providers
apache-airflow-providers-databricks 7.16.0
Official Helm Chart version
1.21.0
Kubernetes Version
No response
Helm Chart configuration
No response
Docker Image customizations
No response
Anything else?
No response
Are you willing to submit PR?
Code of Conduct