Skip to content

Fix templated validation in Azure operators - #70322

Open
Revanth14 wants to merge 2 commits into
apache:mainfrom
Revanth14:fix-azure-template-field-init
Open

Fix templated validation in Azure operators#70322
Revanth14 wants to merge 2 commits into
apache:mainfrom
Revanth14:fix-azure-template-field-init

Conversation

@Revanth14

Copy link
Copy Markdown
Contributor

Moves template-field-dependent logic out of __init__ for three Microsoft Azure provider classes, so it runs against Jinja-rendered values rather than the raw {{ ... }} templates that __init__ sees before rendering:

  • AzureVirtualMachineStateSensor now validates the rendered target_state during execution.
  • GCSToAzureBlobStorageOperator now checks rendered match_glob compatibility during execution.
  • OracleToAzureDataLakeOperator now uses a value-preserving default assignment for sql_params.

As a result, validation for the sensor and the GCS-to-WASB operator now surfaces at task-execution time rather than at DAG-parse time an intentional consequence of not touching template fields before they are rendered.

The corresponding entries were removed from validate_operators_init_exemptions.txt.
Regression tests render actual Jinja expressions before execution.

Related: #70296

Testing

console uv run --project providers/microsoft/azure pytest \ providers/microsoft/azure/tests/unit/microsoft/azure/sensors/test_compute.py \ providers/microsoft/azure/tests/unit/microsoft/azure/transfers/test_gcs_to_wasb.py \ providers/microsoft/azure/tests/unit/microsoft/azure/transfers/test_oracle_to_azure_data_lake.py \ -xvs ​

@Revanth14

Copy link
Copy Markdown
Contributor Author

@shahar1 This is for Azure Operators

@shahar1 shahar1 removed the backport-to-v3-3-test Backport to v3-3-test label Jul 24, 2026

@shahar1 shahar1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for your contribution!
Please add gen-AI disclosure section to the PR body.
Also, a minor comment.

assert sensor.azure_conn_id == CONN_ID

def test_init_invalid_target_state(self):
def test_invalid_templated_target_state_is_validated_after_rendering(self):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only the rejection path is tested; the behaviour the PR restores (valid templated value surviving to execution) is untested.

Comment on lines 70 to -76
super().__init__(**kwargs)
if sql_params is None:
sql_params = {}
self.filename = filename
self.oracle_conn_id = oracle_conn_id
self.sql = sql
self.sql_params = sql_params

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please restore the changes here (see #70505 for details)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants