Skip to content

Add Azure Analysis Services hook, operator, sensor, and trigger#65879

Open
coleheflin wants to merge 9 commits into
apache:mainfrom
coleheflin:add-azure-analysis-services-provider
Open

Add Azure Analysis Services hook, operator, sensor, and trigger#65879
coleheflin wants to merge 9 commits into
apache:mainfrom
coleheflin:add-azure-analysis-services-provider

Conversation

@coleheflin
Copy link
Copy Markdown
Contributor

@coleheflin coleheflin commented Apr 26, 2026

Adds support for triggering and monitoring Azure Analysis Services model refreshes via the Analysis Services REST API, without requiring Azure Data Factory as an intermediary.

New components

  • AzureAnalysisServicesHook: authenticates with client secret or managed identity and wraps the REST API for triggering and polling refreshes
  • AzureAnalysisServicesRefreshOperator: triggers a model refresh and optionally waits for completion (sync or deferrable)
  • AzureAnalysisServicesSensor: polls an in-progress refresh by refresh ID
  • AzureAnalysisServicesRefreshTrigger: async trigger for deferrable mode

Connection type

A new azure_analysis_services connection type is registered with:

  • host: Region endpoint (e.g. eastus.asazure.windows.net)
  • login: Client ID
  • password: Client Secret
  • tenantId (extra): Azure Tenant ID

Managed identity and workload identity auth are also supported.

closes: #51377

Was generative AI tooling used to co-author this PR?
  • Yes — Claude Sonnet 4.6 (Claude Code)

Generated-by: Claude Sonnet 4.6 (Claude Code) following the guidelines

@coleheflin
Copy link
Copy Markdown
Contributor Author

coleheflin commented Apr 26, 2026

Manual Testing Completed

The following manual tests were run locally against the Breeze development environment:

Import sanity check ✅
All four new classes import cleanly:

from airflow.providers.microsoft.azure.hooks.analysis_services import AzureAnalysisServicesHook
from airflow.providers.microsoft.azure.operators.analysis_services import AzureAnalysisServicesRefreshOperator
from airflow.providers.microsoft.azure.sensors.analysis_services import AzureAnalysisServicesSensor
from airflow.providers.microsoft.azure.triggers.analysis_services import AzureAnalysisServicesRefreshTrigger
# All imports OK

DAG authoring ✅
A DAG using AzureAnalysisServicesRefreshOperator parses without errors inside Breeze.

Connection type UI ✅
The azure_analysis_services connection type appears in Admin → Connections with the correct custom field labels (Region Endpoint, Client ID, Client Secret, Tenant ID) and hidden fields (Schema, Port, Extra).

Unit tests ✅ (40/40 passing)
All hook, operator, sensor, and trigger tests pass with no live Azure credentials required.

The only remaining test is the end-to-end system test against a live Azure Analysis Services instance, which requires real credentials.

coleheflin and others added 4 commits April 28, 2026 20:08
@coleheflin
Add Azure Analysis Services hook, operator, sensor, and trigger
8bc4cff 
Adds support for triggering and monitoring Azure Analysis Services model
refreshes via the Analysis Services REST API, without requiring Azure
Data Factory as an intermediary.

New components:
- AzureAnalysisServicesHook: authenticates with client secret or managed
  identity and wraps the REST API for triggering and polling refreshes
- AzureAnalysisServicesRefreshOperator: triggers a model refresh and
  optionally waits for completion (sync or deferrable)
- AzureAnalysisServicesSensor: polls an in-progress refresh by ID
- AzureAnalysisServicesRefreshTrigger: async trigger for deferrable mode

closes: apache#51377
@coleheflin
Regenerate get_provider_info.py with Azure Analysis Services connecti…
dbefdf4 
…on type

The get_provider_info.py file is auto-generated from provider.yaml. Running
breeze release-management prepare-provider-documentation --reapply-templates-only
to include the new azure_analysis_services connection type registration.
@coleheflin
Fix CI failures: add missing docs and mypy guard for Analysis Service…
0c93dcb 
…s provider

- Add docs/connections/aas.rst with required howto/connection anchor
- Add docs/operators/analysis_services.rst how-to guide referenced in provider.yaml
- Guard settings.Session None check to fix mypy error in system test

Co-Authored-By: Cole Heflin <cole.heflin@astronomer.io>
@coleheflin
Replace raise AirflowException with dedicated exception class in Anal…
e37435c 
…ysis Services

Use AzureAnalysisServicesRefreshException (already defined in the hook) instead
of the broad AirflowException in the operator and sensor deferrable callbacks,
satisfying the check-no-new-airflow-exceptions static check.

Co-Authored-By: Cole Heflin <cole.heflin@astronomer.io>
@coleheflin coleheflin force-pushed the add-azure-analysis-services-provider branch from 4d7193d to e37435c Compare April 29, 2026 03:08
@coleheflin
Polish Azure Analysis Services provider: fix API correctness and docs
ae12b1d 
- Fix status values to match REST API: notStarted (was notProcessed),
  add timedOut as terminal failure status
- Remove .lower() on status response that broke camelCase comparisons
- Add RefreshType Literal and runtime validation with clear error message
- Fix trigger serialize() to use dynamic class path
- Fix asyncio.get_event_loop() -> get_running_loop() in trigger
- Add refresh types table and exampleinclude to operator docs
- Fix connection doc link to point to service principal setup page
@coleheflin
Fix connection doc link to point to AAS service principal setup page
f757c0a
@coleheflin coleheflin marked this pull request as ready for review April 29, 2026 03:36
@coleheflin
Copy link
Copy Markdown
Contributor Author

coleheflin commented May 1, 2026 •
edited
Loading

The remaining CI failure (Static checks / ruff) is unrelated to this PR.

The error is ASYNC110 flagged in providers/edge3/src/airflow/providers/edge3/cli/worker.py:557 — a pre-existing lint issue in the edge3 provider. None of the Azure Analysis Services files introduced here appear in the ruff output.

The previous Compat 2.11.1 failure was also unrelated — all 50 errors were Timeout >60.0s in Amazon provider tests (providers/amazon/tests/unit/...), again nothing touching Azure.

Drafted-by: Claude Code (claude-sonnet-4-6)

@potiuk potiuk added the ready for maintainer review Set after triaging when all criteria pass. label May 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

area:providers provider:microsoft-azure Azure-related issues ready for maintainer review Set after triaging when all criteria pass.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Azure Virtual Analysis Services Operator

2 participants

@coleheflin @potiuk