Skip to content

Add Cloud SQL Auth Proxy IAM authentication#66510

Merged
shahar1 merged 4 commits into
apache:mainfrom
nailo2c:feat-35347_support_enable_iam_login
May 12, 2026
Merged

Add Cloud SQL Auth Proxy IAM authentication#66510
shahar1 merged 4 commits into
apache:mainfrom
nailo2c:feat-35347_support_enable_iam_login

Conversation

@nailo2c
Copy link
Copy Markdown
Contributor

@nailo2c nailo2c commented May 7, 2026

closes: #35347

How

Add the sql_proxy_enable_iam_login parameter to support -enable_iam_login in CloudSqlProxyRunner.

What

Reproducer DAG:

import os
from datetime import datetime

from airflow import DAG
from airflow.providers.google.cloud.operators.cloud_sql import CloudSQLExecuteQueryOperator

DAG_ID = "cloudsql_proxy_iam_reproduce"
GCP_CONN_ID = os.environ.get("AIRFLOW_35347_GCP_CONN_ID", "google_cloud_default")
CLOUDSQL_CONN_ID = os.environ.get("AIRFLOW_35347_CLOUDSQL_CONN_ID", "cloudsql_proxy_iam_postgres")

with DAG(
    dag_id=DAG_ID,
    schedule=None,
    start_date=datetime(2026, 1, 1),
) as dag:
    CloudSQLExecuteQueryOperator(
        task_id="query_cloudsql_with_proxy_iam",
        gcp_conn_id=GCP_CONN_ID,
        gcp_cloudsql_conn_id=CLOUDSQL_CONN_ID,
        sql=["SELECT 1"],
    )

Connection:

{
	"conn_type": "gcpcloudsql",
	"login": "<your-sa-name>@<your-project>.iam.gserviceaccount.com",
	"password": "",
	"host": "127.0.0.1",
	"port": 5432,
	"schema": "test_db",
	"extra": {
		"database_type": "postgres",
		"project_id": "<your-project>",
		"location": "us-central1",
		"instance": "<your-instance-name>",
		"use_proxy": "True",
		"sql_proxy_use_tcp": "True",
		"sql_proxy_enable_iam_login": "True"
	}
} 

Before fix

af_35347_before_fix

After fix

af_35347_after_fix

System test

af_35347_system_test

Cloud SQL instance lifecycle (create → delete) executed as expected.
Screenshot 2026-05-06 at 8 24 43 PM


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)
    Generated-by: Codex 5.5 following the guidelines

  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@nailo2c nailo2c requested a review from shahar1 as a code owner May 7, 2026 04:13
@boring-cyborg boring-cyborg Bot added area:providers kind:documentation provider:google Google (including GCP) related issues labels May 7, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for Cloud SQL Auth Proxy IAM DB authentication in the Google Cloud SQL integration by introducing a new sql_proxy_enable_iam_login option that maps to the proxy’s -enable_iam_login flag, allowing connections with an empty password when using the proxy.

Changes:

  • Add sql_proxy_enable_iam_login plumbing to CloudSqlProxyRunner and CloudSQLDatabaseHook, including validation and connection-URI/parameters handling for empty passwords.
  • Add unit tests covering Postgres/MySQL proxy IAM behavior and flag propagation to the proxy runner.
  • Add a new system test example DAG and update connection documentation to describe the two IAM authentication paths.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
providers/google/src/airflow/providers/google/cloud/hooks/cloud_sql.py Adds the new sql_proxy_enable_iam_login parameter, validation, and proxy flag injection.
providers/google/tests/unit/google/cloud/hooks/test_cloud_sql.py Adds unit tests for proxy IAM behavior, empty-password URI generation, and proxy runner flag behavior.
providers/google/tests/system/google/cloud/cloud_sql/example_cloud_sql_query_proxy_iam.py Adds a system test example DAG demonstrating proxy IAM authentication flow.
providers/google/docs/connections/gcp_sql.rst Documents direct IAM token auth vs proxy IAM auth and updates password requirements accordingly.

Comment thread providers/google/src/airflow/providers/google/cloud/hooks/cloud_sql.py Outdated
@potiuk potiuk added the ready for maintainer review Set after triaging when all criteria pass. label May 11, 2026
Copy link
Copy Markdown
Contributor

@shahar1 shahar1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great feature!!!
#PROTM

@shahar1 shahar1 merged commit bfec6cf into apache:main May 12, 2026
94 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:providers kind:documentation provider:google Google (including GCP) related issues ready for maintainer review Set after triaging when all criteria pass.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Enable IAM based authentication from CloudSQL airflow operators to connect to Cloudsql in GCP

4 participants