Add configuration support for aiohttp.ClientSession kwargs (e.g. trust_env) in deferrable operators / triggerer

[michaelneely]

Description

Airflow should provide a global supported config mechanism to pass additional kwargs to aiohttp.ClientSession at construction time. For example, an Airflow config key — e.g. [aiohttp] trust_env = True in airflow.cfg, read by a shared session factory used across all providers.

Use case/motivation

Problem

Deferrable operators that use aiohttp internally (e.g. SnowflakeSqlApiOperator, and others backed by async HTTP clients) create aiohttp.ClientSession instances without passing trust_env=True. This means they silently ignore proxy-related environment variables (HTTPS_PROXY, HTTP_PROXY, NO_PROXY) set in the execution environment.

In managed environments like AWS MWAA — where corporate proxy settings are injected via a startup script into environment variables — this causes triggers to bypass the proxy entirely and time out, while synchronous operators using requests or urllib work correctly.

Current workaround

You can set trust_env globally via a Monkey-patch of aiohttp.ClientSession.__init__ in airflow_local_settings.py:

import aiohttp
from typing import Any

_original_init = aiohttp.ClientSession.__init__

def _trust_env_init(self: aiohttp.ClientSession, *args: Any, **kwargs: Any) -> None:
    kwargs.setdefault("trust_env", True)
    _original_init(self, *args, **kwargs)

aiohttp.ClientSession.__init__ = _trust_env_init

This is fragile, not discoverable, and should not be necessary for what is effectively a standard proxy configuration requirement.

Environment

Version: Apache Airflow 3.x (probably also affects 2.x)
Provider: apache-airflow-providers-snowflake (and any other provider using aiohttp in triggers)
Deployment: AWS MWAA (corporate proxy injected via startup script env vars)

Impact

Any organisation running Airflow behind a corporate proxy will hit this silently — deferrable operators time out with no clear indication that proxy settings are being ignored.

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[Codingaditya17]

Hi, I can take a look at this.

I will start by investigating where aiohttp.ClientSession is constructed in core and provider trigger paths, then add a shared supported config mechanism for session kwargs such as trust_env if that direction fits the existing config patterns.

I will keep the initial PR focused on trust_env / proxy support and include tests for the config parsing and session construction behavior.

[onlyarnav]

I'd like to work on this issue.

My initial approach is:

• Add a new [aiohttp] config section with trust_env support
• Create a shared function for aiohttp.ClientSession creation
• Update existing usages to use the helper function
• Add tests for config propagation

Please let me know if there are any preferred implementation details.

[Codingaditya17]

Hi @onlyarnav, I opened a PR for this here: #67906

I kept the first implementation focused on the Snowflake SQL API deferrable path since that was one of the affected examples mentioned in the issue.

The PR adds [aiohttp] trust_env config support, applies it when Snowflake creates async aiohttp.ClientSession instances, and also passes aiohttp_session_kwargs through SnowflakeSqlApiTrigger serialization so the triggerer does not lose those session options while polling.

Happy to adjust the scope if maintainers prefer this to be expanded into a shared helper used by more providers in the same PR.