ODBC Provider Configuration

[hbc-acai]

Apache Airflow version

Other Airflow 2 version (please specify below)

If "Other Airflow 2 version" selected, which one?

2.9.1

What happened?

I added environment variable AIRFLOW__PROVIDERS_ODBC__ALLOW_DRIVER_IN_EXTRA, but it seems it is not picked up by the config parser.

This is what I see:

1. environment variable is there. It is set in the helm chart overrides.

airflow@airflow-scheduler-69d5c977c-r85tz:/opt/airflow$ echo $AIRFLOW__PROVIDERS_ODBC__ALLOW_DRIVER_IN_EXTRA
True

2. The list command does not return anything

airflow@airflow-scheduler-69d5c977c-r85tz:/opt/airflow$ airflow config list --section providers.odbc
/home/airflow/.local/lib/python3.10/site-packages/airflow/metrics/statsd_logger.py:184 RemovedInAirflow3Warning: The basic metric validator will be deprecated in the future in favor of pattern-matching.  You can try this now by setting config option metrics_use_pattern_match to True.

3. Interestingly, the get-value command shows the variable

airflow@airflow-scheduler-69d5c977c-r85tz:/opt/airflow$ airflow config get-value providers.odbc allow_driver_in_extra
/home/airflow/.local/lib/python3.10/site-packages/airflow/metrics/statsd_logger.py:184 RemovedInAirflow3Warning: The basic metric validator will be deprecated in the future in favor of pattern-matching.  You can try this now by setting config option metrics_use_pattern_match to True.
True

4. When I use SQLSensor, I get this warning:

[2024-06-24, 13:58:51 UTC] {odbc.py:119} WARNING - You have supplied 'driver' via connection extra but it will not be used. In order to use 'driver' from extra you must set airflow config setting `allow_driver_in_extra = True` in section `providers.odbc`. Alternatively you may specify driver via 'driver' parameter of the hook constructor or via 'hook_params' dictionary with key 'driver' if using SQL operators.

What you think should happen instead?

No response

How to reproduce

Add the following section in the helm chart overrides:

- Env
  - name: AIRLFOW__PROVIDERS_ODBC__ALLOW_DRIVER_IN_EXTRA
    value: "True"

Operating System

Azure Kubernetes Service

Versions of Apache Airflow Providers

airflow@airflow-scheduler-69d5c977c-r85tz:/opt/airflow$ pip freeze |grep airflow-prov
apache-airflow-providers-amazon==8.20.0
apache-airflow-providers-celery==3.6.2
apache-airflow-providers-cncf-kubernetes==8.3.0
apache-airflow-providers-common-io==1.3.1
apache-airflow-providers-common-sql==1.12.0
apache-airflow-providers-docker==3.10.0
apache-airflow-providers-elasticsearch==5.3.4
apache-airflow-providers-fab==1.0.4
apache-airflow-providers-ftp==3.8.0
apache-airflow-providers-google==10.17.0
apache-airflow-providers-grpc==3.4.1
apache-airflow-providers-hashicorp==3.6.4
apache-airflow-providers-http==4.10.1
apache-airflow-providers-imap==3.5.0
apache-airflow-providers-microsoft-azure==10.0.0
apache-airflow-providers-mysql==5.5.4
apache-airflow-providers-odbc==4.5.0
apache-airflow-providers-openlineage==1.7.0
apache-airflow-providers-postgres==5.10.2
apache-airflow-providers-redis==3.6.1
apache-airflow-providers-sendgrid==3.4.0
apache-airflow-providers-sftp==4.9.1
apache-airflow-providers-slack==8.6.2
apache-airflow-providers-smtp==1.6.1
apache-airflow-providers-snowflake==5.4.0
apache-airflow-providers-sqlite==3.7.1
apache-airflow-providers-ssh==3.10.1

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[potiuk]

No idea what's the reason, but this is more of a discussion on how to set it up, not "airflow" issue.

[potiuk]

Converted it to a discussion - hopefully someone who has some odbc experience might help

[FrankYang0529]

I open a PR to fix config list part. The root cause is the config was not declared in provider.yaml.

#72547