Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support retriveing extra fields which do not have extra prefix set - Azure ADF DAG failure #899

Merged
merged 3 commits into from
Feb 27, 2023
Merged

Support retriveing extra fields which do not have extra prefix set - Azure ADF DAG failure #899

merged 3 commits into from
Feb 27, 2023

Commits on Feb 27, 2023

  1. Support retriveing extra fields which do not have extra prefix set 

    With the 5.0.0 release of the Microsoft Azure PR, the PR apache/airflow#27758
introduced a breaking change where while creating new connections,
the ``extra__`` prefix is no longer set for extra fields in the conneciton.
This issue was not identified with testing the 5.0.0 RC because, it
only happens for new connections that are created. The existing connections
still contain the extra fiels with the ``extra__`` prefix. Hence, the
existing code which looks for the connection field with the prefix
``extra__azure_data_factory__subscriptionId`` works on the older deployment
with the new provider release as the connection was created before the release,
but it fails on new deployments when a fresh connection is created.

To fix this, we're removing the prefix while retrieving the connection
field and at the same time we're supporting previously created connections
by using the same ``get_field`` method from Airflow OSS introduced
in the same PR above to allow backward compatibility.
    @pankajkoti
    pankajkoti committed Feb 27, 2023
    Configuration menu
    Copy the full SHA
    2204b1d View commit details
    Browse the repository at this point in the history

  2. Add tests to test the get field method for prefixed and non-prefixed … 

    …extras
    @pankajkoti
    pankajkoti committed Feb 27, 2023
    Configuration menu
    Copy the full SHA
    a51449b View commit details
    Browse the repository at this point in the history

  3. Fix mypy failure

    @pankajkoti
    pankajkoti committed Feb 27, 2023
    Configuration menu
    Copy the full SHA
    029bf25 View commit details
    Browse the repository at this point in the history