Airflow dependencies not working in 1.10.15

[sabarisan11]

Hi Everyone,

We are trying to migrate our code from Airflow version 1.10.5 to 1.10.15 (bridge version) and having plans to use Airflow 2.0. But post migrating our code into 1.10.15 we are facing some dependency issues where the same code works fine in 1.10.5.

When looked further, we identified that upstream and downstream functions with parameter is causing the issue. we tried modifying it with >> & << and that works. We have around 100 DAG in our system and it would be a huge task to modify all 100 DAG and test those. we also have some interdependencies like Task A is upstream to 3 tasks & also serves as downstream to 5 other tasks. In such cases it would be really difficult to go for some automated way to modify our code

Airflow Hanging Dependencies in 1.10.15

In graph view, dependencies appear like they are not properly connected but are actually connected. If I check the task instance details in this, upstream and downstream are set properly but it doesn't appear as expected

Same Code works in Airflow 1.10.5

where the same code works properly and dependencies shows up in 1.10.5 version

Sample Code snippet

Code block that has function parameters, here if the dependencies is between the operator it works fine, but when we brought python function which returns operator that is working well for past 2 years in Airflow version 1.10.5 but shows up this graph dependency issues in new version

Kindly take a look at this and provide some advice on how to proceed further

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[potiuk]

The 1.10 version of Airflow reached and of life as of June 17 2021 and it will not receive any more fixes (even critical ones). This looks like some of the code of yours does not work in 1.10.15 and it was a buggy behaviour in 1.10.5. It is highly unlikely it is going to work in Airflow 2 (which we encourage everyone to use). I am afraid fixing your DAGs to work with 1.10.15 is the only way to proceed. Since you already know a solution that works, I think this is the only way you can proceed.

Since we are not going to do anything with the issue, I am closing this one - feel free to copy it to a Github discussion instead if you have hopes that you will find someone with similar issue and a fix.