DagFileProcessor 'NoneType' is not iterable

[momoadc]

Apache Airflow version

2.2.2

What happened

I'm seeing the same log repeating in the Scheduler.
I'm working in a restricted network so I cannot bring the entire log:

in bulk_write_to_db
    if orm_tag.name not in set(dag.tags)
TypeError: 'NoneType' object is not iterable

I saw that a single DAG didn't have any labels and i tried to add a label but the log is still showing

What you expected to happen

No response

How to reproduce

No response

Operating System

Debian 10 (Scheduler image)

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

I'm deploying on OpenShift 4.8 using the official Helm Chart v1.3.0

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template!

[ephraimbuddy]

Could you provide a simple DAG to reproduce this behaviour? I have tried setting tags=None but I couldn't get the error

[lizPayson]

I've experienced this issue with the following steps (the actual DAG contents are arbitrary, besides the tags)

1. You've got a dag, it gets parsed and it works. it does not have any tags associated with it. it does not have tags=None nor tags=[]
   example snippet:

with DAG(
    dag_id='SSH_Check',
    default_args = default_args,
    description='Checks that we can SSH in to a given host',
    schedule_interval=None #note that there are no tags here
) as dag:

2. give it a tag. maybe tags=["demo"]
   example snippet:

with DAG(
    dag_id='SSH_Check',
    default_args = default_args,
    description='Checks that we can SSH in to a given host',
    schedule_interval=None,
    tags=["demo"] #now we've got a tag. also a comma on preceding line
) as dag:

3. allow for reparsing (or kill scheduler to trigger reparse)
4. observe in webserver GUI that the dag has the given tag
5. in the dag code, remove the 'tags' line (removing the comma is optional, makes no difference)
   example snippet:

with DAG(
    dag_id='SSH_Check',
    default_args = default_args,
    description='Checks that we can SSH in to a given host',
    schedule_interval=None #now it looks like it's back to how it was in step 1
) as dag:

6. allow for reparsing. or restart scheduler. you may notice the mentioned error now

The issue appears to be related to removing all tags once a dag has been given a tag. a workaround is to put in tags=[] instead of removing the tag line completely. this will allow the dag to be parsed correctly, though it doesn't resolve the underlying issue, which seems to be that if a dag has a tag, the parser will struggle with the tags line being removed completely. you need to set it to an empty list instead. once the dag has been parsed once with an empty tag list, you can remove the entire line and it seems to be fine (if you just hate having that empty list there).

[potiuk]

Converting it to a discussion until more info is provided allowing to reproduce it.

[viktorvia]

This happened to our 2.2.2 deployment. We've had some DAGs with tags=[...] and when we completely removed that line, those DAGs caused this error at the scheduler.

The issue is that for some reason this did not result in import error that was visible anywhere, we had to be alerted by a partner that data was no longer being processed.

Another quick fix is to manually remove those entries from dag_tag in the matadb.

[potiuk]

Created an issue #22289 - thanks for providing reproducible steps.