Broken grid view

[AlexSingle]

Apache Airflow version

3.1.6

If "Other Airflow 3 version" selected, which one?

No response

What happened?

Request /ui/grid/structure/dag_id?limit=10&order_by=-run_after failed with 500 Internal Server Error

Full error in server log

ERROR:    Exception in ASGI application
  + Exception Group Traceback (most recent call last):
  |   File "/usr/local/lib/python3.10/dist-packages/starlette/_utils.py", line 79, in collapse_excgroups
  |     yield
  |   File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/base.py", line 183, in __call__
  |     async with anyio.create_task_group() as task_group:
  |   File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 763, in __aexit__
  |     raise BaseExceptionGroup(
  | exceptiongroup.ExceptionGroup: unhandled errors in a TaskGroup (1 sub-exception)
  +-+---------------- 1 ----------------
    | Traceback (most recent call last):
    |   File "/usr/local/lib/python3.10/dist-packages/uvicorn/protocols/http/httptools_impl.py", line 416, in run_asgi
    |     result = await app(  # type: ignore[func-returns-value]
    |   File "/usr/local/lib/python3.10/dist-packages/fastapi/applications.py", line 1082, in __call__
    |     await super().__call__(scope, receive, send)
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/applications.py", line 113, in __call__
    |     await self.middleware_stack(scope, receive, send)
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 186, in __call__
    |     raise exc
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 164, in __call__
    |     await self.app(scope, receive, _send)
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/gzip.py", line 29, in __call__
    |     await responder(scope, receive, send)
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/gzip.py", line 130, in __call__
    |     await super().__call__(scope, receive, send)
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/gzip.py", line 46, in __call__
    |     await self.app(scope, receive, self.send_with_compression)
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/cors.py", line 85, in __call__
    |     await self.app(scope, receive, send)
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/base.py", line 182, in __call__
    |     with recv_stream, send_stream, collapse_excgroups():
    |   File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    |     self.gen.throw(typ, value, traceback)
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/_utils.py", line 85, in collapse_excgroups
    |     raise exc
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/base.py", line 184, in __call__
    |     response = await self.dispatch_func(request, call_next)
    |   File "/usr/local/lib/python3.10/dist-packages/airflow/api_fastapi/auth/middlewares/refresh_token.py", line 51, in dispatch
    |     response = await call_next(request)
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/base.py", line 159, in call_next
    |     raise app_exc
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/base.py", line 144, in coro
    |     await self.app(scope, receive_or_disconnect, send_no_error)
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/exceptions.py", line 63, in __call__
    |     await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    |     raise exc
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    |     await app(scope, receive, sender)
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 716, in __call__
    |     await self.middleware_stack(scope, receive, send)
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 736, in app
    |     await route.handle(scope, receive, send)
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 290, in handle
    |     await self.app(scope, receive, send)
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 78, in app
    |     await wrap_app_handling_exceptions(app, request)(scope, receive, send)
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    |     raise exc
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    |     await app(scope, receive, sender)
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 75, in app
    |     response = await f(request)
    |   File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 308, in app
    |     raw_response = await run_endpoint_function(
    |   File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 221, in run_endpoint_function
    |     return await run_in_threadpool(dependant.call, **values)
    |   File "/usr/local/lib/python3.10/dist-packages/starlette/concurrency.py", line 38, in run_in_threadpool
    |     return await anyio.to_thread.run_sync(func)
    |   File "/usr/local/lib/python3.10/dist-packages/anyio/to_thread.py", line 56, in run_sync
    |     return await get_async_backend().run_sync_in_worker_thread(
    |   File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 2441, in run_sync_in_worker_thread
    |     return await future
    |   File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 943, in run
    |     result = context.run(func, *args)
    |   File "/usr/local/lib/python3.10/dist-packages/airflow/api_fastapi/core_api/routes/ui/grid.py", line 185, in get_dag_structure
    |     _merge_node_dicts(merged_nodes, nodes)
    |   File "/usr/local/lib/python3.10/dist-packages/airflow/api_fastapi/core_api/services/ui/grid.py", line 42, in _merge_node_dicts
    |     _merge_node_dicts(current_node["children"], node["children"])
    |   File "/usr/local/lib/python3.10/dist-packages/airflow/api_fastapi/core_api/services/ui/grid.py", line 42, in _merge_node_dicts
    |     _merge_node_dicts(current_node["children"], node["children"])
    |   File "/usr/local/lib/python3.10/dist-packages/airflow/api_fastapi/core_api/services/ui/grid.py", line 37, in _merge_node_dicts
    |     for node in new:
    | TypeError: 'NoneType' object is not iterable
    +------------------------------------

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/uvicorn/protocols/http/httptools_impl.py", line 416, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/usr/local/lib/python3.10/dist-packages/fastapi/applications.py", line 1082, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/applications.py", line 113, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/gzip.py", line 29, in __call__
    await responder(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/gzip.py", line 130, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/gzip.py", line 46, in __call__
    await self.app(scope, receive, self.send_with_compression)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/cors.py", line 85, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/base.py", line 182, in __call__
    with recv_stream, send_stream, collapse_excgroups():
  File "/usr/lib/python3.10/contextlib.py", line 153, in __exit__
    self.gen.throw(typ, value, traceback)
  File "/usr/local/lib/python3.10/dist-packages/starlette/_utils.py", line 85, in collapse_excgroups
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/base.py", line 184, in __call__
    response = await self.dispatch_func(request, call_next)
  File "/usr/local/lib/python3.10/dist-packages/airflow/api_fastapi/auth/middlewares/refresh_token.py", line 51, in dispatch
    response = await call_next(request)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/base.py", line 159, in call_next
    raise app_exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/base.py", line 144, in coro
    await self.app(scope, receive_or_disconnect, send_no_error)
  File "/usr/local/lib/python3.10/dist-packages/starlette/middleware/exceptions.py", line 63, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 716, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 736, in app
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 290, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 78, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    raise exc
  File "/usr/local/lib/python3.10/dist-packages/starlette/_exception_handler.py", line 42, in wrapped_app
    await app(scope, receive, sender)
  File "/usr/local/lib/python3.10/dist-packages/starlette/routing.py", line 75, in app
    response = await f(request)
  File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 308, in app
    raw_response = await run_endpoint_function(
  File "/usr/local/lib/python3.10/dist-packages/fastapi/routing.py", line 221, in run_endpoint_function
    return await run_in_threadpool(dependant.call, **values)
  File "/usr/local/lib/python3.10/dist-packages/starlette/concurrency.py", line 38, in run_in_threadpool
    return await anyio.to_thread.run_sync(func)
  File "/usr/local/lib/python3.10/dist-packages/anyio/to_thread.py", line 56, in run_sync
    return await get_async_backend().run_sync_in_worker_thread(
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 2441, in run_sync_in_worker_thread
    return await future
  File "/usr/local/lib/python3.10/dist-packages/anyio/_backends/_asyncio.py", line 943, in run
    result = context.run(func, *args)
  File "/usr/local/lib/python3.10/dist-packages/airflow/api_fastapi/core_api/routes/ui/grid.py", line 185, in get_dag_structure
    _merge_node_dicts(merged_nodes, nodes)
  File "/usr/local/lib/python3.10/dist-packages/airflow/api_fastapi/core_api/services/ui/grid.py", line 42, in _merge_node_dicts
    _merge_node_dicts(current_node["children"], node["children"])
  File "/usr/local/lib/python3.10/dist-packages/airflow/api_fastapi/core_api/services/ui/grid.py", line 42, in _merge_node_dicts
    _merge_node_dicts(current_node["children"], node["children"])
  File "/usr/local/lib/python3.10/dist-packages/airflow/api_fastapi/core_api/services/ui/grid.py", line 37, in _merge_node_dicts
    for node in new:
TypeError: 'NoneType' object is not iterable

What you think should happen instead?

No response

How to reproduce

The errors started after I deleted a TaskGroup from DAG. Other Tasks and TaskGroups depended on this TaskGroup. And I deleted all dependencies between them, it should be a valid DAG. This problem didn't exist in airflow 2, the grid drew correctly after deleting any task or TaskGroup.

After I deleted older DagRuns grid started drawing correctly

Operating System

Ubuntu 22.04.4 LTS

Versions of Apache Airflow Providers

No response

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[tirkarthi]

Can you please share a sample dag with task group and the steps to reproduce this?

[AlexSingle]

My production DAG is really complex, basically i have a process, that reads a json config and generates full dag with task groups and dependencies. I tried to create a simplified example, but it worked correctly. Example was like this

group_a
task_a1
task_a2
task_a3
sensor_a4
task_a4

task_a1 -> task_a2
task_a1 -> task_a3
sensor_a4 -> task_a4
task_a3 -> task_a4

group_b
sensor_b1
task_b1
task_b2

sensor_b1 -> task_b1

group_c
task_c1
task_c2

task_a4 -> task_c1
task_c1 -> task_c2
task_b2 -> task_c2

group_d
sensor_d1
task_d1
task_d2
task_d3
task_d4

sensor_d1 -> task_d1
task_d1 -> task_d2
task_d1 -> task_d2
task_d2 -> task_d4
task_d3 -> task_d4

group_e
task_e1
task_e2

task_d4 -> task_e1
task_d4 -> task_e2

group_f
task_f1

task_e2 -> task_f1
task_c1 -> task_f1

group_g
task_g1
sensor_g2
task_g2
task_g3

task_e2 -> task_g1
task_c1 -> task_g1
sensor_g2 -> task_g2
task_c1 -> task_g2

and I deleted group_c and all tasks.

This is the first time I've encountered this error, so maybe I'm missing something. But before, I could simply add and remove any tasks and everything worked fine. Perhaps if I need to remove something in the future, I can create a more accurate example.

[jaymasiwal]

Hi @AlexSingle , thanks for the detailed report.
I’ll try to reproduce this on my side and investigate the NoneType issue in _merge_node_dicts.
I’ll share findings or a PR soon.