RabbitMQ connection unexpectedly closing

[kurtrwall]

Apache Airflow version

2.10.4

If "Other Airflow 2 version" selected, which one?

No response

What happened?

Using Airflow with RabbitMQ produces these RabbitMQ logs that seem to mean it's not properly closing connections interally.

rabbitmq-1   | 2025-01-02 19:12:24.816895+00:00 [info] <0.1548.0> accepting AMQP connection <0.1548.0> (172.19.0.8:34678 -> 172.19.0.2:5672)
rabbitmq-1   | 2025-01-02 19:12:24.818316+00:00 [info] <0.1548.0> connection <0.1548.0> (172.19.0.8:34678 -> 172.19.0.2:5672): user 'airflow' authenticated and granted access to vhost 'airflow'
rabbitmq-1   | 2025-01-02 19:12:24.822822+00:00 [info] <0.1565.0> accepting AMQP connection <0.1565.0> (172.19.0.8:34694 -> 172.19.0.2:5672)
rabbitmq-1   | 2025-01-02 19:12:24.823657+00:00 [info] <0.1565.0> connection <0.1565.0> (172.19.0.8:34694 -> 172.19.0.2:5672): user 'airflow' authenticated and granted access to vhost 'airflow'
rabbitmq-1   | 2025-01-02 19:12:24.898763+00:00 [warning] <0.1565.0> closing AMQP connection <0.1565.0> (172.19.0.8:34694 -> 172.19.0.2:5672, vhost: 'airflow', user: 'airflow'):
rabbitmq-1   | 2025-01-02 19:12:24.898763+00:00 [warning] <0.1565.0> client unexpectedly closed TCP connection
rabbitmq-1   | 2025-01-02 19:12:24.898856+00:00 [warning] <0.1548.0> closing AMQP connection <0.1548.0> (172.19.0.8:34678 -> 172.19.0.2:5672, vhost: 'airflow', user: 'airflow'):
rabbitmq-1   | 2025-01-02 19:12:24.898856+00:00 [warning] <0.1548.0> client unexpectedly closed TCP connection

What you think should happen instead?

No warning messages about unexpected connection closures.

How to reproduce

Start Airflow with RabbitMQ as the broker.

Operating System

OSX, docker

Versions of Apache Airflow Providers

apache-airflow-providers-apache-hdfs==4.7.0
apache-airflow-providers-apache-kafka==1.7.0
apache-airflow-providers-apache-spark==5.0.0
apache-airflow-providers-atlassian-jira==3.0.0
apache-airflow-providers-celery==3.8.5
apache-airflow-providers-common-compat==1.3.0
apache-airflow-providers-common-io==1.5.0
apache-airflow-providers-common-sql==1.21.0
apache-airflow-providers-docker==4.0.0
apache-airflow-providers-elasticsearch==6.0.0
apache-airflow-providers-fab==1.5.1
apache-airflow-providers-ftp==3.12.0
apache-airflow-providers-http==5.0.0
apache-airflow-providers-imap==3.8.0
apache-airflow-providers-postgres==6.0.0
apache-airflow-providers-redis==4.0.0
apache-airflow-providers-smtp==1.9.0
apache-airflow-providers-sqlite==4.0.0

Deployment

Official Apache Airflow Helm Chart

Deployment details

No response

Anything else?

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[Nikunj-Aggarwal]

Could you please assign this bug to me?

[ishtiaqSamdani007]

Hi @kurtrwall ,

I tried to replicate this issue and tested it on my local machine using Breeze with the following command:

breeze start-airflow --python 3.9 --backend postgres --dev-mode --executor CeleryExecutor --celery-broker rabbitmq

However, I did not encounter any errors related to connecting with RabbitMQ.
docker logs -f <rabbitmq-container-id> :

I also attempted to set up Airflow using the official Helm chart, but I noticed that there is no direct integration for RabbitMQ within the official Airflow Helm chart.

Could you provide more details on how to replicate the issue?

Thanks!

[potiuk]

It's likely that it's your rabbitmq that simply shuts down for whatever reason. Check this @kurtrwall . Converting it into a discussion, this is not "airflow" issue.