Skip to content

Errors are not sent to Sentry in Airflow 3.0.2 #52368

Open
@fcapovilla

Description

@fcapovilla

Apache Airflow version

3.0.2

If "Other Airflow 2 version" selected, which one?

No response

What happened?

After upgrading an Airflow instance from 2.10.5 to 3.0.2, we noticed that errors generated by our tasks weren't sent to Sentry, even when the AIRFLOW__SENTRY__SENTRY_ON and AIRFLOW__SENTRY__SENTRY_DSN are set.

The same Sentry configuration worked on 2.10.5.

What you think should happen instead?

Task errors should be sent to Sentry when AIRFLOW__SENTRY__SENTRY_ON is activated and AIRFLOW__SENTRY__SENTRY_DSN is set correctly.

How to reproduce

  • Start an Airflow 3.0.2 project using the docker-compose.yml file provided in the documentation. Add apache-airflow[sentry] in the requirements with _PIP_ADDITIONAL_REQUIREMENTS and add the AIRFLOW__SENTRY__SENTRY_ON: "true" environment variable to the docker-compose.yml.
  • Set up a Sentry server and create a project to get a DSN.
  • Add the Sentry project's DSN to the AIRFLOW__SENTRY__SENTRY_DSN environment variable of the containers in docker-compose.yml
  • Create a DAG with a task only raising an exception
  • Check Sentry issues. No issue is created when the task fails.

Operating System

Fedora Linux 42

Versions of Apache Airflow Providers

package_name                              | description                                                                                  | version
==========================================+==============================================================================================+========
apache-airflow-providers-amazon           | Amazon integration (including Amazon Web Services (AWS) https://aws.amazon.com/)             | 9.8.0  
apache-airflow-providers-celery           | Celery https://docs.celeryq.dev/en/stable/                                                   | 3.11.0 
apache-airflow-providers-cncf-kubernetes  | Kubernetes https://kubernetes.io/                                                            | 10.5.0 
apache-airflow-providers-common-compat    | Common Compatibility Provider - providing compatibility code for previous Airflow versions   | 1.7.0  
apache-airflow-providers-common-io        | Common IO Provider                                                                           | 1.6.0  
apache-airflow-providers-common-messaging | Common Messaging Provider                                                                    | 1.0.3  
apache-airflow-providers-common-sql       | Common SQL Provider https://en.wikipedia.org/wiki/SQL                                        | 1.27.1 
apache-airflow-providers-docker           | Docker https://www.docker.com/                                                               | 4.4.0  
apache-airflow-providers-elasticsearch    | Elasticsearch https://www.elastic.co/elasticsearch                                           | 6.3.0  
apache-airflow-providers-fab              | Flask App Builder https://flask-appbuilder.readthedocs.io/                                   | 2.2.1  
apache-airflow-providers-ftp              | File Transfer Protocol (FTP) https://tools.ietf.org/html/rfc114                              | 3.13.0 
apache-airflow-providers-git              | Distributed version control system (GIT) https://git-scm.com/                                | 0.0.2  
apache-airflow-providers-google           | Google services including:                                                                   | 15.1.0 
                                          |                                                                                              |        
                                          |   - Google Ads https://ads.google.com/                                                       |        
                                          |   - Google Cloud (GCP) https://cloud.google.com/                                             |        
                                          |   - Google Firebase https://firebase.google.com/                                             |        
                                          |   - Google LevelDB https://github.com/google/leveldb/                                        |        
                                          |   - Google Marketing Platform https://marketingplatform.google.com/                          |        
                                          |   - Google Workspace https://workspace.google.com/ (formerly Google Suite)                   |        
apache-airflow-providers-grpc             | gRPC https://grpc.io/                                                                        | 3.8.0  
apache-airflow-providers-hashicorp        | Hashicorp including Hashicorp Vault https://www.vaultproject.io/                             | 4.2.0  
apache-airflow-providers-http             | Hypertext Transfer Protocol (HTTP) https://www.w3.org/Protocols/                             | 5.3.0  
apache-airflow-providers-microsoft-azure  | Microsoft Azure https://azure.microsoft.com/                                                 | 12.4.0 
apache-airflow-providers-mysql            | MySQL https://www.mysql.com/                                                                 | 6.3.0  
apache-airflow-providers-odbc             | ODBC https://github.com/mkleehammer/pyodbc/wiki                                              | 4.10.0 
apache-airflow-providers-openlineage      | OpenLineage https://openlineage.io/                                                          | 2.3.0  
apache-airflow-providers-postgres         | PostgreSQL https://www.postgresql.org/                                                       | 6.2.0  
apache-airflow-providers-redis            | Redis https://redis.io/                                                                      | 4.1.0  
apache-airflow-providers-sendgrid         | Sendgrid https://sendgrid.com/                                                               | 4.1.0  
apache-airflow-providers-sftp             | SSH File Transfer Protocol (SFTP) https://tools.ietf.org/wg/secsh/draft-ietf-secsh-filexfer/ | 5.3.0  
apache-airflow-providers-slack            | Slack https://slack.com/ services integration including:                                     | 9.1.0  
                                          |                                                                                              |        
                                          |   - Slack API https://api.slack.com/                                                         |        
                                          |   - Slack Incoming Webhook https://api.slack.com/messaging/webhooks                          |        
apache-airflow-providers-smtp             | Simple Mail Transfer Protocol (SMTP) https://tools.ietf.org/html/rfc5321                     | 2.1.0  
apache-airflow-providers-snowflake        | Snowflake https://www.snowflake.com/                                                         | 6.3.1  
apache-airflow-providers-ssh              | Secure Shell (SSH) https://tools.ietf.org/html/rfc4251                                       | 4.1.0  
apache-airflow-providers-standard         | Airflow Standard Provider                                                                    | 1.2.0  

Deployment

Docker-Compose

Deployment details

No response

Anything else?

I also tried to use the AIRFLOW__SENTRY__DSN environment variable to set the DSN, but it didn't work.

I checked in the code of Airflow 3.0.2, and it seems the only place where airflow.sentry.Sentry is used is in the _execute_in_fork fonction in https://github.com/apache/airflow/blob/main/providers/celery/src/airflow/providers/celery/executors/celery_executor_utils.py#L211 , but this function doesn't seem to be called in Airflow 3.0.2?

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:corekind:bugThis is a clearly a bugneeds-triagelabel for new issues that we didn't triage yet

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions