Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Python] Enhancements to Queue instrumentation #2951

Closed
1 task done
Tracked by #63
antonpirker opened this issue Apr 9, 2024 · 1 comment
Closed
1 task done
Tracked by #63

[Python] Enhancements to Queue instrumentation #2951

antonpirker opened this issue Apr 9, 2024 · 1 comment
Assignees
Labels
Enhancement New feature or request

Comments

@antonpirker
Copy link
Member

antonpirker commented Apr 9, 2024

We want to improve the data we collect for task queues.
We will start with enhancing our Celery integration. This should work with redis, and rabbitmq, and as a stretch goal amazon sqs as brokers. (Kafka should follow later.)

How spans are right now:

  • Producer creates <span op="queue.submit.celery"> when task is put into queue. (start_timestamp is when putting into queue is started and end_timestamp is when the item is in the queue.)
  • Consumer creates <transaction op="queue.task.celery"> when task is fetched and processed. (start_timestamp is when the item was received from the queue, end_timestamp is when the processing of the task has finished. )
    • Transaction has a status:
      • ok in case of success
      • aborted if Celery raises Retry, Ignore or Reject celery exceptions
      • internal_error if an unhandled exception was raised during execution of the task

New data we want to add to queue spans:

(for naming see: https://opentelemetry.io/docs/specs/semconv/attributes-registry/messaging/)

  • retry count (can be found in args of existing _wrap_tracer, the attached PR)
  • byte size of message (messaging.message.body.size)
  • byte size of message plus metadata (messaging.message.envelope.size)
  • success/failure status - already there
  • host (server.address) - (can be found in args of existing _wrap_tracer, the attached PR)
  • queue or topic (messaging.destination.name) - not sure where to get this.
  • message id (messaging.message.id) - should also be in args of the comments in the attached PR)
  • consumer id (unique id of the consumer) (messaging.client_id) - dont know yet where to get.
  • what broker is used in celery. (messaging.system)

See this PR for some points where to get the data: #2962

I also added a sample project that uses RabbitMQ with Celery: https://github.com/antonpirker/testing-sentry/tree/main/test-celery-queues-module

Mock of how it will look in the product:

image

Tasks

  1. 8 of 8
    Enhancement Feature: Performance Integration: Celery
    szokeasaurusrex
@antonpirker antonpirker added the Enhancement New feature or request label Apr 9, 2024
@antonpirker antonpirker self-assigned this Apr 9, 2024
@antonpirker antonpirker changed the title Enhancements to Queue instrumentation [Python] Enhancements to Queue instrumentation Apr 9, 2024
@szokeasaurusrex szokeasaurusrex removed their assignment Apr 10, 2024
szokeasaurusrex added a commit that referenced this issue May 3, 2024
The messaging.system span data attribute should be set to the Celery broker being used, e.g. "amqp" for RabbitMQ, "redis" for Redis, and "sqs" for Amazon SQS.

ref #2951
szokeasaurusrex added a commit that referenced this issue May 3, 2024
The messaging.system span data attribute should be set to the Celery broker being used, e.g. "amqp" for RabbitMQ, "redis" for Redis, and "sqs" for Amazon SQS.

ref #2951
szokeasaurusrex added a commit that referenced this issue May 3, 2024
The messaging.system span data attribute should be set to the Celery broker being used, e.g. "amqp" for RabbitMQ, "redis" for Redis, and "sqs" for Amazon SQS.

ref #2951
szokeasaurusrex added a commit that referenced this issue May 3, 2024
The messaging.system span data attribute should be set to the Celery broker being used, e.g. "amqp" for RabbitMQ, "redis" for Redis, and "sqs" for Amazon SQS.

ref #2951
szokeasaurusrex added a commit that referenced this issue May 3, 2024
The messaging.system span data attribute should be set to the Celery broker being used, e.g. "amqp" for RabbitMQ, "redis" for Redis, and "sqs" for Amazon SQS.

ref #2951
szokeasaurusrex added a commit that referenced this issue May 3, 2024
The messaging.system span data attribute should be set to the Celery broker being used, e.g. "amqp" for RabbitMQ, "redis" for Redis, and "sqs" for Amazon SQS.

ref #2951
szokeasaurusrex added a commit that referenced this issue May 7, 2024
The messaging.system span data attribute should be set to the Celery broker being used, e.g. "amqp" for RabbitMQ, "redis" for Redis, and "sqs" for Amazon SQS.

ref #2951
szokeasaurusrex added a commit that referenced this issue May 7, 2024
The messaging.system span data attribute should be set to the Celery broker being used, e.g. "amqp" for RabbitMQ, "redis" for Redis, and "sqs" for Amazon SQS.

ref #2951
szokeasaurusrex added a commit that referenced this issue May 7, 2024
The messaging.system span data attribute should be set to the Celery broker being used, e.g. "amqp" for RabbitMQ, "redis" for Redis, and "sqs" for Amazon SQS.

ref #2951
szokeasaurusrex added a commit that referenced this issue May 10, 2024
The messaging.system span data attribute should be set to the Celery broker being used, e.g. "amqp" for RabbitMQ, "redis" for Redis, and "sqs" for Amazon SQS.

ref #2951
szokeasaurusrex added a commit that referenced this issue May 15, 2024
Set the "messaging.system" data on the "queue.process" span in the Celery integration. The messaging.system span data attribute should be set to the Celery broker being used, e.g. "amqp" for RabbitMQ, "redis" for Redis, and "sqs" for Amazon SQS. Also, add tests for this feature.

ref #2951
@antonpirker
Copy link
Member Author

This issue can be closed because the Celery integration was updated to emit spans that are eligible for the new Insights Queues module. (see mentioned PRs above)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants