Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Laravel Queue Integration with Span Links #2026

Merged
merged 65 commits into from
May 8, 2023

Conversation

PROFeNoM
Copy link
Contributor

@PROFeNoM PROFeNoM commented Apr 27, 2023

Description

This PR comprises three features:

  • A Laravel queue integration for Laravel 5+
  • Implements Span Links
  • Fixes an issue where internal server errors would not be set on the root span
  • Fixes a bug where the tracer would enter limited mode because of an underflow in the number of open spans with multiple parallel traces

The Laravel Queue integration traces the following methods:

  • Illuminate\Queue\Worker::process
  • Illuminate\Queue\Jobs\Job::fire
  • Illuminate\Queue\Jobs\Job::resolve
  • (Laravel 8+) Illuminate\Bus\Batch::add
  • (Laravel 8+) Illuminate\Queue\Queue::enqueueUsing
  • Illuminate\Contracts\Queue\Queue::push
  • Illuminate\Contracts\Queue\Queue::later
  • The job's action

Things to note:

  • If applicable, the batch id will be retrieved and set in the metadata of the span. This will only be the case in Laravel 8+, as batches were implemented in this version.
  • It could be argued that tracing Illuminate\Queue\Queue::enqueueUsing doesn't provide much information. It just provides some context onto 'why are all these events happening'. However, I don't mind removing it.
  • When setting a span's metadata for a job that is an object - i.e., not an instance of Job, nor a string (e.g., redis), information about the connection and the queue will be tried to be retrieved. In the event that it wasn't possible to retrieve it in from the object or the parameters, the configuration defaults (config/queue.php) will be used.
  • (Artisan) In laravel's tests, the migrations will be run during the post-autoload-dump phase. Considering the sample apps are in production mode, the migrations are forced and the whole database is basically dropped and rebuilt from the migrations. This was needed to ran the jobs-related migrations.
  • (Laravel 4) This Laravel Queue integration doesn't provide support with Laravel 4.2
  • (Apache) While testing, I faced an issue where the apache server would not be restarted/reloaded after the configuration changed, hence that's why there is a change in PhpApache.php.
  • (CI) The Laravel Testsuite was disabled in the CI, as some Queue tests weren't passing as they would include additional information in their payload because of the distributed tracing. This could be fixed by modifying the corresponding docker image, but it would require cross-compilation.

Readiness checklist

  • (only for Members) Changelog has been added to the release document.
  • Tests added for this feature/bug.

Reviewer checklist

  • Appropriate labels assigned.
  • Milestone is set.
  • Changelog has been added to the release document. For community contributors the reviewer is in charge of this task.

Copy link
Collaborator

@pierotibou pierotibou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. I've just reviewed the tags you used based on the documentation for messaging system in confluence. Could you check your entries? I'd expect a bit more Tag::MQ_* to be used honestly, but maybe I'm wrong.
Can you also then fill the confluence page with the laravel specifics? It's not mandatory as won't be reused by others but it's a good habit to take I believe

{
$metadata = [
'messaging.laravel.attempts' => $job->attempts(),
'messaging.laravel.connection' => $job->getConnectionName() ?? config('queue.default'),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SHould we use messaging.url ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The connection name will be something like 'database', 'sync', or 'redis' (non-exhaustive, these are defined in the config/queue.php file). It doesn't really have the format we would expect from a url in my opinion. However, if you feel like this makes sense, then that's ok for me.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Indeed. Though I'd say it doesn't have the format of a connection either. Underlying system would seem more appropriate but 🤷

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this like this? 'mysql:host=mysql_integration;dbname=test'
In those cases my concerns are more about leaking sensitive info?

Copy link
Contributor Author

@PROFeNoM PROFeNoM Apr 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Laravel connections are defined using an associative array (as can be seen in this sample configuration) with specific configuration keys, rather than a single string containing all parameters. Therefore, the value of messaging.laravel.connection would be one of these configuration keys, referencing to one of the defined queue connection types. Hence, it shouldn't leak any sensitive information.

Note: A 'connection' in the context of laravel queue refers to the method by which the queue system communicates with the underlying storage mechanism used to store and retrieve queued jobs.

ext/ddtrace.stub.php Outdated Show resolved Hide resolved
'messaging.laravel.max_tries' => $job->maxTries(),
'messaging.laravel.timeout' => $job->timeout(),
'messaging.laravel.name' => $job->resolveName(),
Tag::MQ_SYSTEM => 'laravel',
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't this supposed to be like kafka, rabbitmq, rocketmq, activemq, AmazonSQS, etc?

Copy link
Collaborator

@bwoebi bwoebi Apr 28, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, Laravel queues aren't necessarily dispatched via these mechanisms. It's a separate mechanism.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On this, I was also planning to update the confluence page to include the newly introduced span attributes; messaging.system would be one of these.

ext/ddtrace.c Outdated Show resolved Hide resolved
ext/ddtrace.stub.php Outdated Show resolved Hide resolved
Copy link
Collaborator

@bwoebi bwoebi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a couple nits, but looks fine for merge then.

@bwoebi bwoebi merged commit d76aa19 into master May 8, 2023
1 of 22 checks passed
@bwoebi bwoebi deleted the alex/integration/laravel-queue branch May 8, 2023 16:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏆 enhancement A new feature or improvement 🎉 new-integration A new integration tracing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants