Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Allow a way to remove some spans from default integrations #966

Closed
khepin opened this issue Jul 16, 2020 · 6 comments
Closed

[Feature] Allow a way to remove some spans from default integrations #966

khepin opened this issue Jul 16, 2020 · 6 comments
Labels
cat:manual-instrumentation Issue which are related to manual tracing feature-request

Comments

@khepin
Copy link

khepin commented Jul 16, 2020

Is your feature request related to a problem? Please describe.
The default laravel tracing of every single event becomes problematic when loading large amounts of models. Here's an example of a trace:

Screen Shot 2020-07-16 at 2 59 49 PM

This query retrieves ~200 models and later on, we retrieve models attached to each of those models, which very rapidly brings us into the thousands of spans which are all of the type laravel.event.handle which in our case are not valuable to trace and bring us over the limit of 999 spans per trace.

Describe the solution you'd like
Since I seem to be able to replace the default handler for tracing those methods, I'd like to be able to disable it entirely as well and have no span created.

Potentially, this should be part of the SpanData itself so that we could keep some events while ignoring others though we don't have a use case for this at the moment.

Describe alternatives you've considered
Here's everything I've tried:

\dd_trace_method('Illuminate\Events\Dispatcher', 'fire', function() {});
\dd_trace_method('Illuminate\Events\Dispatcher', 'fire', null);
\dd_trace_method('Illuminate\Events\Dispatcher', 'fire', false);
\dd_trace_method('Illuminate\Events\Dispatcher', 'fire');
\dd_trace_method('Illuminate\Events\Dispatcher', 'fire', function ($span) {
    $span->setTag(\DDTrace\Tag::MANUAL_DROP, true);
});

The final alternative would be to not use the Laravel integration at all and rebuild it on our own. Which isn't a great solution in my opinion.

Additional context
Nope

@khepin
Copy link
Author

khepin commented Jul 16, 2020

image
This is all I've been able to get when setting the trace handler to false or null

@morrisonlevi
Copy link
Collaborator

morrisonlevi commented Jul 16, 2020

You can choose to drop a span by using returning false from trace_method/trace_function, but there are some issues with it because it effectively orphans (or drops) all its children spans too. If that's a side-effect you can live with (in many cases you can, because they won't make database calls or do anything else that is interesting) then that's your work-around.

A month or so ago I put in a feature request for being able to drop these kinds of things more reliably for exactly the kind of use case you see here! It hasn't been scheduled, but I'll be sure that it's brought up at the next planning meeting to see when we think we can schedule it.

@khepin
Copy link
Author

khepin commented Jul 16, 2020

Sweet! we can live with that in this case so I'll move forward with that solution. Greatly appreciate your help.

@labbati
Copy link
Member

labbati commented Sep 9, 2020

closing this issue as the solution proposed by Levi fixes the issue.

@kirkbushell
Copy link

closing this issue as the solution proposed by Levi fixes the issue.

It doesn't fix it, it's a workaround with a caveat that is potentially damaging.

We need (imho) the ability to configure what is actually recorded by datadog. We have the exact same issue.

@acarpio89
Copy link

@morrisonlevi I was hoping you could help us out on this one. We need to exclude eloquent.retrieved events because some requests have hundreds of these and we quickly reach the 1000 spans limit.

We're adding this to our code following the suggestion in this thread:

\DDTrace\trace_method(
    'Illuminate\Events\Dispatcher',
    'dispatch',
    function (\DDTrace\SpanData $span, array $args) {
        return false;
    }
);

but still getting all those eloquent.retrieved spans that we want to ignore:

Screenshot 2024-07-03 at 7 23 25 PM

I've added logging inside the closure and I can confirm it's running. I've also inspected the $args array and have been able to pinpoint the eloquent.retrieved events specifically, but returning false does not seem to drop them.

Do you have any suggestions?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cat:manual-instrumentation Issue which are related to manual tracing feature-request
Projects
None yet
Development

No branches or pull requests

5 participants