[Feature] Make autoinstrumentation trace all custom PHP methods by default

[tmotyl]

Is your feature request related to a problem? Please describe.
Right now datadog APM is not useful for PHP code which is not written in one of the supported frameworks.
As the only things user see on the flame graph are db calls (e.g. mysqli, redis).

This means that for any PHP application which is not on the list of the supported frameworks Datadog requires plenty of time to integrate. You need to write custom instrumentation and whitelist functions you want to monitor.
So if you see slow traces in datadog, which spends much time in non instrumented functions (you see "gaps" in the chart) then you have to investigate, write more instrumentation code, deploy, monitor and repeat, untill you finnally find by trial end error what should be instrumented.
This approach makes performance debugging so much harder and timeconsuming.

Another issue is that there is no real guide to follow how to write custom framerowk/application support, the docs like https://docs.datadoghq.com/tracing/setup_overview/custom_instrumentation/php/?tab=tracingfunctioncalls don't cover some important aspects like how to make tracing available before application starts or at the very beginngin of requets handling.

Describe the solution you'd like
Datadog should capture method calls spans by default even for "unsupported" frameworks.
Other APM's (e.g. new relic) provide much better user experience out of the box - after installing the php extension, all userland PHP methods calls are being tracked.
This way you don't have to guess which methods should be instrumented, but see a full stacktrace every time

[labbati]

Hi @tmotyl. Let me answer to the various (good) points.

Most of the initial ideas fall under the following solution:

Other APM's (e.g. new relic) provide much better user experience out of the box - after installing the php extension, all userland PHP methods calls are being tracked.

We are taking a different approach to this, which in our very specific use case (I cannot talk about other companies), presents a lower overhead. Early in Q1-2021 we will start enrolling beta users for what we call method level tracing. For every slow span, you can follow a link that will bring you to the specific section in profiling data where you can look at stack trace (and not only that). This while providing a low performance overhead and no memory related critical issues.

Datadog should capture method calls spans by default even for "unsupported" frameworks.

The alternative to this would be to have a "trace all" approach by default (what you are suggesting in the title). Unless you are running a trivial application, this would be unacceptable from both performance and out-of-the-box stability stand point. Performance: because you would at list multiply by 2 the number of function invocations you have to do for every call. Stability: because of the tens of hundreds (if not thousands) of requests would count against memory_limit that in many production environments is kept reasonably low.

Another issue is that there is no real guide to follow how to write custom framerowk/application support, the docs like https://docs.datadoghq.com/tracing/setup_overview/custom_instrumentation/php/?tab=tracingfunctioncalls don't cover some important aspects like how to make tracing available before application starts or at the very beginngin of requets handling.

True that. Unfortunately, though, we have not a plan for that, yet. Let me explain. We are moving more and more code from PHP into C for a number of reasons related to stability and performance: so native integrations will be rewritten in C. Work is starting right now. We still have to design the API that can be used to provide custom integrations and we will definitely keep in mind. Among other things:

• users should be able to register an integration name, that then you can use for settings like in trace analytics configuration or sampling rate configuration
• users should be able to provide entry points to register the custom instrumentations to be run early on (so any bootstrap phase is instrumented).

If you have any idea in this direction we will be happy to consider it and review it.

[labbati]

@tmotyl I am closing this issue as a) tracing all custom methods would add a very large overhead and b) we are currently working on profiling which will add the sort of visibility you are looking for. We target early Q2 to onboard first customers in profiling and the work is currently in progress.