Rake:Require instrumented task list #2174
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Release notes
Users of the Rake instrumentation will have to explicitly list the tasks that they want instrumented. This was done to prevent memory bloat issues that can arise from instrumenting long-running Rake tasks.
From today's configuration:
The list of desired Rake tests to instrument should be added:
Not providing this list will effectively instrument no Rake tasks.
What does this PR do?
This PR requires the Rake instrumentation to receive an explicit list of task to be instrumented. This list can be a list of String task names, or other values that can be stringified (e.g. it's common to declare Rake tasks with Symbol names).
Motivation
Long-running Rake tasks, those measured in days, can accumulate a very large active trace in memory. This normally won't be flushed until the application terminates.
Such traces can cause catastrophic memory accumulation, possibly triggering an out-of-memory process shutdown.
Examples of such cases can be found in #2045.
Additional Notes
I chose to no require users to have all their Rake loaded before invoking
Datadog.configure
: our current documentation suggests that users configureddtrace
before Rake task definition, so I chose to maintain compatibility and such flexibility.Also, the implementation is not very complicated: there's a small performance overhead on non-instrumented tasks, but it boils down to a
Set#include?
call.The dynamic implementation also allows for dynamic reconfiguration of the
tasks:
configuration list. The eager approach will either "patch" the Rake tasks and never be able to "unpatch", or it will implement the dynamic approach in this PR as well as selective patching.Testing for the unrelated
when tracing is disabled
case was a bit misleading around tracer#shutdown!
calls: it was asserting on aDatadog::Tracing.trace
instance, but another instance of the tracer was used during testing. This PR fixes that.How to test the change?
Unit test coverage around Rake tasks is already pretty good. I added the relevant cases introduced in this PR.