[NO-TICKET] Replace profiling.advanced.experimental_allocation_enabled with profiling.allocation_enabled and remove experimental warning

[ivoanjo]

What does this PR do?

In the next dd-trace-rb release, allocation profiling is no longer experimental. To reflect this, this PR includes two changes:

• It replaces the profiling.advanced.experimental_allocation_enabled with the new profiling.allocation_enabled option
• It turns the previous warning when turning on allocation profiling into a debug log message

Motivation:

Update code to match non-experimental status of feature.

Additional Notes:

I've decided to move the new setting outside the advanced configuration group, as enabling allocation profiling is not considered an "advanced" option.

To allow for smooth upgrades, I've left profiling.advanced.experimental_allocation_enabled in place with a deprecation warning, but unfortunately I was not able to find a nice way to automatically set the new profiling.allocation_enabled when the old one was set.

This means that users that had previously-enabled allocation profiling via code will need to update their configuration after upgrading, otherwise they won't get allocation profiling.

On the bright side, I was still able to support the old environment variable, so any customers that enabled the feature previously using that path will still get allocation profiling, plus a deprecation warning suggesting moving to the new environment variable.

How to test the change?

Change includes test coverage.

For Datadog employees:

• If this PR touches code that signs or publishes builds or packages, or handles
  credentials of any kind, I've requested a review from @DataDog/security-design-and-guidance.
• This PR doesn't touch any of that.

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 98.23%. Comparing base (77f8b0f) to head (ddf6b58).
Report is 5 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3520      +/-   ##
==========================================
- Coverage   98.24%   98.23%   -0.01%     
==========================================
  Files        1275     1275              
  Lines       75273    75295      +22     
  Branches     3554     3556       +2     
==========================================
+ Hits        73949    73968      +19     
- Misses       1324     1327       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[AlexJF]

Related to your PR description, is the problem that set_option(:allocation_enabled, value) would act on the advanced settings, not on the parent profiling settings?

I'm still not fully acquainted with the option code but it could be interesting if you could do allocation_enabled_option = option :allocation_enabled and then call allocation_enabled_option.set(value) from here. Although yeah if not heavily restricted I can see how this can create a mess of setting value discoverability over time 😂

[ivoanjo]

Related to your PR description, is the problem that set_option(:allocation_enabled, value) would act on the advanced settings, not on the parent profiling settings?

Yes, exactly -- unless I saw it completely wrong, the option groups objects form a tree and are linked only to their children.

I'm still not fully acquainted with the option code but it could be interesting if you could do allocation_enabled_option = option :allocation_enabled and then call allocation_enabled_option.set(value) from here. Although yeah if not heavily restricted I can see how this can create a mess of setting value discoverability over time 😂

Hmm... I'll admit I didn't try something like this. Let me give a final try to see if I can make it work.

[ivoanjo]

Ok so after thinking about it some more, here's how far I got:

1. allocation_enabled_option = option :allocation_enabled was a good try, but doesn't quite work because we get an OptionDefinition instance. The actual Option instance that keeps the data is created separately (in resolve_option) and so we don't actually have the object we need at this point.

2. It's possible to add a link to the parent group when defining the options -- here but I'm somewhat reluctant to add a feature to the options system (which is already quite complex) just for this

3. One thing that occurred to me would be inverting the order of the options. That is, allocation_enabled, since it's higher in the tree, is able to access the experimental_allocation_enabled, so one option would be for allocation_enabled to write its result into experimental_allocation_enabled and then have the code read the result from experimental_allocation_enabled until we deprecated it. That... would somewhat work, although it would interfere with the deprecation warning messages (e.g. not sure if we could still have them).

Overall... Given the current number of users we have using the experimental feature and that we're going to remove the option soon (e.g. for 2.0), I'm not sure it's worth trying really hard to support reading from the old config. (But let me know if you disagree!)

And overall our settings system is too damn complex >_>

[ivoanjo]

Given you approved, I'm going to go ahead and merge this, but happy to keep iterating on it as a separate PR if you're not convinced by my attempts above ;)