Contrib:Make registry a global constant repository #1572

marcotc · 2021-06-30T23:34:03Z

Changelog text

The tracer's integration registry cannot be directly configured anymore (e.g. c.registry = CustomRegistry.new). As we make the tracer initialization process more strict, to ensure correctness, we removed the ability to arbitrarily modify the global registry.

It is still possible to register custom integrations into the global registry at any point of the application execution.

PR description

As part of #1527, this PR make the integration registry a global constant, instead of a configurable value.

The registry today is effectively a declarative, additive list of supported integrations. These integrations are made available for configuration by being added to such register.

With changes to initialization coming in #1527, the registry will need to be strict with it's data policy: arbitrary modifications of the registry won't be allowed, and registered integrations will have to support being reconfigured without requiring re-registration.

This is already the case today, but will be enforced to ensure strict initialization corrected in #1527.

Accessing the registry through Datadog.registry is allowed, but internal references should use Datadog::Contrib::REGISTRY, as Datadog.registry is only available after the tracer has fully initialized.

This PR also creates the lib/ddtrace/contrib.rb, which declares the registry, plus takes care of requiring all integrations. This helps with isolating contrib from the top-level ddtrace. This caused changes to the top-level lib/ddtrace.rb file, which I believe more explicitly shows what's modified by loading contrib extensions.

codecov-commenter · 2021-06-30T23:59:14Z

Codecov Report

Merging #1572 (dca2ae5) into master (1914b0c) will increase coverage by 0.00%.
The diff coverage is 100.00%.

@@           Coverage Diff            @@
##           master    #1572    +/-   ##
========================================
  Coverage   98.30%   98.30%            
========================================
  Files         900      901     +1     
  Lines       43194    43294   +100     
========================================
+ Hits        42461    42560    +99     
- Misses        733      734     +1

Impacted Files	Coverage Δ
lib/datadog/contrib.rb	`100.00% <100.00%> (ø)`
lib/datadog/core/environment/container.rb	`93.02% <100.00%> (ø)`
lib/ddtrace.rb	`98.55% <100.00%> (+0.04%)`	⬆️
lib/ddtrace/contrib/auto_instrument.rb	`95.23% <100.00%> (ø)`
lib/ddtrace/contrib/extensions.rb	`100.00% <100.00%> (ø)`
lib/ddtrace/contrib/registerable.rb	`100.00% <100.00%> (ø)`
spec/datadog/core/environment/cgroup_spec.rb	`100.00% <100.00%> (ø)`
spec/datadog/core/environment/container_spec.rb	`100.00% <100.00%> (ø)`
spec/ddtrace/contrib/extensions_spec.rb	`95.45% <100.00%> (+0.40%)`	⬆️
spec/support/container_helpers.rb	`100.00% <100.00%> (ø)`
... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e58e79c...dca2ae5. Read the comment docs.

marcotc · 2021-07-01T00:00:06Z

I've moved some stuff around, specially in lib/ddtrace.rb and lib/ddtrace/contrib.rb, so I appreciate any feedback on that part.

ivoanjo

Left a few notes, but nothing big/blocking. 👍

lib/ddtrace.rb

lib/ddtrace/contrib.rb

lib/ddtrace/contrib/auto_instrument.rb

ivoanjo · 2021-07-05T13:43:13Z

lib/ddtrace/contrib/extensions.rb

      # Helper methods for Datadog module.
      module Helpers
+        # Returns the global integration registry.
+        #
+        # This method is not safe to use while the tracer is initializing,
+        # thus access to the registry should go through
+        # ::Datadog::Contrib::REGISTRY for internal tracer work.
+        #
+        # External use of this method is always safe.
        def registry
-          configuration.registry
+          Contrib::REGISTRY
        end
      end


This got me thinking: I understand that the integrations are not part of the core. Would it make sense to make the registry part of the core itself? It seems to be a bit "too modular" to have the extension support itself be modular, rather than something that's always there as part of the core.

TL;DR the main difference would be that we'd move this method directly inside the Datadog module.

I can take a stab at that and see how it looks

After looking into it, I agree with this.
But I think this will modify too much to be able to fit nicely in this PR.

I documented this sentiment in code, so we know this is the direction we want for this part of our code in the future.

Minor: I agree with avoiding the bigger change, but on my second go around reviewing this I was thinking that we may be able to just inline the Helper module into datadog.rb, and thus start a bit on the correct path.

I tried to refactor Helper, but it looked very ugly without moving the rest of extensions.rb with it 😐

Fair enough, thanks for making a try :)

lib/ddtrace/contrib/extensions.rb

ivoanjo · 2021-07-05T13:51:59Z

lib/ddtrace/contrib/extensions.rb

+          def registry
+            Datadog.logger.warn('Deprecated access to `Datadog.configuration.registry`, use `Datadog.registry` instead.')
+            Contrib::REGISTRY


Minor:

I recommend that we also state "Deprecated for removal" in the log message, to make it clear that we don't plan on leaving this around for long

Should the last line just be Datadog.registry?

I updated the message.

The reason to use Datadog::Contrib::REGISTRY internally in the tracer is that it's a direct access to the constant, and it's the only safe way to access the registry before the tracer initializes.

Datadog.registry is only available after the tracer has fully initialized.

Today this makes no different because calls to Datadog.registry lazily trigger the initialization of the whole tracer, but after we eager initialize the tracer that won't be the case. Users will always receive the tracer initialized, but internal components might see it uninitialized during gem loading.

ivoanjo · 2021-07-05T13:52:42Z

lib/ddtrace/contrib/extensions.rb

 module Datadog
  module Contrib


Definitely not for this PR, but this whole file makes me sad. We shouldn't be monkey patching our own code to support extensions 😭

spec/ddtrace/auto_instrument_spec.rb

ivoanjo · 2021-07-05T13:58:04Z

spec/ddtrace/contrib/action_cable/patcher_spec.rb

  around do |example|
    # Reset before and after each example; don't allow global state to linger.
-    Datadog.registry[:action_cable].reset_configuration!
+    Datadog::Contrib::REGISTRY[:action_cable].reset_configuration!
    example.run
-    Datadog.registry[:action_cable].reset_configuration!
+    Datadog::Contrib::REGISTRY[:action_cable].reset_configuration!
  end


Minor: This seems to be repeated so often that I wonder if we should have a nice helper for this -- something along the lines of with_clean_configuration(:action_cable) { example.run }.

Spoiler: 99% of these reset_configuration! will go away when the tracer is eager initialized (more so when the lifecyle object is correctly disposed and re-created on every run). So we'll soon remove these altogether instead :)

marcotc · 2021-07-23T21:49:40Z

@ivoanjo I think I addresses all points you brought up, please let me know if anything is still missing.

ivoanjo

Almost there! Thanks for addressing my comments so far :)

lib/ddtrace.rb

lib/datadog/contrib.rb

lib/ddtrace/contrib/extensions.rb

ivoanjo · 2021-07-26T10:17:24Z

lib/ddtrace/contrib/extensions.rb

      # Helper methods for Datadog module.
      module Helpers
+        # Returns the global integration registry.
+        #
+        # This method is not safe to use while the tracer is initializing,
+        # thus access to the registry should go through
+        # ::Datadog::Contrib::REGISTRY for internal tracer work.
+        #
+        # External use of this method is always safe.
        def registry
-          configuration.registry
+          Contrib::REGISTRY
        end
      end


Minor: I agree with avoiding the bigger change, but on my second go around reviewing this I was thinking that we may be able to just inline the Helper module into datadog.rb, and thus start a bit on the correct path.

lib/ddtrace/contrib/extensions.rb

spec/ddtrace/contrib/extensions_spec.rb

marcotc added the dev/refactor Involves refactoring existing components label Jun 30, 2021

marcotc self-assigned this Jun 30, 2021

marcotc requested a review from a team June 30, 2021 23:34

Contrib:Make registry a global constant repository

ebfe116

marcotc force-pushed the global-registry branch from 8ead0e4 to ebfe116 Compare June 30, 2021 23:53

ivoanjo approved these changes Jul 5, 2021

View reviewed changes

marcotc added 9 commits July 23, 2021 14:03

Merge branch 'master' into global-registry

f379daa

Move requires to outside module

0967aee

Move new contrib file to new namespace

72e8d43

Fix registry= deprecation message

741b362

Better user message on registry=

f9e67d7

Better message for future deprecation removal

8714ae5

Use Datadog.registry on contrib test

ac2c6bb

Match new warning message correctly in test

0416d9f

Document registry conundrum

dca2ae5

marcotc requested a review from ivoanjo July 23, 2021 21:49

Add draft release action

537a9c0

ivoanjo reviewed Jul 26, 2021

View reviewed changes

marcotc added 9 commits July 26, 2021 12:07

Merge branch 'master' into global-registry

8ed81ac

Re-add semantic logger

bf4ff8b

Remove bad merge restoring require at top leve

d0faf6d

Document initialization in constant place

d1d0c66

Suggest public API

92dcfca

Fix typo

d7ff61c

Another typo

25e99d3

Add mock name

95ed435

Oops file

141736c

Merge branch 'master' into global-registry

7d14b39

marcotc merged commit 85ba2db into master Jul 26, 2021

marcotc deleted the global-registry branch July 26, 2021 21:49

github-actions bot added this to the 0.52.0 milestone Jul 26, 2021

marcotc mentioned this pull request Nov 4, 2021

Deprecate:Contrib::Configuration#registry #1758

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Contrib:Make registry a global constant repository #1572

Contrib:Make registry a global constant repository #1572

marcotc commented Jun 30, 2021 •

edited

codecov-commenter commented Jun 30, 2021 •

edited

marcotc commented Jul 1, 2021

ivoanjo left a comment

ivoanjo Jul 5, 2021

marcotc Jul 5, 2021

marcotc Jul 23, 2021

ivoanjo Jul 26, 2021

marcotc Jul 26, 2021

ivoanjo Jul 27, 2021

ivoanjo Jul 5, 2021

marcotc Jul 23, 2021

ivoanjo Jul 5, 2021

ivoanjo Jul 5, 2021

marcotc Jul 23, 2021

marcotc commented Jul 23, 2021

ivoanjo left a comment

ivoanjo Jul 26, 2021

Contrib:Make registry a global constant repository #1572

Contrib:Make registry a global constant repository #1572

Conversation

marcotc commented Jun 30, 2021 • edited

Changelog text

PR description

codecov-commenter commented Jun 30, 2021 • edited

Codecov Report

marcotc commented Jul 1, 2021

ivoanjo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marcotc commented Jul 23, 2021

ivoanjo left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

marcotc commented Jun 30, 2021 •

edited

codecov-commenter commented Jun 30, 2021 •

edited