Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor:Move distributed propagation to Contrib #2352

Merged
merged 56 commits into from
Nov 22, 2022
Merged

Conversation

marcotc
Copy link
Member

@marcotc marcotc commented Nov 4, 2022

The Datadog::Tracing::Distributed namespace lives in the core Datadog::Tracing namespace, but a lot of its logic is propagation style-specific. We have two propagation styles, based on different protocols: HTTP Headers and gRPC metadata.

The current implementation contains a lot of complex, duplicated logic between these two styles. Also, some features from the HTTP headers implementation are missing from the gRPC version because it's easy to forget to keep them in sync.

This PR moves style-specific log from the Datadog::Tracing::Distributed namespace to Datadog::Tracing::Contrib.

This is simply a large refactor: there are no changes to externally visible behavior.
Tests remain the same, and passing, with coverage increased in a few cases.

Affected modules that are part of the public API, Datadog::Tracing::Distributed::Headers::Ext and Datadog::Tracing::Propagation::HTTP, have a compatibility layer added to this PR to ensure there are no breaking changes.

@marcotc marcotc self-assigned this Nov 4, 2022
@marcotc marcotc added the dev/refactor Involves refactoring existing components label Nov 4, 2022
Base automatically changed from deprecated-tags to feat-Sampling-Propagation November 7, 2022 21:19
@github-actions github-actions bot added appsec Application Security monitoring product integrations Involves tracing integrations tracing labels Nov 8, 2022
@github-actions github-actions bot removed the appsec Application Security monitoring product label Nov 8, 2022
Base automatically changed from feat-Sampling-Propagation to master November 8, 2022 20:38
@github-actions github-actions bot added the core Involves Datadog core libraries label Nov 10, 2022
Comment on lines -22 to -23
require_relative '../../propagation/grpc'
require_relative 'datadog_interceptor'
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These were just unnecessary and actually the responsibility of intercept_with_datadog.rb.

.with(Datadog::Tracing::Distributed::Headers::Ext::HTTP_HEADER_TRACE_ID, trace_id.to_s)
.with('x-datadog-trace-id', trace_id.to_s)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of fixing the namespace changes, I chose to inline the string values here: it makes tests more robust and easier to read.

@marcotc marcotc marked this pull request as ready for review November 10, 2022 02:08
@marcotc
Copy link
Member Author

marcotc commented Nov 15, 2022

spec/datadog/tracing/metadata_spec.rb is now reverted, it was indeed a mistake! 🙇

@github-actions github-actions bot removed the core Involves Datadog core libraries label Nov 16, 2022
Copy link
Member

@ivoanjo ivoanjo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did a final pass, this is shaping up really great! I was about to press the Approve button but noticed CI is badly broken so I'll hold on for after that ;)

lib/datadog/tracing/distributed/datadog.rb Outdated Show resolved Hide resolved
# HTTP headers one should set for distributed tracing.
# These are cross-language (eg: Python, Go and other implementations should honor these)
# DEV-2.0: This module only exists for backwards compatibility with the public API. It should be removed.
# @deprecated use [Datadog::Tracing::Distributed::Ext]
# @public_api
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: Would it make sense to remove the @public_api now that we decided to deprecate this module? We wouldn't break any code, and would further signal that you shouldn't be looking in here.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can't, as it will completely hide it from our public API docs and that's technically a breaking change.

Best I can do is a stern comment for now :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... and that's technically a breaking change.

Is it though? I'd claim a breaking change is one where you upgrade from version N to version N+1 and your existing code stops working.

In this case, your code wouldn't stop working. And it's not like the lib isn't opensource, and we don't ship the source to customers, so they can always still reference it; we just don't present it in the docs.

(I'm fine with not changing it, just saying that I don't think it's that much of a problem if we did! :P)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marcotc @ivoanjo

I'm pretty sure y'all implemented a breaking change anyway.

I've got a bot PR that broke trying to Bump [ddtrace](https://github.com/DataDog/dd-trace-rb) from 1.6.1 to 1.7.0. because NameError: uninitialized constant Datadog::Tracing::Distributed::Headers

Also, I could be wrong, but I don't think Datadog::Tracing::Distributed::Ext exists.

If it does, I'd appreciate a reference to it. ^^

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for the delay in answering @Judahmeek.

Most modules under Datadog::Tracing::Distributed::Headers are currently not considered part of the public API, as they are not tagged with @public_api.

Nevertheless, we have broken your setup, which is definitely not what we intended!

Let us know if we can help you move to 1.7.0+, and also, if you can, can you share how you're using these modules so we can hopefully take that into consideration for future changes? That will help us evaluate the current API and if there's something we're currently not exposing and that we should.

lib/datadog/tracing/distributed/propagation.rb Outdated Show resolved Hide resolved
Comment on lines +104 to +105
unless trace_digest.trace_id == extracted_trace_digest.trace_id \
&& trace_digest.span_id == extracted_trace_digest.span_id
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we also look to see if non-datadog digests match the datadog digest? This was not in the code prior to the refactoring either, but I'm curious if it would make sense.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm making larger changes to this logic, including how we prioritize the Datadog digest over others in a follow up PR. I'd rather touch this part there. I believe this logic will change quite a bit.

spec/datadog/opentracer/text_map_propagator_spec.rb Outdated Show resolved Hide resolved
@marcotc marcotc requested a review from ivoanjo November 16, 2022 23:11
Copy link
Member

@ivoanjo ivoanjo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great refactoring! LGTM 👍

@@ -73,12 +73,10 @@
context 'when value is' do
[
[nil, nil],
['not a number', nil],
['1.to_i returns 1', nil],
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: I'm not entirely sure it's clear what we're testing here (and to be fair, I think this was part of the problem of the old test -- it was actually not clear what it was testing).

I suggest perhaps ['1 2 3', nil], # '1 2 3'.to_i => 1 and we have extra logic to avoid this.

(Also applies below)

@marcotc marcotc merged commit e101e19 into master Nov 22, 2022
@marcotc marcotc deleted the refactor-propagation-2 branch November 22, 2022 23:43
@github-actions github-actions bot added this to the 1.7.0 milestone Nov 22, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
dev/refactor Involves refactoring existing components integrations Involves tracing integrations tracing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants