Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

profiler: use instrumentation telemetry #1348

Merged
merged 19 commits into from
Aug 4, 2022
Merged

Conversation

nsrip-dd
Copy link
Contributor

@nsrip-dd nsrip-dd commented Jun 17, 2022

Use the telemetry client from internal/telemetry for the profiler to report the profiler's environment (Go version, OS version, imported modules, etc.) and configuration (enabled profiles, profile rates, etc.) to the Datadog instrumentation telemetry backend.

This PR starts with the client off by default so it can be selectively enabled in our testing/staging environment. It must be explicitly enabled with the environment variable DD_INSTRUMENTATION_TELEMETRY_ENABLED=true.

Also, we no longer need to set the Type field of telemetry.Dependency since that field is no longer part of the telemetry API.

@nsrip-dd nsrip-dd added this to the Triage milestone Jun 17, 2022
profiler/profiler.go Outdated Show resolved Hide resolved
profiler/profiler.go Outdated Show resolved Hide resolved
@nsrip-dd nsrip-dd marked this pull request as ready for review June 23, 2022 14:57
@nsrip-dd nsrip-dd requested review from a team as code owners June 23, 2022 14:57
Copy link
Contributor

@ajgajg1134 ajgajg1134 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good! Just a few questions here

profiler/profiler.go Show resolved Hide resolved
profiler/profiler.go Show resolved Hide resolved
profiler/profiler.go Outdated Show resolved Hide resolved
The profiler sets agentURL to http://host:port/profiling/v1/input.
So doing path.Join on that to add the telemetry proxy endpoint is
wrong for two reasons: The "//" at the beginning will be converted
to a "/", and the endpoint will be wrong. We need to parse the
URL and replace the path with the telemetry proxy endpoint.
@katiehockman
Copy link
Contributor

katiehockman commented Jun 28, 2022

@nsrip-dd Can you update the PR description with more details? Additionally, should this be associated with a milestone?

Copy link
Contributor

@katiehockman katiehockman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can there be a test for this?

internal/telemetry/client.go Outdated Show resolved Hide resolved
internal/telemetry/client.go Show resolved Hide resolved
profiler/profiler.go Outdated Show resolved Hide resolved
In particular, explain what the default http.Client will be and why to
use a non-default http.Client
Copy link
Contributor Author

@nsrip-dd nsrip-dd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the review @katiehockman! I'll see if I can come up with a reasonable unit test for this change.

internal/telemetry/client.go Show resolved Hide resolved
internal/telemetry/client.go Outdated Show resolved Hide resolved
profiler/profiler.go Outdated Show resolved Hide resolved
Copy link
Contributor

@katiehockman katiehockman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah sorry, prematurely approved. Let's make sure we have a unit test first.

Copy link
Contributor

@katiehockman katiehockman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah sorry, prematurely approved. Let's make sure we have a unit test first.

katiehockman
katiehockman previously approved these changes Aug 3, 2022
Copy link
Contributor

@katiehockman katiehockman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Sorry for the delayed follow-up review. Didn't realize the unit tests were written and that it was ready. Just a few comments, but otherwise looks good!

profiler/profiler_test.go Show resolved Hide resolved
func TestTelemetryEnabled(t *testing.T) {
received := make(chan *telemetry.AppStarted, 1)
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/telemetry/proxy/api/v2/apmtelemetry" {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe add a test where the URL is invalid to make sure it finishes gracefully?

felixge
felixge previously approved these changes Aug 4, 2022
Copy link
Member

@felixge felixge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, see nit

profiler/profiler.go Show resolved Hide resolved
The debug feature (which causes the backend to emit more verbose
logging) is only intended for development. There's no point having it be
a public member of the telemetry Client since it can be enabled via an
environment variable for development.
@nsrip-dd nsrip-dd dismissed stale reviews from felixge and katiehockman via 8a2e08b August 4, 2022 12:54
Copy link
Member

@felixge felixge left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@nsrip-dd nsrip-dd merged commit 3251b9c into main Aug 4, 2022
@nsrip-dd nsrip-dd deleted the feature-profiler-telemetry branch August 4, 2022 17:18
@wczifrojc
Copy link

Hello, I have been trying to track down the release that this change is included in. I managed to locate it in this diff: v1.40.1...v1.41.0#diff-84cf63f4014172e12f686aac850adafb3669dd237fe2f64267bd20cc5e2b1860 , but the corresponding release notes doesn't mention it: https://github.com/DataDog/dd-trace-go/releases/tag/v1.41.0 . Was this by choice to not mention it? If not, it would be much appreciated to have the release notes amended, thanks!

@nsrip-dd
Copy link
Contributor Author

Hi! At the time of this PR, we hadn't actually enabled instrumentation telemetry collection by default. It had to be manually enabled via an environment variable, which we were only doing for internal testing. Since there weren't any user-facing changes related to this, it wasn't mentioned in the notes. We didn't actually revisit telemetry until later, in v1.49.0, and the implementation changed quite a bit. But at that point, it was documented. Was/is this code causing issues?

@wczifrojc
Copy link

Hi @nsrip-dd , the code in question is not causing issues. My team and I detected an attempt to connect to the host instrumentation-telemetry-intake.datadoghq.com by one of our services. We were trying to figure out which version of this library introduced this since we weren't seeing it in the rest of our services. Thank you for clarifying the matter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants