internal/telemetry: enable instrumentation telemetry in the tracer and profiler #1769
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…metry on by default for profiler
lievan
changed the title
3 tasks
nsrip-dd
reviewed
Mar 24, 2023
ajgajg1134
previously approved these changes
Mar 24, 2023
Co-authored-by: Andrew Glaude <andrew.glaude@datadoghq.com>
ajgajg1134
approved these changes
Mar 24, 2023
nsrip-dd
approved these changes
Mar 24, 2023
3 tasks
nsrip-dd
pushed a commit
that referenced
this pull request
Mar 30, 2023
…d profiler (#1769) Send instrumentation telemetry data from the tracer and profiler. Only the tracer can start the instrumentation telemetry client on tracer.Start. If the profiler attempts to send telemetry info before the tracer has started, no info is sent. Nothing telemetry-related happens when the profiler stops. * start telemetry client when tracer starts * enable telemetry by default and refactor transport unit tests * fixed race condition in transport tests * add new telemetry function with starter options to telemetry package * add options file * find API key when agentless and disable telemetry for clean stop test * remove telemetry logs in tracer log tests * call telemetry.NewClient to init profiler telemetry client, turn telemetry on by default for profiler * change test telemetry enabled * lint * disable instrumentation telemetry for transport tests to prevent flakiness * remove non-valid config values * change config to single test value * send empty payload * add service mappings and global tags to telemetry config * add product version and enabled details * add product field to newrequest * enabled field change to bool * required field of enabled * remove profiler from client products info * remove telemetry client from the profiler for now, keep it for the tracer * add sampling rules to config data, clarify sampling rule comments * retry sending telemetry with agentless endpoint given agent errors * update headers to v2 * update host spec to v2 * remove dependencies from app-started into seperate event * add support for dependency collection env var * remove lib fields from metrics * fix race on temp agentless bool * add products to app-started and fix errors in configuration schema * error should not be arr * lint * fix mismatched type err' * send start error data for telemetry * changed config key values to be consistent with telemetry backend normalization * del withstarterrors fn * user tracer log for instrumentation telemetry client: * fix lint and flaky tests * implement global telemetry client * fix race cond on temp agentless var * refactor telemetry starting logic, reset global telemetry client default values on tracer start * enable instrumentation telemetry for the profiler * fix hanging test case * fix timeout in profiler test, rememebr to stop telemetry * remove sending error telemetry information for now * refactor telemetry code in the tracer and profiler * remove flush timer and just flush during heartbeat interval * fix lint * remove the ability for the profiler to start telemetry client * dont send app-product-change if client hasn't started, update api version in headers * Update ddtrace/tracer/telemetry.go Co-authored-by: Andrew Glaude <andrew.glaude@datadoghq.com> * Update internal/telemetry/client.go Co-authored-by: Andrew Glaude <andrew.glaude@datadoghq.com> * Update internal/telemetry/message.go Co-authored-by: Andrew Glaude <andrew.glaude@datadoghq.com> * update comments and reformat * add api key to headers when retrying: * add agentlessRetry function for retry logic * switch statement for namespace selection when sending product data * git rid of black blox testing to speed up the tests, minimize possibility of ci timeout * fix lint * move around some functions for clarity * differentiate namespace for each metric * Update ddtrace/tracer/telemetry.go Co-authored-by: Katie Hockman <katie@hockman.dev> * Update internal/telemetry/client.go Co-authored-by: Katie Hockman <katie@hockman.dev> * Update internal/telemetry/client.go Co-authored-by: Katie Hockman <katie@hockman.dev> * Update internal/telemetry/client.go Co-authored-by: Katie Hockman <katie@hockman.dev> * Update internal/telemetry/option.go Co-authored-by: Katie Hockman <katie@hockman.dev> * Update internal/telemetry/client.go Co-authored-by: Katie Hockman <katie@hockman.dev> * export Disabled fn for the purpose of checking whether instrumentation telemetry is disabled * log at info level * rename submit fn and add comments describing functionality * add kernel architecture * dont stop telemetry client when profiler stops * move all env var config logic into Start * take away responsibility of disabling the client from With.. functions * removed unused field * add an events file and move all events-related code to the file * rename events file to telemetry * send produc change on profiler stop * lint * ignore telemetry logs * set api key manually for testing * no need to check for api key in applydefaultops * api key validation not our job i think * readd checking fro agentless for setting api key * update with latest telemetry changes * fix lint * resolve missing telemetry when start is called multiple times without stop * Update ddtrace/tracer/tracer_test.go Co-authored-by: Andrew Glaude <andrew.glaude@datadoghq.com> * add note about telemetry when tracing is disabled --------- Co-authored-by: Andrew Glaude <andrew.glaude@datadoghq.com> Co-authored-by: Katie Hockman <katie@hockman.dev>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR is meant to enable telemetry by default in the tracer and profiler. It is a "minimal version" of adding instrumentation telemetry data, because:
tracer.Start. If the profiler attempts to send telemetry info before the tracer has started, no info is sent.The important file changes are in:
Profiler:
profiler/profiler.goprofiler/telemetry.goprofiler.telemetry_test.goTracer:
ddtrace/tracer/tracer.goddtrace/tracer/telemetry.goddtrace/tracer/telemetry_test.goThe rest of the changes are in the tracer test files, since telemetry logs caused some logging-related unit tests to fail
Motivation
Describe how to test/QA your changes
Reviewer's Checklist
Triagemilestone is set.