Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ddtrace/tracer: add support for agent discovery endpoint, feature flags, stats & drops #859

Merged
merged 3 commits into from
Apr 28, 2021

Conversation

gbbr
Copy link
Contributor

@gbbr gbbr commented Mar 3, 2021

This change adds a multitude of new features, such as support for feature flags, agent discovery, client-side stats computations and enabling the dropping of P0 traces in the client.

Feature flags

Support for feature flags by means of:

  • WithFeatureFlags tracer start option.
  • DD_TRACE_FEATURES environment variable which takes a list of comma or space separated flag tokens.

Agent discovery

When the feature flag discovery is set, support for detecting features by means of the new agent (7.27.0) discovery endpoint added in DataDog/datadog-agent#7344 and DataDog/datadog-agent#7495 becomes enabled.

Client-computed stats and dropping p0's

When discovery is enabled, if the agent supports client-computed stats and dropping p0's in the client, the tracer can now do this by means of the newly added concentrator and drop logic.

TODO

  • Obfuscation is not yet supported in the client but will be added in a subsequent PR by factoring out the obfuscator code from the agent into a separate package to be shared with this module.

@gbbr gbbr added this to the Triage milestone Mar 3, 2021
@gbbr gbbr force-pushed the gbbr/discovery branch 2 times, most recently from 546b793 to 64a53f0 Compare March 3, 2021 12:14
@richardstartin
Copy link
Member

Don't you need to implement both obfuscation and metrics as a prerequisite of this change?

@gbbr gbbr force-pushed the gbbr/discovery branch 3 times, most recently from ba120e6 to c91a8af Compare March 16, 2021 10:05
@gbbr gbbr requested a review from piochelepiotr March 16, 2021 10:07
@gbbr gbbr force-pushed the gbbr/discovery branch 2 times, most recently from f5d3f01 to 59f6055 Compare March 16, 2021 10:09
ddtrace/tracer/stats.go Outdated Show resolved Hide resolved
ddtrace/tracer/span.go Outdated Show resolved Hide resolved
ddtrace/tracer/span.go Outdated Show resolved Hide resolved
ddtrace/tracer/span.go Outdated Show resolved Hide resolved
ddtrace/tracer/span.go Outdated Show resolved Hide resolved
ddtrace/tracer/span.go Outdated Show resolved Hide resolved
ddtrace/tracer/stats.go Outdated Show resolved Hide resolved
ddtrace/tracer/stats.go Outdated Show resolved Hide resolved
@gbbr
Copy link
Contributor Author

gbbr commented Mar 25, 2021

@richardstartin I think I got it now. Won't be doing obfuscation though. At least not at this point...

ddtrace/tracer/span.go Show resolved Hide resolved
ddtrace/tracer/stats.go Outdated Show resolved Hide resolved
ddtrace/tracer/tracer.go Outdated Show resolved Hide resolved
ddtrace/tracer/tracer.go Outdated Show resolved Hide resolved
ddtrace/tracer/tracer.go Outdated Show resolved Hide resolved
ddtrace/tracer/transport.go Outdated Show resolved Hide resolved
ddtrace/tracer/span.go Show resolved Hide resolved
ddtrace/tracer/span.go Show resolved Hide resolved
ddtrace/tracer/span.go Outdated Show resolved Hide resolved
ddtrace/tracer/span.go Outdated Show resolved Hide resolved
@gbbr gbbr force-pushed the gbbr/discovery branch 5 times, most recently from b3095d5 to 63c16b6 Compare March 30, 2021 13:43
@gbbr gbbr force-pushed the gbbr/discovery branch 4 times, most recently from a5d6c4c to ce324c7 Compare April 20, 2021 08:40
@gbbr gbbr force-pushed the gbbr/discovery branch 4 times, most recently from 45173c8 to 4be4428 Compare April 26, 2021 09:38
@gbbr gbbr force-pushed the gbbr/discovery branch 3 times, most recently from d7baedd to d95e0ce Compare April 26, 2021 13:42
@gbbr gbbr marked this pull request as ready for review April 26, 2021 13:50
@gbbr gbbr modified the milestones: Triage, 1.31.0 Apr 27, 2021
@gbbr gbbr changed the title ddtrace/tracer: add agent /info discovery and drop p0's ddtrace/tracer: add support for agent discovery endpoint, feature flags, stats & drops Apr 27, 2021
piochelepiotr
piochelepiotr previously approved these changes Apr 27, 2021
// This product includes software developed at Datadog (https://www.datadoghq.com/).
// Copyright 2016 Datadog, Inc.

//go:generate msgp -unexported -marshal=false -o=stats_msgp.go -tests=false
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe we don't use msgp on this file right?

}

// nsTimestampToFloat converts a nanosec timestamp into a float nanosecond timestamp truncated to a fixed precision
func nsTimestampToFloat(ns int64) float64 {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not divide by 1024 and multiply again?
Division by power of two should be really efficient.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I copied this from the agent. I don't know what you mean. You'd have to spell (code) it out 🙂

if err != nil {
return err
}
resp, err := t.client.Do(req)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to add retries here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe? Might be a good idea. If we do, we should do it in a separate PR for both traces and stats. We currently don't have it for traces either.

raphaelgavache
raphaelgavache previously approved these changes Apr 27, 2021
ddtrace/tracer/span.go Outdated Show resolved Hide resolved
@gbbr gbbr merged commit b74a5df into v1 Apr 28, 2021
@gbbr gbbr deleted the gbbr/discovery branch April 28, 2021 08:46
nicoledanuwidjaja pushed a commit that referenced this pull request May 5, 2021
…gs, stats & drops (#859)

This change adds a multitude of new features, such as support for feature flags, agent discovery, client-side stats computations and enabling the dropping of P0 traces in the client.

### Feature flags

Support for feature flags by means of:
* `WithFeatureFlags` tracer start option.
* `DD_TRACE_FEATURES` environment variable which takes a list of comma or space separated flag tokens.

### Agent discovery

When the feature flag `discovery` is set, support for detecting features by means of the new agent (7.27.0) discovery endpoint added in DataDog/datadog-agent#7344 and DataDog/datadog-agent#7495 becomes enabled.

### Client-computed stats and dropping p0's

When `discovery` is enabled, if the agent supports client-computed stats and dropping p0's in the client, the tracer can now do this by means of the newly added concentrator and drop logic.

### TODO

* Obfuscation is not yet supported in the client but will be added in a subsequent PR by factoring out the obfuscator code from the agent into a separate package to be shared with this module.
stlimtat pushed a commit to stlimtat/dd-trace-go that referenced this pull request May 17, 2021
* 'v1' of https://github.com/DataDog/dd-trace-go:
  Implement DD_PROFILING_OUTPUT_DIR for dev/debug (DataDog#924)
  contrib/go-pg/pg.v10: add INTEGRATION flag check for tests (DataDog#921)
  contrib/go-redis/redis.v8: optimize BeforeProcess and BeforeProcessPipeline (DataDog#920)
  CI: check go.sum is up-to-date (DataDog#918)
  README: Fix go get instructions (DataDog#913)
  internal/log: Improve API for testing (DataDog#916)
  ddtrace/tracer: add support for agent discovery endpoint, feature flags, stats & drops (DataDog#859)
  internal/version: bump to v1.31.0 (DataDog#910)
  CONTRIBUTING.md: add link to contrib/README.md (DataDog#802)
  profiler: Disable agentless mode for WithAPIKey (DataDog#906)
  contrib/Shopify/sarama: fix possible deadlock in WrapAsyncProducer (DataDog#907)
  contrib/database/sql.tracedConn implement driver.SessionResetter (DataDog#908)
  ddtrace/opentracer: consider FollowsFrom references as children (DataDog#905)
  ddtrace/opentracer: add support for opentracing.TracerContextWithSpanExtension (DataDog#855)
  ddtrace/tracer: follow noDebugStack setting when using SetTag with an error (DataDog#900)
  contrib/net/http: add a getter to retrieve original round tripper (DataDog#903)
  ddtrace/tracer: ensure Flush call is synchronous (DataDog#901)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants