Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Provide a DogStatsD API in the sidecar #399

Merged
merged 22 commits into from
May 17, 2024
Merged

Conversation

iamluc
Copy link
Contributor

@iamluc iamluc commented Apr 22, 2024

What does this PR do?

Add a DogStatsD client in the sidecar, and expose an API to send metrics through it.

Motivation

Is part of the PHP tasks for Q2. It adds new possibilities for the integration team.

Additional Notes

The underlying StatsD client used by this PR is Cadence.
It is already used by some teams at Datadog, even if does not complies exactly to other DogStatsD implementations like the one in PHP or Go

For example, it lacks support of the "sample rate", or requires an integer for the Set metric, while Go and PHP can accept a string.

But I think there is nothing critical that blocks us.

How to test the change?

Describe here in detail how the change can be validated.

For Reviewers

  • If this PR touches code that signs or publishes builds or packages, or handles credentials of any kind, I've requested a review from @DataDog/security-design-and-guidance.
  • This PR doesn't touch any of that.

sidecar/src/dogstatsd.rs Outdated Show resolved Hide resolved
@iamluc iamluc force-pushed the lv/sidecar-dogstatsd-bridge branch 2 times, most recently from db1fdc9 to d4507c7 Compare April 25, 2024 12:40
@github-actions github-actions bot removed the common label Apr 25, 2024
@iamluc iamluc force-pushed the lv/sidecar-dogstatsd-bridge branch 5 times, most recently from 0a8b35b to 417dd3f Compare April 30, 2024 08:59
@iamluc iamluc marked this pull request as ready for review April 30, 2024 09:18
@iamluc iamluc requested review from a team as code owners April 30, 2024 09:18
@iamluc iamluc force-pushed the lv/sidecar-dogstatsd-bridge branch 2 times, most recently from a647b7b to 567bded Compare May 13, 2024 09:21
@codecov-commenter
Copy link

codecov-commenter commented May 13, 2024

Codecov Report

Attention: Patch coverage is 51.32450% with 147 lines in your changes are missing coverage. Please review.

Project coverage is 66.00%. Comparing base (1ecd94c) to head (6bcb3a0).

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #399      +/-   ##
==========================================
- Coverage   66.17%   66.00%   -0.17%     
==========================================
  Files         187      188       +1     
  Lines       23171    23470     +299     
==========================================
+ Hits        15334    15492     +158     
- Misses       7837     7978     +141     
Components Coverage Δ
crashtracker 19.34% <ø> (ø)
datadog-alloc 98.37% <ø> (ø)
data-pipeline 51.45% <ø> (ø)
data-pipeline-ffi 0.00% <ø> (ø)
ddcommon 81.52% <ø> (+0.29%) ⬆️
ddcommon-ffi 74.93% <ø> (ø)
ddtelemetry 53.72% <ø> (ø)
ipc 81.27% <ø> (ø)
profiling 76.83% <ø> (ø)
profiling-ffi 60.05% <ø> (ø)
serverless 0.00% <ø> (ø)
sidecar 31.75% <51.32%> (+2.07%) ⬆️
sidecar-ffi 0.00% <0.00%> (ø)
spawn-worker 54.98% <ø> (ø)
trace-mini-agent 69.12% <ø> (ø)
trace-normalization 97.79% <ø> (ø)
trace-obfuscation 95.74% <ø> (ø)
trace-protobuf 25.64% <ø> (ø)
trace-utils 68.85% <ø> (ø)

Copy link
Contributor

@bwoebi bwoebi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me :-)

Copy link
Contributor

@ekump ekump left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Warning - I know nothing about dogstatsd

Some non-blocking questions and style comments, but LGTM

sidecar/src/dogstatsd.rs Show resolved Hide resolved
sidecar/src/dogstatsd.rs Outdated Show resolved Hide resolved
sidecar/src/setup/unix.rs Show resolved Hide resolved
sidecar/src/dogstatsd.rs Show resolved Hide resolved
sidecar/src/service/session_info.rs Show resolved Hide resolved
sidecar/src/dogstatsd.rs Show resolved Hide resolved
Copy link
Contributor

@pierotibou pierotibou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had started the review yesterday so pushing those 2 questions anyway

@@ -334,6 +334,15 @@ dependencies = [
"serde",
]

[[package]]
name = "cadence"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know it's less of a problem in the sidecar than in the library but what's the size impact of this library?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean on libdatadog_profiling.{a,so}? The size is exactly the same

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It won't change the profiling lib. It will change the final package tracer size I assume as the sidecar is used by tracing only for now. Basically, our release artifacts are 500MB already, the profiling team starts looking into reducing package size, we should globally be very careful of the size impact of adding any new dependency.

Copy link
Contributor Author

@iamluc iamluc May 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The dd-trace-php PR implementing the DogStatsD API generated a package of 521 MB.

The last PR merged on master generated a package of 516 MB.

It's about a 1% increase

Copy link
Contributor

@pierotibou pierotibou May 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks. We support 12 versions in the package IIRC. So I assume we have an extra 400KB per version here. Not saying it comes from cadence only though. Anyway, this is something that we will need to be more and more careful about I believe even for the sidecar (not a blocker here I suppose).

instance_id: &InstanceId,
metric: ffi::CharSlice,
value: u64,
tags: Option<&ddcommon_ffi::Vec<Tag>>,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So we passa all the tags all the time? Some of them (e.g lang, lang_interpreter, lang_version, tracer_version, runtimeId won't change from one session to another.
Also, not saying we should do it, but we do process at least some tags in .NET (cf this), do you do it on php side?

Copy link
Contributor

@bwoebi bwoebi May 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we do that on the PHP side. The language/version/runtime id tags could be probably directly provided within the sidecar. PR #398 anyway transfers that language/version info as part of session info. We could just use that part and merge it here. Should be more efficient too :-)
Would make sense to me, great point, Pierre!

Copy link
Contributor

@pierotibou pierotibou May 15, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If processing happens on the client side we should document it Somewhere in the API that we expect input data to be already processed.

@iamluc iamluc force-pushed the lv/sidecar-dogstatsd-bridge branch from 567bded to d0c543a Compare May 16, 2024 07:34
@iamluc iamluc force-pushed the lv/sidecar-dogstatsd-bridge branch from d0c543a to 6bcb3a0 Compare May 17, 2024 06:45
@iamluc iamluc merged commit 0730f67 into main May 17, 2024
23 of 24 checks passed
@iamluc iamluc deleted the lv/sidecar-dogstatsd-bridge branch May 17, 2024 06:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants