Kuma Logs #20597

nubtron · 2025-06-26T15:30:53Z

What does this PR do?

This PR adds logs pipelines and saved views for the Kuma integration.

Motivation

This is a step in the creation of the Kuma integration.

Review checklist (to be filled by reviewers)

Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
Add the qa/skip-qa label if the PR doesn't need to be tested during QA.
If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

Review from michaelcretzman is dismissed. Related teams and files:

documentation
- kuma/assets/saved_views/logs_overview.json

Review from michaelcretzman is dismissed. Related teams and files:

documentation
- kuma/manifest.json

Review from steveny91 is dismissed. Related teams and files:

agent-integrations
- kuma/manifest.json

audesikorav · 2025-07-03T14:44:46Z

kuma/assets/logs/kuma.yaml

+          maybeInt     %{regex("(?:-|\\d+)" )}
+          notBackslash %{regex("[^\\\\]*")}
+          notColon     %{regex("[^:]*")}
+          notClosingParens %{regex("[^\\)]*")}


if you expect your services to be always named / formatted this way (e.g edge-gateway_kuma-demo_svc, redis_kuma-demo_svc_6379, etc), I'll suggested to scope characters that can be matched with a rule like this
serviceName %{regex("[a-zA-Z0-9_-]*")}.

Regex will run faster and it will prevent unnecessary backtracking on unexpected content vs using negated class

@audesikorav thanks for the tip! Avoided using negated classes when possible. I kept the negated class for URLs since the alternative seemed pretty complex, and I wasn't sure about the performance impact. Let me know what you think!

audesikorav · 2025-07-03T14:45:02Z

kuma/assets/logs/kuma.yaml

+          notQuote     %{regex("[^\"]*")}
+          maybeInt     %{regex("(?:-|\\d+)" )}
+          notBackslash %{regex("[^\\\\]*")}
+          notColon     %{regex("[^:]*")}


You can remove this one (not used)

Right, removed!

audesikorav · 2025-07-03T14:54:52Z

kuma/assets/logs/kuma.yaml

+          # Sample:
+          # [2025-06-26T21:29:21.535Z] - default 10.42.1.5(unknown)->10.42.1.5:6379(redis_kuma-demo_svc_6379) took 14004ms, sent 6195 bytes, received: 194 bytes
+
+          kuma_dp_tcp_log \[%{isoTimestamp:date}\] %{notSpace:response.flags} %{notSpace:kuma.mesh} %{ipOrHost:kuma.source_address_without_port}\(%{notClosingParens:kuma.source_service}\)\->%{notBackslash:kuma.upstream.host}\(%{notClosingParens:kuma.destination_service}\) took %{integer:duration:scale(1000000)}ms, sent %{integer:network.bytes_written} bytes, received: %{integer:network.bytes_read} bytes


same for here, if you can replace the negated character and matching only expected characters

Replaced the negated characters, except where I wasn't sure about the format (kuma.mesh) or the regex would have been relatively complex (url).

yep not worry, if you do the format its always best matching less char but its fine if not sure 👍

@audesikorav I saw an instance where the docs appear to contradict their own examples for the format of kuma.upstream.host, I'm not so sure about the format anymore, I'll go back to the negated class for that one to be safe.

audesikorav · 2025-07-03T14:56:55Z

kuma/assets/logs/kuma_tests.yaml

+        mesh: "default"
+        source_address_without_port: "10.42.3.27"
+        source_service: "unknown"
+        trace_id: "-"


are you sure this field is populated as you expected ?

According to the Kuma source code these are either Datadog or zipkin trace IDs. Their macro is '"%REQ(X-B3-TRACEID?X-DATADOG-TRACEID)%"'. Datadog trace IDs are ints if I understand correctly, and zipkin's traces are hex.

I switched to "(unknown|)" or "(-|)" to leave out the placeholder patterns used when no value is available.

audesikorav · 2025-07-03T14:57:06Z

kuma/assets/logs/kuma_tests.yaml

+        flags: "NR"
+        x_envoy_upstream_service_time: "-"
+      upstream:
+        host: "-"


same question here

The docs and my testing indicate <host>:<port>. However an example in the docs showed tcp://<host>:<port>. For safety I went back to "notQuote" since I'm not sure the docs are 100% up-to-date.

Oh, and I added "(-|)" to discard placeholders!

* Avoid using negated classes in regexes when possible (except for urls, for which it doesn't seem to be standard practice) * Use rules with alternatives such as "(-|<ITEM>)" or "(unknown|<ITEM>)" to discard placeholder values for empty fields. * Delete unused support rules

… about the format.

Review from audesikorav is dismissed. Related teams and files:

logs-backend
- kuma/assets/logs/kuma.yaml
- kuma/assets/logs/kuma_tests.yaml
logs-integrations-reviewers
- kuma/assets/logs/kuma.yaml
- kuma/assets/logs/kuma_tests.yaml

nubtron · 2025-07-04T09:17:47Z

@audesikorav need a moment to update the rules, will tell you when ready again!

nubtron · 2025-07-04T09:32:24Z

@audesikorav rules updated!

audesikorav

One nit comment but otherwise seems good on our side !

audesikorav · 2025-07-04T10:10:11Z

kuma/assets/logs/kuma.yaml

+      enabled: true
+      sources:
+        - date
+    - type: service-remapper


are you sure you want to use the source ("kuma") as the official service name ?
(not a blocker if you do, just since you are extracting some service name / maybe the service is already adding service as attribute)

@audesikorav I'm not entirely sure about this one! The reasoning is to have a fallback in case the Agent doesn't set it. But from what I've seen the Agent already sets it to the kubernetes service which is ideal.

I picked the source because I didn't see way to get a service in all variations of the logs. Let me know what you think!

mmh if its already defined by the agent and you want to keep this value then you should remove it, otherwise it will get override by the remapper

@audesikorav Removed it! Thanks for the input!

Review from audesikorav is dismissed. Related teams and files:

logs-backend
- kuma/assets/logs/kuma.yaml
logs-integrations-reviewers
- kuma/assets/logs/kuma.yaml

Add assets for Kuma logs.

6a07669

datadog-agent-integrations-bot bot added the integration/kuma label Jun 26, 2025

nubtron added 4 commits June 27, 2025 15:10

Update Kuma logs.

10c0dc7

Set app IDs.

3284de7

Remove extra line from test.

56e39f1

Add logs to manifest.json.

d769407

datadog-agent-integrations-bot bot added the documentation label Jun 27, 2025

nubtron added 7 commits June 27, 2025 15:38

Add saved views.

8277d99

Add test results.

fa92923

Reformat tests yaml

4981e47

Attempt fixing yaml parsing issues.

cf4b9c8

Improve formatting.

e069293

Set "kuma" as the pipeline source.

c128150

Add standard attributes to facets.

b41fc2f

nubtron added qa/skip-qa Automatically skip this PR for the next QA changelog/no-changelog labels Jun 27, 2025

michaelcretzman previously approved these changes Jun 27, 2025

View reviewed changes

nubtron marked this pull request as ready for review June 27, 2025 16:57

nubtron requested review from a team as code owners June 27, 2025 16:57

datadog-agent-integrations-bot bot added team/agent-integrations team/documentation team/logs-backend team/logs-integrations-reviewers labels Jun 27, 2025

Remove kuma-dp source from saved view.

b5c300d

Remove columns from saved view.

c0b95db

michaelcretzman previously approved these changes Jun 27, 2025

View reviewed changes

steveny91 previously approved these changes Jun 27, 2025

View reviewed changes

nubtron added 2 commits June 30, 2025 13:40

Remove the service remapper - it doesn't seem to be needed.

e5e9ce3

Merge branch 'master' into nubtron/kuma-logs

09b171b

nubtron added assets/no-deploy Prevents APW from deploying this PR in staging and removed assets/no-deploy Prevents APW from deploying this PR in staging labels Jul 2, 2025

audesikorav reviewed Jul 3, 2025

View reviewed changes

nubtron added 6 commits July 4, 2025 09:26

Restore the service remapper.

ab7429a

Update tests.

879c84a

Add authority helper rule.

cd7fc00

Update test results

e2fc5a9

Remove dash for trace id.

1a150bc

audesikorav previously approved these changes Jul 4, 2025

View reviewed changes

audesikorav added the assets/deploy-logs-staging ONLY USED BY Logs Backend - Validates that a PR is OK to go to staging label Jul 4, 2025

Go back to negated class for kuma.upstream.host since I'm not certain…

48db9f1

… about the format.

Use "notOpeningParens" for kuma_dp_tcp_log upstream host.

b8394df

audesikorav previously approved these changes Jul 4, 2025

View reviewed changes

Remove service remapper.

4d259cc

Kuma Logs #20597

Are you sure you want to change the base?

Kuma Logs #20597

Conversation

nubtron commented Jun 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Review checklist (to be filled by reviewers)

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nubtron Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nubtron Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nubtron Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nubtron commented Jul 4, 2025

Uh oh!

nubtron commented Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

audesikorav left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

nubtron commented Jun 26, 2025 •

edited

Loading

nubtron Jul 4, 2025 •

edited

Loading

nubtron Jul 4, 2025 •

edited

Loading

nubtron Jul 4, 2025 •

edited

Loading

nubtron commented Jul 4, 2025 •

edited

Loading