Datadog tracer logs are not json formatted #10208
Description
Welcome!
- Yes, I've searched similar issues on GitHub and didn't find any.
- Yes, I've searched similar issues on the Traefik community forum and didn't find any.
What did you do?
Hello, I've been investigating an issue we initially spotted in our Fluentd log collector and traced it down to usage of the datadog tracer in Traefik.
We have Traefik configured in our kubernetes as the ingress controller, and in this cluster we are also running datadog.
In order to get traces from Traefik to Datadog, we enabled the traces in Traefik by adding the following to the traefik container args:
- --tracing.datadog=true
- --tracing.datadog.localAgentHostPort=$(DD_AGENT_HOST):8126
- --tracing.datadog.prioritySampling=true
And we set the DD_AGENT_HOST environment variable:
env:
- name: DD_ENTITY_ID
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.uid
- name: DD_AGENT_HOST
value: datadog.datadog.svc.cluster.local
In Traefik, we use json logging format:
- --log.format=json
However, we're noticing some log parsing errors in Fluentd due to logs coming from Traefik which aren't json-formatted.
On investigation, we found that traefik is outputting messages which are initially created by the datadog tracer. So I opened an issue with their team to ask for a fix and from what I understood in the response, Traefik simply needs to provide its json logger to the tracer in order to make the logs json formatted.
Could this be implemented wherever is appropriate in order to ensure that logs output from external tooling are json formatted?
What did you see instead?
Log messages which are not json formatted
What version of Traefik are you using?
2.10.4
What is your environment & configuration?
# (paste your configuration here)Add more configuration information here.
If applicable, please paste the log output in DEBUG level
No response