Contextual structured logging

[paullaffitte]

/kind feature

Describe the solution you'd like
It would be nice to be able to use a context for structured logging. Something like described in this article. Here is a short example from the article:

ctx := context.Background()
log.Infof(ctx, "test")    // Message: “test”
ctx = log.WithLogTag(ctx, "n", 1)
log.Infof(ctx, "test")    // Message: “[n1] test”
ctx = log.WithLogTag(ctx, "s", 2)
log.Infof(ctx, "test")    // Message: “[n1,s2] test”
ctx = log.WithLogTag(ctx, "bootstrap", nil)
log.Infof(ctx, "test")    // Message: “[n1,s2,bootstrap] test”

Something similar could be implemented in order to populate structured logs, example inspired from kubenetes introduction to structured logs:

ctx := context.Background()
ctx = klog.WithLogTag(ctx, "pod", klog.KObj(pod))
ctx = klog.WithLogTag(ctx, "status", status)
klog.WithContext(ctx).InfoS("Pod status updated")

Output:

I1025 00:15:15.525108       1 controller_utils.go:116] "Pod status updated" pod="kube-system/kubedns" status="ready"

Anything else you would like to add:

The form klog.WithContext(ctx).InfoS("") would help to avoid breaking changes over klog.InfoS(ctx, ""). Moreover, introducing new functions like klog.ContextualInfoS risks to bloat the api. These are the reasons which make me think that introducing a single new function klog.WithContext would be rather a good solution.

This article provides some good reasons to use context in logging.

[serathius]

/wg structured-logging

[serathius]

Thanks for creating issue, contextual logging is a great idea!
It should be something in scope of newly created Structured Logging Working Group.

I have added it to WG Roadmap
/assign

As for api design, I think wait for Kubernetes to migrate to https://github.com/go-logr/logr as it already supports contextual logging. Example:

func external() {
  ctx := logr.NewContext(context.Background(), baseLogger)
  internal(ctx)
}

func internal(ctx context.Context) {
  lg := logr.FromContext(ctx).WithValues("pod", klog.KObj(pod))
  lg.InfoS("Pod status updated")
}

[yuzhiquan]

/cc
And i meet this context logging need in my current work, interesting feature.

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle stale
• Mark this issue or PR as rotten with /lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[k8s-triage-robot]

The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs.

This bot triages issues and PRs according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue or PR as fresh with /remove-lifecycle rotten
• Close this issue or PR with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle rotten

[serathius]

/remove-lifecycle rotten
/cc @pohly

[pohly]

I'm working on a KEP for this. The logr API already supports this, we "just" need to figure out how to use that in Kubernetes.

/assign

[pohly]

Here's the KEP: kubernetes/enhancements#3078

[pohly]

/assign

The KEP was approved. I'll work on this.

[pohly]

It's worth pointing out that the solution will not be exactly as in the description. Have a look at the KEP for the proposed API.

In contrast to what is currently described in the KEP, we'll probably just add FromContext to klog itself and not introduce a separate klogr.

[pohly]

Support for contextual logging as I described it in the KEP (logger is part of the context, not "logger uses context") has been implemented in klog and Kubernetes 1.24. I consider that the solution for this issue.

/close

[k8s-ci-robot]

@pohly: Closing this issue.

In response to this:

Support for contextual logging as I described it in the KEP (logger is part of the context, not "logger uses context") has been implemented in klog and Kubernetes 1.24. I consider that the solution for this issue.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.