Kubelet OpenTelemetry Tracing

[sallyom]

Enhancement Description

• One-line enhancement description (can be used as a release note): Instrument kubelet to generate and export OpenTelemetry trace data.

• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/blob/master/keps/sig-instrumentation/2831-kubelet-tracing/README.md

• Discussion Link:

• Primary contacts (assignee): @somalley, @dashpole

• Responsible SIGs: SIG-Instrumentation

• Enhancement target (which target equals to which milestone):

  • Alpha release target (1.25)
  • Beta release target (1.27)
  • Stable release target (1.29)

• Alpha

  • KEP (k/enhancements) update PR(s):
    • KEP-2831: Instrumenting Kubelet for OpenTelemetry Tracing #2832
  • Code (k/k) update PR(s):
    • kubelet tracing instrumentation kubernetes#105126
  • Docs (k/website) update PR(s):
    • Add docs for KubeletTracing feature gate 1.25 website#35137

• Beta

  • KEP (k/enhancements) update PR(s):
    • KEP-2831: adding beta graduation criteria #3714
  • Code (k/k) update PR(s):
    • kubelet: create top-level traces for pod sync and GC kubernetes#114504
    • Graduate KubeletTracing feature to beta kubernetes#115750
  • Docs (k/website) update(s):
    • [KEP-2731] Add docs for Kubelet OpenTelemetry tracing graduation website#39904

• Stable

  • KEP (k/enhancements) update PR(s):
  • Code (k/k) update PR(s):
  • Docs (k/website) update(s):

Please keep this description up to date. This will help the Enhancement Team to track the evolution of the enhancement efficiently.

[kikisdeliveryservice]

@sallyom can you please add the mandatory discussion link to this issue?

[ehashman]

/sig node instrumentation

[logicalhan]

/assign

[shivanshuraj1333]

/cc

[yangjunmyfm192085]

/cc

[249043822]

/cc

[hosseinsalahi]

Hello @sallyom

v1.24 Enhancements team here.

Just checking in as we approach enhancements freeze on 18:00pm PT on Thursday Feb 3rd, 2022. This enhancement is targeting alpha for v1.24,

Here’s where this enhancement currently stands:

• Updated KEP file using the latest template has been merged into the k/enhancements repo.
  - KEP file PR is pending.
• KEP status is marked as implementable for this release
• KEP has a test plan section filled out.
• KEP has up to date graduation criteria.
• KEP has a production readiness review that has been completed and merged into k/enhancements.

The status of this enhancement is marked as tracked. Please keep the issue description up-to-date for release v1.24.
Thanks!

[sallyom]

@encodeflush thanks for the check-in - I'm not sure what the production readiness review is edit: found it, will add the questionnaire, I don't see it on the enhancements README can you point me to the steps to get that done? Thanks again. Also, am I meant to check those off as they are done or are they for someone else to decide?

[hosseinsalahi]

@sallyom Thanks for the update. Those criterion should be fulfilled form your side. From enhancements team point of view, we go through them and check if your KEP meets the requirements.

[chrisnegus]

Hi @sallyom 👋 1.24 Docs shadow here.

This enhancement is marked as 'Needs Docs' for the 1.24 release.

Please follow the steps detailed in the documentation to open a PR against the dev-1.24 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thu March 31, 11:59 PM PDT.

Also, if needed take a look at Documenting for a release to familiarize yourself with the docs requirement for the release.

Thanks!

[valaparthvi]

Hi @sallyom 👋 1.24 Release Comms team here.

We have an opt-in process for the feature blog delivery. If you would like to publish a feature blog for this issue in this cycle, then please opt in on this tracking sheet.

The deadline for submissions and the feature blog freeze is scheduled for 01:00 UTC Wednesday 23rd March 2022 / 18:00 PDT Tuesday 22nd March 2022. Other important dates for delivery and review are listed here: https://github.com/kubernetes/sig-release/tree/master/releases/release-1.24#timeline.

For reference, here is the blog for 1.23.

Please feel free to reach out any time to me or on the #release-comms channel with questions or comments.

Thanks!

[hosseinsalahi]

Hello @sallyom

I'm just checking in once more as we approach the 1.24 Code Freeze on 18:00 PDT, Tuesday, March 29th 2022

Please ensure the following items are completed:

• All PRs to the Kubernetes repo that are related to your enhancement are merged by the code freeze deadline.
• Have a documentation placeholder PR open by 18:00 PDT, Thursday, March 31, 2022.

For note, the status of this enhancement is currently marked as at risk.

Could you please confirm if following PR is part of the implementation for this enhancement?

• kubelet tracing instrumentation kubernetes#105126
• KEP-3085: Add KEP for Pod network ready condition #3087

Kindly please let me know if I'm missing any related PRs other than the ones I linked above. Thank you so much!

[ddebroy]

#3087 is not directly related to the overall OpenTelemetry enhancements.

[sallyom]

kubernetes/kubernetes#105126 is the initial implementation for this enhancement

[valaparthvi]

Hi @sallyom 👋 1.24 Release Comms team here.

We have an opt-in process for the feature blog delivery. If you would like to publish a feature blog for this issue in this cycle, then please opt in on this tracking sheet.

The deadline for submissions and the feature blog freeze is scheduled for 01:00 UTC Wednesday 23rd March 2022 / 18:00 PDT Tuesday 22nd March 2022. Other important dates for delivery and review are listed here: https://github.com/kubernetes/sig-release/tree/master/releases/release-1.24#timeline.

For reference, here is the blog for 1.23.

Please feel free to reach out any time to me or on the #release-comms channel with questions or comments.

Thanks!

Hey @sallyom, can you please add a placeholder PR for this feature in https://github.com/kubernetes/website by March 30, that is our extended deadline for feature blog freeze?

Example of a feature blog PR: kubernetes/website#30538
Example of a feature blog: https://github.com/kubernetes/website/blob/main/content/en/blog/_posts/2021-12-08-dual-stack-networking-ga.md

[gracenng]

Hi, 1.24 Enhancements Lead here 👋. With code freeze now in effect, this enhancement has not met the criteria for the freeze and has been removed from the milestone.

As a reminder, the criteria for code freeze is:

All PRs to the kubernetes/kubernetes repo have merged by the code freeze deadline
Feel free to file an exception to add this back to the release. If you plan to do so, please file this as early as possible.

Thanks!
/milestone clear

[katmutua]

Hello @sallyom 👋🏾 !

@katmutua 1.27 Release Docs shadow here. This enhancement is marked as ‘Needs Docs’ for 1.27 release.

Please follow the steps detailed in the documentation to open a PR against dev-1.27 branch in the k/website repo. This PR can be just a placeholder at this time, and must be created by March 16. For more information, please take a look at Documenting for a release to familiarize yourself with the documentation requirements for the release.

If you already have existing open PRs please link them to the description so we can easily track them. Thanks!

[saschagrunert]

Thank you @katmutua, the placeholder is now available in kubernetes/website#39904

[Atharva-Shinde]

Hey again @saschagrunert @vrutkovs 👋 Enhancements team here,
Just checking in as we approach 1.27 code freeze at 17:00 PDT on Tuesday 14th March 2023.

Here's where this enhancement currently stands:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).
• All PR/s are fully merged by the code freeze deadline.

For this enhancement, it looks like the following PRs are open and need to be merged before code freeze (and we need to update the Issue description to include all the related PRs of this KEP):

• kubelet: create top-level traces for pod sync and GC kubernetes#114504
• Graduate KubeletTracing feature to beta kubernetes#115750

Also please let me know if there are other PRs in k/k we should be tracking for this KEP.
As always, we are here to help if any questions come up. Thanks!

[SergeyKanzhelev]

@somalley, @dashpole do you want to GA in 1.28 as was planned in the issue description? It is not typical to do GA one release after beta as it typically doesn't give enough time to collect feedback.

[saschagrunert]

(I agree this should stay in beta for at least one more release)

[dashpole]

@saschagrunert Is GA planned for 1.30?

If it is, we need to add the 1.30 milestone, and the lead-opted-in label

[saschagrunert]

@dashpole not planned for 1.30, do we want to target it for 1.31?

[dashpole]

We definitely can, but we don't need to decide until the start of the 1.31 process. I'll check back in here then

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[knight42]

/remove-lifecycle stale

[saschagrunert]

We definitely can, but we don't need to decide until the start of the 1.31 process. I'll check back in here then

Is it still something we want to consider or have priorities shifted?

[k8s-triage-robot]

The Kubernetes project currently lacks enough contributors to adequately respond to all issues.

This bot triages un-triaged issues according to the following rules:

• After 90d of inactivity, lifecycle/stale is applied
• After 30d of inactivity since lifecycle/stale was applied, lifecycle/rotten is applied
• After 30d of inactivity since lifecycle/rotten was applied, the issue is closed

You can:

• Mark this issue as fresh with /remove-lifecycle stale
• Close this issue with /close
• Offer to help out with Issue Triage

Please send feedback to sig-contributor-experience at kubernetes/community.

/lifecycle stale

[saschagrunert]

/remove-lifecycle stale

[kannon92]

1.32 is coming up soon so please let us know if you want to graduate this to stable.

[haircommander]

Having not heard anything in two weeks, I've moved this as not planned for 1.32. Please comment if you disagree

[dashpole]

I might have time to graduate this (need to double-check the criterion), but we can add it to the milestone in the meantime.

[SergeyKanzhelev]

Moved to proposed so we can discuss who can be revioewer/approver from SIG Node side.