cAdvisor-less, CRI-full Container and Pod Stats

[haircommander]

Enhancement Description

• One-line enhancement description (can be used as a release note): cAdvisor-less, CRI-full Container and Pod Stats
• Kubernetes Enhancement Proposal: https://github.com/kubernetes/enhancements/blob/master/keps/sig-node/2371-cri-pod-container-stats/README.md
• Primary contact (assignee): @haircommander, @bobbypage
• Responsible SIGs: sig-node
• Enhancement target (which target equals to which milestone):
  • Alpha release target (x.y): 1.26
  • Beta release target (x.y): 1.29
  • Stable release target (x.y): 1.30
• Alpha

  • KEP (k/enhancements) update PR(s):

    • KEP-2371: update kep to reflect current state of enhancement #2812
    • KEP-2371: Add cgroup metrics + CRI implementation plan #3559
    • Add Windows pod sandbox stats information to KEP 2371 - cri pod container stats #3439
    • CRI: Add Windows Podsandbox Stats kubernetes#110754
    • kubelet: add support for broadcasting metrics from CRI kubernetes#113609

  • Code (k/k) update PR(s):

    • Kubelet: implement support for podAndContainerStatsFromCRI kubernetes#103095
    • CRI: Add Windows Podsandbox Stats kubernetes#110754

  • Docs (k/website) update PR(s): Document PodAndContainerStatsFromCRI feature website#30535

• Beta
  • KEP (k/enhancements) update PR(s): KEP 2371: target to beta for 1.28 #3770
  • Code (k/k) update PR(s):
    • e2e_node: add container metrics from cadvisor test kubernetes#118838
    • kubelet: Implement support for Windows podAndContainerStatsFromCRI kubernetes#116968
  • Docs (k/website) update PR(s):

[haircommander]

/sig node

[ehashman]

/milestone v1.21

[JamesLaverack]

Hey @haircommander and @bobbypage, enhancements 1.21 shadow here,

Enhancements Freeze is 2 days away, Feb 9th EOD PST

The enhancements team is aware that KEP update is currently in progress (PR #2364). Please make sure to work on PRR questionnaires and requirements and get it merged before the freeze. For PRR related questions or to boost the PR for PRR review, please reach out in Slack on the #prod-readiness channel.

Any enhancements that do not complete the following requirements by the freeze will require an exception.

• [IN PROGRESS] The KEP must be merged in an implementable state
• [IN PROGRESS] The KEP must have test plans
• [IN PROGRESS] The KEP must have graduation criteria
• [IN PROGRESS] The KEP must have a production readiness review

[annajung]

Hi @haircommander and @bobbypage, 1.21 Enhancements Lead here.

Enhancements Freeze is now in effect.

Unfortunately, your KEP needed to be updated and the PR has not yet merged. If you wish to be included in the 1.21 Release, please submit an Exception Request as soon as possible.

/milestone clear

[ehashman]

/milestone v1.22

[salaxander]

Hey @haircommander and @bobbypage - 1.22 enhancements team here! Just a reminder that enhancements freeze is coming up on 5/13. We'll need the KEP merged before then to get this included in the 1.22 release.

Let us know if there's anything we can do to help before then!

[salaxander]

@haircommander @bobbypage #2364 merged so we've got you tracked for 1.22 :)

[PI-Victor]

Hello @haircommander @bobbypage 👋, 1.22 Docs release lead here.
This enhancement is marked as ‘Needs Docs’ for 1.22 release.

Please follow the steps detailed in the documentation to open a PR against dev-1.22 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Fri July 9, 11:59 PM PDT.
 Also, take a look at Documenting for a release to familiarize yourself with the docs requirement for the release.

Thank you!

[haircommander]

thanks for the heads up @PI-Victor !

[ehashman]

xref kubernetes/kubernetes#102789

[haircommander]

xref initial kubelet implementation kubernetes/kubernetes#103095

[salaxander]

Hey @bobbypage and @haircommander - Just checking in as we're about 2 weeks away from 1.22 code freeze. I've got kubernetes/kubernetes#103095 and kubernetes/kubernetes#103095 tracked as the open k/k PRs. Are there any other open or merged PRs we should be tracking? Thanks!

[haircommander]

none opened yet! I will post them here if we do. Thanks for your work @salaxander

[salaxander]

Hey @haircommander - One more check-in as we're a week out from 1.22 code freeze. Any updates on if you expect kubernetes/kubernetes#103095 and kubernetes/kubernetes#103095 to merge before the deadline?

Thanks!

[haircommander]

Thanks for checking! I do expect them to merge. We're waiting on a e2e POC which I'm working on :)

[salaxander]

Hi @haircommander - One last ping (sorry!). Code freeze is tomorrow evening (PST), so those two open PRs will need to merge before then for this to be included in 1.22. Let me know if there's anything we can do to help :)

[marosset]

My understanding is that Windows support is being added now. Applying the label to track it:

/milestone v1.28

Windows support is being added with kubernetes/kubernetes#116968
/sig windows

[SergeyKanzhelev]

/label lead-opted-in

[Kasambx]

Hello @haircommander 👋, Enhancements team here.

Just checking in as we approach enhancements freeze on 1:00 UTC on Friday 16th June 2023..

This enhancement is targeting for stage beta for v 1.28 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: v 1.28
• KEP readme has a updated detailed test plan section filled out
• KEP readme has up to date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements.

For this KEP, we would just need to update the following:

• Ensure [PR] (KEP 2371: target to beta for 1.28 #3770) is merged.

The status of this enhancement is marked as at risk. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[haircommander]

I've opened kubernetes/kubernetes#118838 to verify metrics are present as expected

[Rishit-dagli]

Hello @bobbypage @haircommander 👋, 1.28 Docs Lead here.

Does this enhancement work planned for 1.28 require any new docs or modification to existing docs?

If so, please follows the steps here to open a PR against dev-1.28 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday 20th July 2023.

Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.

Thank you!

[Rishit-dagli]

Hey @haircommander , could you please create a docs PR even if it is a draft PR with no content yet against dev-1.28 branch in the k/website repo. The deadline to create this draft PR is Thursday 20th July 2023.

[haircommander]

done! kubernetes/website#42012

[Atharva-Shinde]

Hey @haircommander Enhancements team here 👋

Just checking in as we approach Code freeze at 01:00 UTC Wednesday, 19th July 2023 .

Here’s the enhancement’s state for the upcoming code freeze:

• All the PRs that are related to your enhancement are linked in the above issue description (for tracking purposes). This includes code, tests, and documentation related PR/s.
• All code related PR/s are merged or are in merge-ready state ( i.e they have approved and lgtm labels applied) by the code freeze deadline. This includes any tests related PR/s too.

For this enhancement, it looks like the following code related PR is open and it needs to be merged or should be in merge-ready state before the code freeze commences :

• kubelet: Implement support for Windows podAndContainerStatsFromCRI kubernetes#116968
• e2e_node: add container metrics from cadvisor test kubernetes#118838 (must be merged before Test Freeze i.e 01:00 UTC Wednesday, 26th July, 2023)

This is the code related PR/s that I found on this KEP issue

• kubelet: Implement support for Windows podAndContainerStatsFromCRI kubernetes#116968
• e2e_node: add container metrics from cadvisor test kubernetes#118838

Please keep the issue description up-to-date with all the PR/s that are associated with this KEP and let me know if there are other PR/s in k/k we should be tracking for this KEP.

As always, we are here to help if any questions come up. Thanks!

[Atharva-Shinde]

Hello @haircommander 👋, 1.28 Enhancements Lead here.

Unfortunately, the e2e test PR associated with this enhancement was not in the merge-ready state by the code-freeze and hence this enhancement is now being removed from the v1.28 milestone.

If you still wish to progress this enhancement in v1.28, please file an exception request. Thanks!

/milestone clear

[SergeyKanzhelev]

/milestone v1.29

@haircommander can you please send a PR to bump kep.yaml metadata?

[npolshakova]

Hi @haircommander 👋, 1.29 Enhancements team here!

Just checking in as we approach enhancements freeze on 01:00 UTC, Friday, 6th October, 2023.

This enhancement is targeting for stage alpha for 1.29 (correct me, if otherwise)

Here's where this enhancement currently stands:

• KEP readme using the latest template has been merged into the k/enhancements repo.
• KEP status is marked as implementable for latest-milestone: 1.29. KEPs targeting stable will need to be marked as implemented after code PRs are merged and the feature gates are removed.
• KEP readme has up-to-date graduation criteria
• KEP has a production readiness review that has been completed and merged into k/enhancements. (For more information on the PRR process, check here).

It looks like https://github.com/kubernetes/enhancements/pull/4249/files will address most of these issues.

The status of this enhancement is marked as at risk for enhancement freeze. Please keep the issue description up-to-date with appropriate stages as well. Thank you!

[SergeyKanzhelev]

/stage alpha

[npolshakova]

Hi @haircommander, just checking in once more as we approach the 1.29 enhancement freeze deadline this week on 01:00 UTC, Friday, 6th October, 2023. The status of this enhancement is marked as at risk for enhancement freeze.

It looks like when #4249 is merged in it will address most of the requirements. Let me know if I missed anything. Thanks!

[npolshakova]

With #4249 merged this enhancement is tracked for enhancement freeze! 🚀

[drewhagen]

Hello @bobbypage @haircommander 👋, v1.29 Docs Shadow here.
Does this enhancement work planned for v1.29 require any new docs or modification to existing docs?
If so, please follows the steps here to open a PR against dev-1.29 branch in the k/website repo. This PR can be just a placeholder at this time and must be created before Thursday, 19 October 2023.
Also, take a look at Documenting for a release to get yourself familiarize with the docs requirement for the release.
Thank you!

[haircommander]

I don't believe so. thanks!

[a-mccarthy]

Hi @bobbypage @haircommander, 👋 from the v1.29 Release Team-Communications! We would like to check if you have any plans to publish a blog for this KEP regarding new features, removals, and deprecations for this release.

If so, you need to open a PR placeholder in the website repository.
The deadline will be on Tuesday 14th November 2023 (after the Docs deadline PR ready for review)

Here is the 1.29 calendar

[AnaMMedina21]

Hello @haircommander @bobbypage 👋, v1.29 Enhancements team here.

Just checking in as we approach code freeze at 01:00 UTC Wednesday 1st November 2023

Here's where this enhancement currently stands:

• All PRs to the Kubernetes repo that are related to your enhancement are linked in the above issue description (for tracking purposes).

• Can we add KEP 2371: retarget to alpha for 1.29 #4249 this one to the issue description?

• All PR/s are ready to be merged (they have approved and lgtm labels applied) by the code freeze deadline. This includes tests.

Also, please let me know if there are other PRs in k/k we should be tracking for this KEP.

With all this, the status of this KEP is tracked for code freeze.

As always, we are here to help if any questions come up. Thanks!

[npolshakova]

Hello @haircommander @bobbypage 👋, 1.29 Enhancements team here!

With all the implementation(code related) PRs merged as per the issue description, this enhancement is now marked as tracked for code freeze for the 1.29 Code Freeze! 🚀

The test freeze is 01:00 UTC Wednesday 15th November 2023 / 18:00 PDT Tuesday 14th November 2023. Please make sure all test PRs are merged in by then. The tracked test PRs for this KEP are:

• e2e_node: add container metrics from cadvisor test kubernetes#118838

Please let me know if there are additional test PRs we should track. Thanks!