Behavior-driven conformance testing

[johnbelamaric]

Enhancement Description

• One-line enhancement description (can be used as a release note): Behaviors for conformance are defined up front, separately from the tests which validate those behaviors.
• Kubernetes Enhancement Proposal: Add KEP for Behavior-driven Conformance Testing #959
• Primary contact (assignee): @johnbelamaric
• Responsible SIGs: sig-architecture
• Enhancement target (which target equals to which milestone):
  • Stable release target (1.15)

[johnbelamaric]

/assign
/sig architecture

[kacole2]

/milestone v1.15
/stage stable

[mrbobbytables]

@johnbelamaric Kubernetes 1.15 Enhancement Freeze is 4/30/2019. To be included in the Kubernetes 1.15 milestone, KEPs are required to be merged and in an "Implementable" state with proper test plans and graduation criteria. Please submit any PRs needed to make this KEP adhere to inclusion criteria. If this will slip from the 1.15 milestone, please let us know so we can make appropriate tracking changes.

[johnbelamaric]

Yes, this will likely slip, we still have a lot of details to work out.

[johnbelamaric]

/milestone v1.16

[k8s-ci-robot]

@johnbelamaric: You must be a member of the kubernetes/kubernetes-milestone-maintainers GitHub team to set the milestone. If you believe you should be able to issue the /milestone command, please contact your and have them propose you as an additional delegate for this responsibility.

In response to this:

/milestone v1.16

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[kacole2]

/milestone clear

[kacole2]

Hi @johnbelamaric , I'm the 1.16 Enhancement Lead. Is this feature going to be graduating alpha/beta/stable stages in 1.16? Please let me know so it can be added to the 1.16 Tracking Spreadsheet. If not's graduating, I will remove it from the milestone and change the tracked label.

Once coding begins or if it already has, please list all relevant k/k PRs in this issue so they can be tracked properly.

Milestone dates are Enhancement Freeze 7/30 and Code Freeze 8/29.

Thank you.

[palnabarun]

Hey there @johnbelamaric , 1.17 Enhancements shadow here. I wanted to check in and see if you think this Enhancement will be graduating to alpha in 1.17?

The current release schedule is:

• Monday, September 23 - Release Cycle Begins
• Tuesday, October 15, EOD PST - Enhancements Freeze
• Thursday, November 14, EOD PST - Code Freeze
• Tuesday, November 19 - Docs must be completed and reviewed
• Monday, December 9 - Kubernetes 1.17.0 Released

If you do, I'll add it to the 1.17 tracking sheet (https://bit.ly/k8s117-enhancement-tracking). Once coding begins please list all relevant k/k PRs in this issue so they can be tracked properly. 👍

Thanks!

[johnbelamaric]

Yes, I am targeting phase 1 for this cycle. Phase 1 is/will be defined in #1267

[mrbobbytables]

Thanks @johnbelamaric :) We'll add it to the tracking sheet. 👍

[palnabarun]

/milestone v1.17

[pgburt]

Hello @johnbelamaric , I'm one of the v1.17 docs shadows.
Does this enhancement (or the work planned for v1.17) require any new docs (or modifications to existing docs)? If not, please update the 1.17 Enhancement Tracker Sheet (or let me know and I'll do so).

If so, just a friendly reminder we're looking for a PR against k/website (branch dev-1.17) due by Friday, November 8th, it can just be a placeholder PR at this time. Let me know if you have any questions!

[johnbelamaric]

No, no user-facing docs, thanks.

[palnabarun]

Hi @johnbelamaric

I am one of the Enhancements Shadow for the 1.17 Release Team. We are very near to the Code Freeze (Nov 14th) for this release cycle. Just checking in about the progress of this enhancement. see that
kubernetes/kubernetes#83964 was filed in relation to this. Is there any other PR related to this enhancement? If yes, can you please link them here?

Thank you in advance 😄

[Jefftree]

@palnabarun: kubernetes/kubernetes#83964 is the only related PR. It should be merged before Nov 14.

[palnabarun]

Thank you for the updates @Jefftree :)

[palnabarun]

Hi @johnbelamaric @Jefftree , tomorrow is code freeze for the 1.17 release cycle. It looks like kubernetes/kubernetes#83964 has not yet been merged. We're flagging as the enhancement as At Risk in the 1.17 Enhancement Tracking Sheet.

Do you think they will be merged by the EoD of the 14th (Thursday)? After that point, only release-blocking issues and PRs will be allowed in the milestone with an exception.

[Jefftree]

@palnabarun: Yes it should be merged by EOD Thursday.

[palnabarun]

@Jefftree Awesome! 😃

[palnabarun]

@Jefftree I see that kubernetes/kubernetes#83964 has been merged. Really appreciate the efforts that have been put in.

[jeremyrickard]

Hey @Jefftree @johnbelamaric, 1.18 Enhancements lead here 👋. Happy New Year from the 1.18 Release Team! As this graduated in 1.17, could you update the KEP to indicate that this has been implemented? Then we can close this issue out! Thanks!

[johnbelamaric]

/milestone v1.19

[palnabarun]

@johnbelamaric -- Should this enhancement be tracked by us? Or, does it qualify for tracked/out-of-tree?

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[Jefftree]

KEP has been withdrawn.

/close

[k8s-ci-robot]

@Jefftree: Closing this issue.

In response to this:

KEP has been withdrawn.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[palnabarun]

/milestone clear

(since it wasn't part of the 1.19 cycle)