Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Release information to each of the conformance tests. #70626

Merged

Conversation

brahmaroutu
Copy link
Contributor

What this PR does / why we need it:

Since 1.12 we have added a tag named 'Release' as part of conformance documentation this helps to track when a e2e test is added to conformance suite or when a test is modified.
This PR is to print the Release information into the conformance document.

@k8s-ci-robot
Copy link
Contributor

@brahmaroutu: Adding the "do-not-merge/release-note-label-needed" label because no release-note block was detected, please follow our release note process to remove it.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@k8s-ci-robot k8s-ci-robot added size/S Denotes a PR that changes 10-29 lines, ignoring generated files. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. area/conformance Issues or PRs related to kubernetes conformance tests sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. sig/testing Categorizes an issue or PR as relevant to SIG Testing. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Nov 4, 2018
@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Nov 4, 2018
@brahmaroutu
Copy link
Contributor Author

@spiffxp @cheftako @mml this fix help generating proper Conformance document for 1.12. Please review.

@brahmaroutu
Copy link
Contributor Author

Fix required for cncf/k8s-conformance#366

@brahmaroutu
Copy link
Contributor Author

/assign @cheftako
/release-note-none

@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Nov 5, 2018
test/conformance/walk.go Outdated Show resolved Hide resolved
Copy link
Member

@oomichi oomichi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most part looks good for me, just one comment

test/conformance/walk.go Outdated Show resolved Hide resolved
@spiffxp
Copy link
Member

spiffxp commented Nov 30, 2018

/kind cleanup
/priority important-soon

@k8s-ci-robot k8s-ci-robot added kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed needs-kind Indicates a PR lacks a `kind/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Nov 30, 2018
@spiffxp
Copy link
Member

spiffxp commented Dec 1, 2018

Sample output:

screen shot 2018-11-30 at 4 47 30 pm

You've got an extra colon

There are also tests that don't appear to have a release entry, I feel like we should display something other than blank, ie:

  • Unknown
  • pre-v1.9
  • v1.8 or earlier

@duglin
Copy link

duglin commented Jan 5, 2019

/approve
/lgtm

@duglin
Copy link

duglin commented Jan 5, 2019

ping @cheftako

@spiffxp spiffxp added this to To Triage in conformance-definition Jan 11, 2019
@@ -72,6 +72,8 @@ type conformanceData struct {
TestName string
// Extracted from the "Description:" comment before the test
Description string
// Version when this test is added or modified ex: 1.12, 1.13
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably worth being consistent and documenting the extraction prefix.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

if strings.HasPrefix(line, "Testname:") {
line = strings.TrimSpace(line[9:])
cd.TestName = line
if sline := regexp.MustCompile("Testname\\s*:\\s*").Split(line, -1); len(sline) == 2 {
Copy link
Member

@cheftako cheftako Jan 14, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For correctness and to maintain behavior with HasPrefix should we use the start of line indicator?
(Eg. "^Testname\s*:\s*" )

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, it has to be the start of the line, fixed.

continue
}
if sline := regexp.MustCompile("Description\\s*:\\s*").Split(line, -1); len(sline) == 2 {
line = sline[1]
}
cd.Description += line + "\n"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we concerned that newer test files with older walk code will treat newer fields as descriptions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really, we just add every other line in the comment block under the description after extracting the lines that has Testname and Version information.

@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 14, 2019
@brahmaroutu
Copy link
Contributor Author

/test pull-kubernetes-e2e-kops-aws
/test pull-kubernetes-integration

@brahmaroutu
Copy link
Contributor Author

/test pull-kubernetes-e2e-kops-aws

@oomichi
Copy link
Member

oomichi commented Feb 12, 2019

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 12, 2019
Copy link
Member

@cheftako cheftako left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Little concerned that any line we don't recognize is assumed to be a description. This seems to be a real forward compatibility issue. However this change is not making that issue any worse.

@cheftako
Copy link
Member

/lgtm

@spiffxp
Copy link
Member

spiffxp commented May 7, 2019

@brahmaroutu #70626 (comment) remains unaddressed (the issue you opened to address this has aged out: #71627)

Should we still be pushing on this, or close out and try something else?

@brahmaroutu
Copy link
Contributor Author

@spiffxp I reopened the issue #71627. I still see value in adding these changed. I got lgtm's from @oomichi and @cheftako, can we push this out ASAP.

@k8s-ci-robot k8s-ci-robot added area/test and removed lgtm "Looks good to me", indicates that a PR is ready to be merged. labels May 7, 2019
@spiffxp
Copy link
Member

spiffxp commented May 7, 2019

/lgtm
/release-note-none

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 7, 2019
@cheftako
Copy link
Member

cheftako commented May 7, 2019

/lgtm

@cheftako
Copy link
Member

cheftako commented May 7, 2019

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: brahmaroutu, cheftako, duglin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 7, 2019
@k8s-ci-robot k8s-ci-robot merged commit 089e5a7 into kubernetes:master May 7, 2019
conformance-definition automation moved this from To Triage to Done May 7, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/conformance Issues or PRs related to kubernetes conformance tests area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. release-note-none Denotes a PR that doesn't merit a release note. sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
Development

Successfully merging this pull request may close these issues.

None yet

6 participants