Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't rely on contents of optional Condition fields in CSI mock test #88520

Merged

Conversation

bertinatto
Copy link
Member

@bertinatto bertinatto commented Feb 25, 2020

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

In order to promote the volume limits e2e test (from CSI Mock driver) to Conformance, we can't rely on specific output of optional Condition fields (e.g., Reason and Message). Thus, this PR changes the test to only check the presence of the right condition and verify that the optional fields are not empty.

Which issue(s) this PR fixes:

Fixes #

Does this PR introduce a user-facing change?:

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:


In order to promote the volume limits e2e test (from CSI Mock driver)
to Conformance, we can't rely on specific output of optional Condition
fields. Thus, this commit changes the test to only check the presence
of the right condition and verify that the optional fields are not empty.
@k8s-ci-robot k8s-ci-robot added release-note-none Denotes a PR that doesn't merit a release note. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. area/test sig/storage Categorizes an issue or PR as relevant to SIG Storage. sig/testing Categorizes an issue or PR as relevant to SIG Testing. and removed needs-sig Indicates an issue or PR lacks a `sig/foo` label and requires one. labels Feb 25, 2020
@bertinatto
Copy link
Member Author

/assign @gnufied

@bertinatto
Copy link
Member Author

/assign @msau42

@bertinatto
Copy link
Member Author

/sig storage

for _, c := range pod.Status.Conditions {
// Conformance tests cannot rely on specific output of optional fields (e.g., Reason
// and Message) because these fields are not suject to the deprecation policy.
if c.Type == v1.PodScheduled && c.Status == v1.ConditionFalse && c.Reason != "" && c.Message != "" {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From https://github.com/kubernetes/community/blob/master/contributors/devel/sig-architecture/conformance-tests.md#conformance-test-requirements:

anything that checks optional Condition fields, such as Reason or Message, as these may change over time (however it is reasonable to verify these fields exist or are non-empty) 

@bertinatto
Copy link
Member Author

/retest

@msau42
Copy link
Member

msau42 commented Feb 25, 2020

/lgtm
/approve

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Feb 25, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bertinatto, msau42

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 25, 2020
@gnufied
Copy link
Member

gnufied commented Feb 25, 2020

/hold

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 25, 2020
@gnufied
Copy link
Member

gnufied commented Feb 25, 2020

This will cause false positives where test would appear to pass but may be for some other reason. Can we have 2 versions of this test?

@msau42
Copy link
Member

msau42 commented Feb 25, 2020

This will cause false positives where test would appear to pass but may be for some other reason. Can we have 2 versions of this test?

We can have another version of the test that's not part of conformance

@johnbelamaric
Copy link
Member

/retest

@bertinatto
Copy link
Member Author

This will cause false positives where test would appear to pass but may be for some other reason. Can we have 2 versions of this test?

We can have another version of the test that's not part of conformance

The volumeLimits test suite has a test that covers that. That test is even more complete and it uses the csi-hostpath driver instead of the mock driver:

if pod.Status.Phase == v1.PodPending {
reg, err := regexp.Compile(`max.+volume.+count`)
if err != nil {
return false, err
}
for _, cond := range pod.Status.Conditions {
matched := reg.MatchString(cond.Message)
if cond.Type == v1.PodScheduled && cond.Status == v1.ConditionFalse && cond.Reason == "Unschedulable" && matched {
return true, nil
}
}
}

IMO it makes sense to have the less strict mock driver test graduated to Conformance and the stricter volumeLimits testsuite test running in the regular presubmit job.

The only downside I see is that the latter is currently running in a non-required job (pull-kubernetes-e2e-gce-csi-serial), but I don't know if that's a big deal.

@bertinatto
Copy link
Member Author

/retest

@msau42
Copy link
Member

msau42 commented Feb 28, 2020

@gnufied what do you think of the hostpath test being more strict than the mock one used for conformance?

@gnufied
Copy link
Member

gnufied commented Feb 28, 2020

yeah that is fine.

/hold cancel
/lgtm

@k8s-ci-robot k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 28, 2020
@k8s-ci-robot k8s-ci-robot merged commit d6a5fa3 into kubernetes:master Feb 28, 2020
@k8s-ci-robot k8s-ci-robot added this to the v1.18 milestone Feb 28, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/test cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/cleanup Categorizes issue or PR as related to cleaning up code, process, or technical debt. lgtm "Looks good to me", indicates that a PR is ready to be merged. needs-priority Indicates a PR lacks a `priority/foo` label and requires one. release-note-none Denotes a PR that doesn't merit a release note. sig/storage Categorizes an issue or PR as relevant to SIG Storage. sig/testing Categorizes an issue or PR as relevant to SIG Testing. size/S Denotes a PR that changes 10-29 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants