Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Bug 1823494: Storage tests should more carefully select zones for testing #24900

Merged
merged 1 commit into from Apr 23, 2020
Merged

Bug 1823494: Storage tests should more carefully select zones for testing #24900

merged 1 commit into from Apr 23, 2020

Conversation

gnufied
Copy link
Member

@gnufied gnufied commented Apr 20, 2020

@openshift-ci-robot
Copy link

@gnufied: This pull request references Bugzilla bug 1823494, which is valid. The bug has been moved to the POST state. The bug has been updated to refer to the pull request using the external bug tracker.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.5.0) matches configured target release for branch (4.5.0)
  • bug is in the state ASSIGNED, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

Bug 1823494: UPSTREAM: 90147: relax zone requirement from e2e

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot openshift-ci-robot added the bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. label Apr 20, 2020
@openshift-ci-robot openshift-ci-robot added the vendor-update Touching vendor dir or related files label Apr 20, 2020
@gnufied
Copy link
Member Author

gnufied commented Apr 21, 2020

/retest

@gnufied
Copy link
Member Author

gnufied commented Apr 21, 2020

/test ci/prow/etcd-operator-master-e2e-azure

@openshift-ci-robot
Copy link

@gnufied: The specified target(s) for /test were not found.
The following commands are available to trigger jobs:

  • /test artifacts
  • /test e2e-aws
  • /test e2e-aws-csi
  • /test e2e-aws-disruptive
  • /test e2e-aws-fips
  • /test e2e-aws-image-registry
  • /test e2e-aws-jenkins
  • /test e2e-aws-multitenant
  • /test e2e-aws-ovn
  • /test e2e-aws-serial
  • /test e2e-azure
  • /test e2e-cmd
  • /test e2e-conformance-k8s
  • /test e2e-gcp
  • /test e2e-gcp-builds
  • /test e2e-gcp-image-ecosystem
  • /test e2e-gcp-upgrade
  • /test e2e-vsphere
  • /test images
  • /test launch-vsphere
  • /test unit
  • /test verify
  • /test extended_gssapi
  • /test extended_ldap_groups
  • /test extended_networking

Use /test all to run all jobs.

In response to this:

/test ci/prow/etcd-operator-master-e2e-azure

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@jsafrane
Copy link
Contributor

/lgtm
/retest

@openshift-ci-robot openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Apr 21, 2020
@gnufied
Copy link
Member Author

gnufied commented Apr 21, 2020

/test e2e-azure

@gnufied
Copy link
Member Author

gnufied commented Apr 22, 2020

/retest

@gnufied
Copy link
Member Author

gnufied commented Apr 22, 2020

/bugzilla refresh

@openshift-ci-robot
Copy link

@gnufied: This pull request references Bugzilla bug 1823494, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.5.0) matches configured target release for branch (4.5.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@gnufied
Copy link
Member Author

gnufied commented Apr 22, 2020

/bugzilla refresh

knobunc
knobunc approved these changes Apr 22, 2020
View reviewed changes
Copy link
Contributor

@knobunc knobunc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@openshift-ci-robot
Copy link

@gnufied: This pull request references Bugzilla bug 1823494, which is valid.

3 validation(s) were run on this bug
  • bug is open, matching expected state (open)
  • bug target release (4.5.0) matches configured target release for branch (4.5.0)
  • bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, ON_DEV, POST, POST)

In response to this:

/bugzilla refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@openshift-ci-robot
Copy link

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: gnufied, jsafrane, knobunc

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci-robot openshift-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 22, 2020
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@gnufied
Copy link
Member Author

gnufied commented Apr 22, 2020

/retest

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@gnufied
Copy link
Member Author

gnufied commented Apr 22, 2020

The azure failures are unrelated to this PR and issues that this PR set out to fix has been fixed - https://prow.svc.ci.openshift.org/view/gcs/origin-ci-test/pr-logs/pull/24900/pull-ci-openshift-origin-master-e2e-azure/46 (see no more inline volume failures!).

I think e2e-azure is not a blocking job, but since I summoned it manually, we can't merge this PR until it passes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

1 similar comment
@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-bot
Copy link
Contributor

/retest

Please review the full test history for this PR and help us cut down flakes.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Apr 23, 2020 •
edited

@gnufied: The following test failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
ci/prow/e2e-azure 61a14d7 link /test e2e-azure

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@smarterclayton smarterclayton changed the title Bug 1823494: UPSTREAM: 90147: relax zone requirement from e2e Bug 1823494: Storage tests should more carefully select zones for testing Apr 23, 2020
@smarterclayton
Copy link
Contributor

smarterclayton commented Apr 23, 2020 •
edited

Looks like storage tests on azure are failing for other reasons now after this fix:

Apr 22 21:33:58.269: INFO: At 2020-04-22 21:27:52 +0000 UTC - event for pvc-qvt4m: {persistentvolume-controller } ProvisioningFailed: storageclass.storage.k8s.io "e2e-volume-7559" not found
Apr 22 21:33:58.269: INFO: At 2020-04-22 21:31:20 +0000 UTC - event for azure-injector: {attachdetach-controller } FailedAttachVolume: AttachVolume.Attach failed for volume "azure-disk-hdtrl" : Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 409, RawError: {
  "error": {
    "code": "OperationNotAllowed",
    "message": "The maximum number of data disks allowed to be attached to a VM of this size is 4.",
    "target": "dataDisks"
  }
}

and then the injector pod timeout which looks similar:

Apr 22 21:28:41.857: INFO: At 0001-01-01 00:00:00 +0000 UTC - event for azure-injector: {default-scheduler } FailedScheduling: 0/6 nodes are available: 1 node(s) exceed max volume count, 1 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate, 4 node(s) didn't match node selector.

Are there bugs open tracking those?

@smarterclayton
Copy link
Contributor

Also, 4 data disks is crazy small. Is that a bug?

@smarterclayton smarterclayton mentioned this pull request Apr 23, 2020
Merged
@gnufied
Copy link
Member Author

gnufied commented Apr 23, 2020

@smarterclayton this is the table used for computing disk attach limits for Azure - https://github.com/openshift/origin/blob/master/vendor/k8s.io/kubernetes/pkg/volume/azure_dd/azure_dd_max_disk_count.go#L23

@openshift-merge-robot openshift-merge-robot merged commit 980671d into openshift:master Apr 23, 2020
@openshift-ci-robot
Copy link

@gnufied: All pull requests linked via external trackers have merged: openshift/origin#24900. Bugzilla bug 1823494 has been moved to the MODIFIED state.

In response to this:

Bug 1823494: Storage tests should more carefully select zones for testing

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@knobunc knobunc knobunc approved these changes

@deads2k deads2k Awaiting requested review from deads2k

@sjenning sjenning Awaiting requested review from sjenning

Assignees

@jsafrane jsafrane

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. bugzilla/valid-bug Indicates that a referenced Bugzilla bug is valid for the branch this PR is targeting. lgtm Indicates that a PR is ready to be merged. vendor-update Touching vendor dir or related files
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
7 participants
@gnufied @openshift-ci-robot @jsafrane @openshift-bot @smarterclayton @knobunc @openshift-merge-robot