Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run k/k's e2e suite via new kubetest2 make target #10504

Merged
merged 6 commits into from
Dec 31, 2020
Merged

Run k/k's e2e suite via new kubetest2 make target #10504

merged 6 commits into from
Dec 31, 2020

Conversation

rifelpet
Copy link
Member

@rifelpet rifelpet commented Dec 27, 2020 •
edited
Loading

@k8s-ci-robot
Copy link
Contributor

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Dec 27, 2020
@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Dec 27, 2020
@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

1 similar comment
@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

2 similar comments
@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet
Copy link
Member Author

/test pull-kops-e2e-kubernetes-aws-kubetest2

@rifelpet rifelpet marked this pull request as ready for review December 30, 2020 16:34
@rifelpet rifelpet changed the title WIP Run k/k's e2e suite via new kubetest2 make target Run k/k's e2e suite via new kubetest2 make target Dec 30, 2020
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 30, 2020
@rifelpet
Copy link
Member Author

rifelpet commented Dec 30, 2020 •
edited
Loading

I think this is ready for review. The test is failing because kubetest2 doesn't yet download the same kubectl version as the e2e test version because it assumes we're building k/k from source. In our case it falls back to the kubectl baked into the prow job's image which i believe is 1.20.X and causes a minor test failure due to some text change.

Ideally kubetest2 would support downloading the kubernetes client tar along side the test tar but we can probably build that into our implementation in the short term.

Regardless, I dont think that is a blocker for this PR. Next steps are:

  • set KOPS_BASE_URL so that the cluster uses the same nodeup, kops-controller, etc. as the kops cli that is being built - we have a race condition in the current logic and I'm hesitant to add it to the new test code so I might explore alternatives (we publish the URL to a file in the gcs bucket that the tests then reference. two presubmit test runs could overwrite the file and end up using the wrong KOPS_BASE_URL)
  • Download the kubernetes client tar and use its kubectl for the e2e tests
  • Add manifest template support so that we can customize clusters beyond create cluster flags.
  • Figure out how we want to customize the invocation of the e2e tests. We could create a new makefile under tests/e2e that is included by the top level makefile and have it enumerate all of our test scenarios. feedback on this would be appreciated.

@justinsb
Copy link
Member

This looks great - excited to see this! I would love to get a kops upgrade test going!

/approve
/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Dec 31, 2020
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: justinsb, rifelpet

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@fejta-bot
Copy link

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

@k8s-ci-robot
Copy link
Contributor

@rifelpet: The following test failed, say /retest to rerun all failed tests:

Test name Commit Details Rerun command
pull-kops-e2e-kubernetes-aws-kubetest2 92e0ac7 link /test pull-kops-e2e-kubernetes-aws-kubetest2

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@rifelpet rifelpet closed this Dec 31, 2020
@rifelpet rifelpet reopened this Dec 31, 2020
@rifelpet
Use the same package version as that reported by apiserver
83eeccc 
kubetest2 doesn't download a kubectl client from the same version being tested, so the mismatch is causing test failures.
Until we can download kubectl we'll use the same minor version as /usr/local/bin/kubectl baked into the prow image
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Dec 31, 2020
@hakman
Copy link
Member

hakman commented Dec 31, 2020

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Dec 31, 2020
@k8s-ci-robot k8s-ci-robot merged commit 66fe512 into kubernetes:master Dec 31, 2020
@k8s-ci-robot k8s-ci-robot added this to the v1.20 milestone Dec 31, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@KashifSaadat KashifSaadat Awaiting requested review from KashifSaadat

@zetaab zetaab Awaiting requested review from zetaab

Assignees

@justinsb justinsb

@hakman hakman

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.
Projects
None yet
Milestone
v1.20
Development

Successfully merging this pull request may close these issues.

None yet
5 participants
@rifelpet @k8s-ci-robot @justinsb @fejta-bot @hakman