Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add high-level E2E test steps for Skipper #3

Merged
merged 3 commits into from Aug 4, 2020
Merged

Add high-level E2E test steps for Skipper #3

merged 3 commits into from Aug 4, 2020

Conversation

icereed
Copy link

@icereed icereed commented Aug 3, 2020 •
edited by universam1

  • install latest stable kubectl e2e-kind.sh
  • install Kubernetes Kind e2e-kind.sh
  • create local Kubernetes cluster with kind e2e-kind.sh
  • install Skipper ingress with Kustomize e2e-skipper.sh
  • load Flagger image onto the local cluster e2e-skipper.sh
  • install Flagger and Prometheus in the flagger-system namespace e2e-skipper.sh
  • create a test namespace e2e-skipper-tests.sh
  • deploy the load tester in the test namespace e2e-skipper-tests.sh
  • deploy the demo workload (podinfo) and ingress in the test namespace e2e-skipper-tests.sh
  • test the canary initialization e2e-skipper-tests.sh
  • test the canary analysis and promotion using weighted traffic and the load testing webhook [e2e-skipper-tests.sh]e2e-skipper-tests.sh)
  • test the A/B testing analysis and promotion using header filters and pre/post rollout webhooks [e2e-skipper-tests.sh]e2e-skipper-tests.sh)
  • cleanup test environment e2e-skipper-cleanup.sh

Other TODOs:

  • add CircleCI stage

@icereed icereed self-assigned this Aug 3, 2020
@icereed
Copy link
Author

icereed commented Aug 3, 2020 •
edited

How to test:

make build
make loadtester-build
kind create cluster --wait 5m --image kindest/node:v1.16.9
./test/e2e-skipper.sh
./test/e2e-skipper-tests.sh

@icereed
Add high-level E2E test steps for Skipper
222caaa
@icereed
Add e2e-skipper.sh for test setup
3a8f8d5 
It does the following things:
* install Skipper ingress with Kustomize
* load Flagger image onto the local cluster
* install Flagger and Prometheus in the flagger-system namespace
@icereed icereed force-pushed the e2e-tests branch 3 times, most recently from 0b6ea00 to ae03009 Compare August 3, 2020 14:21
@icereed
Copy link
Author

icereed commented Aug 3, 2020

Weird: At this point, a canary release test is successfully run, even though there is no implementation yet for Skipper.
This is a false positive. Next step would be to actually get the E2E test to fail and fix it then afterwards.

@universam1 universam1 changed the base branch from master to feature-Skipper August 4, 2020 09:39
@universam1
Copy link

point 5 from fluxcd#452 (comment)

@universam1 universam1 mentioned this pull request Aug 4, 2020
Closed
@dhohengassner
Copy link

How to test:

make build
make loadtester-build
kind create cluster --wait 5m --image kindest/node:v1.16.9
./test/e2e-skipper.sh
./test/e2e-skipper-tests.sh

docker tag weaveworks/flagger:latest test/flagger:latest is needed as well

@universam1 universam1 marked this pull request as ready for review August 4, 2020 14:37
dhohengassner
dhohengassner approved these changes Aug 4, 2020
View reviewed changes
Copy link

@dhohengassner dhohengassner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good to get real test running - cleanup and circleCI can be added with another PR

@universam1 universam1 merged commit b7a8d37 into feature-Skipper Aug 4, 2020
@universam1 universam1 deleted the e2e-tests branch August 4, 2020 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@szuecs szuecs szuecs left review comments

@dhohengassner dhohengassner dhohengassner approved these changes

@universam1 universam1 Awaiting requested review from universam1

@webframp webframp Awaiting requested review from webframp

Assignees

@dhohengassner dhohengassner

Labels
None yet
Projects
None yet
Milestone
No milestone
4 participants
@icereed @universam1 @dhohengassner @szuecs