Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(e2e): move environment used in e2e to testenv package #4607

Closed
wants to merge 8 commits into from

Conversation

randmonkey
Copy link
Contributor

@randmonkey randmonkey commented Sep 5, 2023

What this PR does / why we need it:

move definition of all environment variable required in e2e tests in testenv package. Also adds some helper functions to the package.

Which issue this PR fixes:

fixes #4593
Special notes for your reviewer:

TODO:

  • update env variables/inputs in .github/workflows (how to update the outputs for image of docker build to put them separately in targeted e2e tests?)
  • update document TESTING.md

REVIEW: the env vars to specify images in e2e tests are changed, from a single *_IMAGE_OVERRIDE to *_IMAGE and *_TAG. Should this change happen?

PR Readiness Checklist:

Complete these before marking the PR as ready to review:

  • the CHANGELOG.md release notes have been updated to reflect any significant (and particularly user-facing) changes introduced by this PR

@randmonkey randmonkey requested a review from a team as a code owner September 5, 2023 09:33
@randmonkey randmonkey self-assigned this Sep 5, 2023
@randmonkey randmonkey marked this pull request as draft September 5, 2023 09:34
.github/workflows/e2e_nightly.yaml Outdated Show resolved Hide resolved
test/internal/testenv/testenv.go Outdated Show resolved Hide resolved
@pmalek pmalek added the area/ci label Sep 5, 2023
@pmalek
Copy link
Member

pmalek commented Sep 5, 2023

REVIEW: the env vars to specify images in e2e tests are changed, from a single *_IMAGE_OVERRIDE to *_IMAGE and *_TAG. Should this change happen?

I'd suggest that we do this change only if the can make it consistent with e.g. env variables used for Kong images

@programmer04
Copy link
Member

For env variables for the Kong image please ensure that passing the image digest is possible too e.g.

kong:3.4.0-ubuntu@sha256:fdfe2b3097e19964e2293ddc218ac714ab630483be0f835f4f5f52ca1ee9a7cb 

it is useful when for instance when you report a bug to Gateway and want to ensure that everybody talks about exactly the same artifact

@codecov
Copy link

codecov bot commented Sep 5, 2023

Codecov Report

All modified lines are covered by tests ✅

Comparison is base (82dbd35) 78.0% compared to head (c3d6bd9) 78.0%.
Report is 4 commits behind head on main.

Additional details and impacted files
@@          Coverage Diff          @@
##            main   #4607   +/-   ##
=====================================
  Coverage   78.0%   78.0%           
=====================================
  Files        163     163           
  Lines      18531   18546   +15     
=====================================
+ Hits       14458   14477   +19     
- Misses      3264    3265    +1     
+ Partials     809     804    -5     

see 5 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@randmonkey randmonkey added the ci/run-e2e Trigger e2e test run from PR label Sep 5, 2023
@team-k8s-bot
Copy link
Collaborator

E2E (targeted) tests with KIND-based clusters were started at https://github.com/Kong/kubernetes-ingress-controller/actions/runs/6085026268

@team-k8s-bot team-k8s-bot removed the ci/run-e2e Trigger e2e test run from PR label Sep 5, 2023
@randmonkey
Copy link
Contributor Author

randmonkey commented Sep 18, 2023

For env variables for the Kong image please ensure that passing the image digest is possible too e.g.

kong:3.4.0-ubuntu@sha256:fdfe2b3097e19964e2293ddc218ac714ab630483be0f835f4f5f52ca1ee9a7cb 

it is useful when for instance when you report a bug to Gateway and want to ensure that everybody talks about exactly the same artifact

Actually we can use the name@digest format like kong@sha256:fdfe2b3097e19964e2293ddc218ac714ab630483be0f835f4f5f52ca1ee9a7cb to specify an image with given SHA digest. So I decide not to support parsing kong version from tag@digest format, but support providing Kong version via env TEST_KONG_EFFECTIVE_VERSION.

@randmonkey randmonkey added the ci/run-e2e Trigger e2e test run from PR label Sep 18, 2023
@randmonkey randmonkey added ci/run-e2e Trigger e2e test run from PR and removed ci/run-e2e Trigger e2e test run from PR labels Sep 18, 2023
@team-k8s-bot
Copy link
Collaborator

E2E (targeted) tests with KIND-based clusters were started at https://github.com/Kong/kubernetes-ingress-controller/actions/runs/6218969564

@team-k8s-bot team-k8s-bot removed the ci/run-e2e Trigger e2e test run from PR label Sep 18, 2023
@randmonkey randmonkey marked this pull request as ready for review September 18, 2023 06:56
@pmalek
Copy link
Member

pmalek commented Sep 18, 2023

@randmonkey The e2e tests failed :(

@randmonkey
Copy link
Contributor Author

randmonkey commented Sep 18, 2023

@randmonkey The e2e tests failed :(

Digged a while in workflow files and logs of the e2e runs and found that I did not update all appearances of inputs.kong-image (

@randmonkey randmonkey force-pushed the test/unify_environments_e2e branch 2 times, most recently from c0602c7 to cd8a066 Compare September 18, 2023 17:00
@randmonkey randmonkey added the ci/run-e2e Trigger e2e test run from PR label Sep 18, 2023
@team-k8s-bot
Copy link
Collaborator

E2E (targeted) tests with KIND-based clusters were started at https://github.com/Kong/kubernetes-ingress-controller/actions/runs/6225645924

@team-k8s-bot team-k8s-bot removed the ci/run-e2e Trigger e2e test run from PR label Sep 18, 2023
.github/workflows/_docker_build.yaml Outdated Show resolved Hide resolved
.github/workflows/_docker_build.yaml Outdated Show resolved Hide resolved
@randmonkey randmonkey added the ci/run-e2e Trigger e2e test run from PR label Oct 12, 2023
@team-k8s-bot
Copy link
Collaborator

E2E (targeted) tests with KIND-based clusters were started at https://github.com/Kong/kubernetes-ingress-controller/actions/runs/6490928774

@team-k8s-bot team-k8s-bot removed the ci/run-e2e Trigger e2e test run from PR label Oct 12, 2023
@randmonkey randmonkey added this to the Fixit 2023 milestone Oct 12, 2023
@randmonkey randmonkey added the ci/run-e2e Trigger e2e test run from PR label Oct 12, 2023
@team-k8s-bot
Copy link
Collaborator

E2E (targeted) tests with KIND-based clusters were started at https://github.com/Kong/kubernetes-ingress-controller/actions/runs/6493972188

@team-k8s-bot team-k8s-bot removed the ci/run-e2e Trigger e2e test run from PR label Oct 12, 2023
@randmonkey randmonkey closed this Oct 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Unify environment variables in tests
4 participants