Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Run conformance tests in pod on cluster #787

Merged
merged 4 commits into from
Jun 28, 2023

Conversation

ciarams87
Copy link
Member

Proposed changes

Problem: The conformance tests send traffic to the IPs of the NKG Pod for the corresponding Gateway. However, all tests that send traffic are failing with:

http.go:217: Request failed, not ready yet: Get "http://<POD-IP>": dial tcp <POD-IP>80: i/o timeout (after 26µs)
and

http.go:217: Request failed, not ready yet: Get "http://<POD-IP>/": context deadline exceeded (after 22.3µs)

This is because we are running the tests in a Docker container on the local machine which cannot access the internal pod network.

Solution: Instead of running the tests in a Docker container, run the tests in a pod on the cluster. This also includes configuring RBAC for the appropriate CRUD access to the required resources. Additionally, the Makefile commands have been updated to preload the NGINX image to the local cluster, and to remove the requirement to manually edit the deployment manifests with the image under test.

Note: This PR adds a requirement for the utility yq to be installed locally - see https://github.com/mikefarah/yq.

Testing: Ran all commands locally. Tests are now passing traffic successfully.

Closes #771

Checklist

Before creating a PR, run through this checklist and mark each as complete.

  • I have read the CONTRIBUTING doc
  • I have added tests that prove my fix is effective or that my feature works
  • I have checked that all unit tests pass after adding my changes
  • I have updated necessary documentation
  • I have rebased my branch onto main
  • I will ensure my PR is targeting the main branch and pulling from my branch from my own fork

@ciarams87 ciarams87 requested a review from a team as a code owner June 26, 2023 13:25
@github-actions github-actions bot added bug Something isn't working documentation Improvements or additions to documentation labels Jun 26, 2023
conformance/Makefile Outdated Show resolved Hide resolved
conformance/Makefile Outdated Show resolved Hide resolved
conformance/Makefile Outdated Show resolved Hide resolved
conformance/Makefile Outdated Show resolved Hide resolved
conformance/README.md Outdated Show resolved Hide resolved
conformance/README.md Outdated Show resolved Hide resolved
conformance/Makefile Outdated Show resolved Hide resolved
conformance/Makefile Show resolved Hide resolved
conformance/README.md Outdated Show resolved Hide resolved
conformance/tests/Dockerfile Outdated Show resolved Hide resolved
conformance/Makefile Outdated Show resolved Hide resolved
conformance/Makefile Outdated Show resolved Hide resolved
conformance/tests/conformance_spec.yaml Outdated Show resolved Hide resolved
conformance/Makefile Show resolved Hide resolved
conformance/Makefile Outdated Show resolved Hide resolved
@ciarams87 ciarams87 force-pushed the fix/run-tests-in-cluster branch 3 times, most recently from 2ec828e to c42a185 Compare June 27, 2023 00:15
Copy link
Contributor

@pleshakov pleshakov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a few nits and consistency comments. otherwise, looks good!

conformance/README.md Show resolved Hide resolved
conformance/Makefile Outdated Show resolved Hide resolved
conformance/tests/conformance_rbac.yaml Outdated Show resolved Hide resolved
@ciarams87 ciarams87 requested a review from pleshakov June 27, 2023 16:16
@ciarams87 ciarams87 merged commit bae93c1 into nginxinc:main Jun 28, 2023
15 checks passed
@ciarams87 ciarams87 deleted the fix/run-tests-in-cluster branch June 28, 2023 08:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working documentation Improvements or additions to documentation
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

Conformance test runner cannot connect to NKG Pods
4 participants