Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test all images #1533

Merged
merged 4 commits into from Apr 19, 2021
Merged

Test all images #1533

merged 4 commits into from Apr 19, 2021

Conversation

ciarams87
Copy link
Member

@ciarams87 ciarams87 commented Apr 12, 2021 •
edited

Proposed changes

Currently, we only run smoke tests against the Debian based image. This commit:

  • Updates the edge workflow to use all the different images at least once in the smoke tests matrix
  • Updates the nightly workflow to run the full suite of smoke tests against each of the images at least once a week on the following schedule:
    oss: Debian M-T; Openshift Friday; Alpine Saturday; Opentracing Sunday
    plus: Debian M-F; Opentracing Saturday; Openshift Sunday
    AppProtect: Debian M-F + Sunday; Openshift Saturday

Checklist

Before creating a PR, run through this checklist and mark each as complete.

  • I have read the CONTRIBUTING doc
  • I have added tests that prove my fix is effective or that my feature works
  • I have checked that all unit tests pass after adding my changes
  • I have updated necessary documentation
  • I have rebased my branch onto master
  • I will ensure my PR is targeting the master branch and pulling from my branch from my own fork

@ciarams87 ciarams87 marked this pull request as draft April 12, 2021 09:34
@github-actions github-actions bot added the chore Pull requests for routine tasks label Apr 12, 2021
@ciarams87 ciarams87 force-pushed the chore/test-all-images branch 6 times, most recently from 19feb73 to 20961d0 Compare April 12, 2021 14:28
@ciarams87 ciarams87 marked this pull request as ready for review April 12, 2021 15:18
@ciarams87 ciarams87 changed the title WIP: Test all images Test all images Apr 12, 2021
vepatel
vepatel reviewed Apr 12, 2021
View reviewed changes
.github/workflows/edge.yml
@@ -100,60 +100,89 @@ jobs:
matrix:
include:
- os: ubuntu-20.04
image: nginx-ingress
file: build/Dockerfile
image: debian
Copy link
Contributor

@vepatel vepatel Apr 12, 2021 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ciarams87 there's an issue with re to test markers, acc. to this only 'ingresses' tests will run on debian and only 'vsr' tests will run on alpine and so on. I think existing matrix will be needed to multiplied by types of images so all markers run on each platform for both oss and plus.
Nightly will be multiplied too but since there are no markers in there, it'll be platform x k8s versions.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed offline, we have to make a trade-off on edge because the matrix of jobs would get too large. I'll add two more jobs so that we are testing everything on Debian Plus image, as agreed.

lucacome
lucacome previously requested changes Apr 13, 2021
View reviewed changes
.github/workflows/nightly.yml Outdated Show resolved Hide resolved
@ciarams87 @lucacome
Update .github/workflows/nightly.yml
c5a8d6c 
Co-authored-by: Luca Comellini <luca.com@gmail.com>
@ciarams87 ciarams87 requested a review from lucacome April 14, 2021 12:28
@ciarams87 ciarams87 dismissed lucacome’s stale review April 19, 2021 10:48

Comment addressed.

@ciarams87 ciarams87 merged commit c3d24c4 into master Apr 19, 2021
@ciarams87 ciarams87 deleted the chore/test-all-images branch April 19, 2021 13:19
lucacome pushed a commit that referenced this pull request Apr 19, 2021
@ciarams87 @lucacome
Test all images (#1533)
7f12658 
* Test on all images

* Update nightly to test all images

* Run all test markers on debian plus also

* Update .github/workflows/nightly.yml
vepatel added a commit that referenced this pull request Apr 26, 2021
@vepatel
# This is a combination of 26 commits.
6c3dd67 
# This is the 1st commit message:

add ingress mtls test

# This is the commit message #2:

add std vs

# This is the commit message #3:

change vs host

# This is the commit message #4:

Update tls secret

# This is the commit message #5:

update certs with host

# This is the commit message #6:

modify get_cert

# This is the commit message #7:

Addind encoded cert

# This is the commit message #8:

Update secrets

# This is the commit message #9:

Add correct cert and SNI module

# This is the commit message #10:

Bump styfle/cancel-workflow-action from 0.8.0 to 0.9.0 (#1527)

Bumps [styfle/cancel-workflow-action](https://github.com/styfle/cancel-workflow-action) from 0.8.0 to 0.9.0.
- [Release notes](https://github.com/styfle/cancel-workflow-action/releases)
- [Commits](styfle/cancel-workflow-action@0.8.0...89f242e)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
# This is the commit message #11:

Remove patch version from Docker image for tests (#1534)


# This is the commit message #12:

Add tests for Ingress TLS termination

# This is the commit message #13:

Improve assertion of TLS errors in tests

When NGINX terminates a TLS connection for a server
with a missing/invalid TLS secret, we expect NGINX
to reject the connection with the error
TLSV1_UNRECOGNIZED_NAME

In this commit we:
* ensure the specific error
* rename the assertion function to be more specific

# This is the commit message #14:

Bump k8s.io/client-go from 0.20.5 to 0.21.0 (#1530)

Bumps [k8s.io/client-go](https://github.com/kubernetes/client-go) from 0.20.5 to 0.21.0.
- [Release notes](https://github.com/kubernetes/client-go/releases)
- [Changelog](https://github.com/kubernetes/client-go/blob/master/CHANGELOG.md)
- [Commits](kubernetes/client-go@v0.20.5...v0.21.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
# This is the commit message #15:

Improve tests Dockerfile

* Reorganize layers so that changes to the tests do not cause a full
image rebuilt
* Use .dockerignore to ignore cache folders
* Convert spaces to tabs for consistency with the other Dockerfiles

# This is the commit message #16:

Upgrade kubernetes-python client to 12.0.1 (#1522)

* Upgrade kubernetes-python client to 12.0.1

Co-authored-by: Venktesh Patel <ve.patel@f5.com>
# This is the commit message #17:

Bump k8s.io/code-generator from 0.20.5 to 0.21.0 (#1531)

Bumps [k8s.io/code-generator](https://github.com/kubernetes/code-generator) from 0.20.5 to 0.21.0.
- [Release notes](https://github.com/kubernetes/code-generator/releases)
- [Commits](kubernetes/code-generator@v0.20.5...v0.21.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
# This is the commit message #18:

Test all images (#1533)

* Test on all images

* Update nightly to test all images

* Run all test markers on debian plus also

* Update .github/workflows/nightly.yml

# This is the commit message #19:

Add tests for default server

# This is the commit message #20:

Support running tests in kind

# This is the commit message #21:

Update badge for Fossa (#1546)


# This is the commit message #22:

Fix ensuring connection in tests

* Add timeout for establishing a connection to prevent potential
"hangs" of the test runs. The problem was noticeable when running
tests in kind.
* Increase the number of tries to make sure the Ingress Controller
pod has enough time to get ready. When running tests in kind
locally the number of tries sometimes was not enough.

# This is the commit message #23:

Ensure connection in Ingress TLS tests

Ensure connection to NGINX before running tests.
Without ensuring, sometimes the first connection to NGINX would
hang (timeout). The problem is noticable when running tests in
kind.

# This is the commit message #24:

Revert changes in nightly for now (#1547)


# This is the commit message #25:

Bump actions/cache from v2.1.4 to v2.1.5 (#1541)

Bumps [actions/cache](https://github.com/actions/cache) from v2.1.4 to v2.1.5.
- [Release notes](https://github.com/actions/cache/releases)
- [Commits](actions/cache@v2.1.4...1a9e213)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
# This is the commit message #26:

Create release workflow
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vepatel vepatel vepatel approved these changes

@soneillf5 soneillf5 soneillf5 approved these changes

@pleshakov pleshakov Awaiting requested review from pleshakov

@lucacome lucacome Awaiting requested review from lucacome

Assignees
No one assigned
Labels
chore Pull requests for routine tasks
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
4 participants
@ciarams87 @lucacome @vepatel @soneillf5