Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add automated test for quick start test #1045

Merged
merged 19 commits into from
Sep 1, 2023

Conversation

susanshi
Copy link
Collaborator

@susanshi susanshi commented Aug 31, 2023

Description

What this PR does / why we need it:

This test is useful for :

  1. post release validation
  2. Catch breaking changes in between releases

This test mirrors our quick start test, it will use helmfile to install dependencies and ran through steps in https://github.com/deislabs/ratify/blob/main/README.md#quick-start

Please see run result at https://github.com/susanshi/ratify/actions/runs/6031928385

Which issue(s) this PR fixes (optional, using fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when the PR gets merged):

Fixes #

Type of change

Please delete options that are not relevant.

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Helm Chart Change (any edit/addition/update that is necessary for changes merged to the main branch)
  • This change requires a documentation update

How Has This Been Tested?

Please see run result at https://github.com/susanshi/ratify/actions/runs/6031928385

Checklist:

  • Does the affected code have corresponding tests?
  • Are the changes documented, not just with inline documentation, but also with conceptual documentation such as an overview of a new feature, or task-based documentation like a tutorial? Consider if this change should be announced on your project blog.
  • Does this introduce breaking changes that would require an announcement or bumping the major version?
  • Do all new files have appropriate license header?

Post Merge Requirements

  • MAINTAINERS: manually trigger the "Publish Package" workflow after merging any PR that indicates Helm Chart Change

@codecov
Copy link

codecov bot commented Aug 31, 2023

Codecov Report

Patch and project coverage have no change.

Comparison is base (b641d33) 58.34% compared to head (f5f2b81) 58.34%.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1045   +/-   ##
=======================================
  Coverage   58.34%   58.34%           
=======================================
  Files          93       93           
  Lines        5541     5541           
=======================================
  Hits         3233     3233           
  Misses       1992     1992           
  Partials      316      316           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@susanshi
Copy link
Collaborator Author

TODO for this PR: 1. Add a v tag trigger, but this MUST run after other publish activity ends.. 2. document in release activity so we should validate the result of this test

test/bats/quickstart-test.bats Outdated Show resolved Hide resolved
.github/workflows/quick-start.yml Show resolved Hide resolved
- name: Save logs
if: ${{ always() }}
run: |
kubectl logs -n gatekeeper-system -l app=ratify --tail=-1 > logs-ratify-preinstall-${{ matrix.KUBERNETES_VERSION }}-${{ matrix.GATEKEEPER_VERSION }}-config-policy.json
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove gatekeeper version

Copy link
Collaborator

@binbin-li binbin-li left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm overall

Signed-off-by: Susan Shi <huish@microsoft.com>
@susanshi susanshi merged commit c51d9a2 into ratify-project:main Sep 1, 2023
20 checks passed
bspaans pushed a commit to bspaans/ratify that referenced this pull request Oct 17, 2023
Signed-off-by: Susan Shi <huish@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants