Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: add e2e tests #76

Closed
Tracked by #75
laurentsimon opened this issue Mar 25, 2022 · 4 comments
Closed
Tracked by #75

Feature: add e2e tests #76

laurentsimon opened this issue Mar 25, 2022 · 4 comments
Labels
area:go Issue related to the Go ecosystem area:tests An issue with tests (unit, e2e, etc.)

Comments

@laurentsimon
Copy link
Collaborator

laurentsimon commented Mar 25, 2022

We need e2e tests. We will need to create test repos that generate provenance and verify via schedule events; and report an issue automatically if they fail

@laurentsimon
Copy link
Collaborator Author

We also need adversarial tests that try to alter data in the artifact registry the builder uses to pass data between jobs

@laurentsimon
Copy link
Collaborator Author

laurentsimon commented Apr 13, 2022

List:

  • Different triggers (push, workflow_dispatch, schedule, new-tag)
  • Different branches (for each trigger above)
  • Setup pre-submit e2e tests to the extent that it is possible
  • Adversarial tests where the caller workflow manipulate the artifacts
  • Adversarial tests with an invalid input file name
  • Workflow input arguments (upload-assets)
  • Different arguments: vary the env variables, make sure the final provenance contains the right information
  • Different config files: make sure the final provenance contains the right information. Remaining: e2e tests for Feature: add support for dir: field of goreleaser slsa-github-generator-go#128
  • Builder information, such as runner env (except hash)
  • Auto-close issues when e2e tests pass
  • Document how it works and what each workflow does
  • compile-builder option Reduce action runtime #74
  • adversarial test with bogus binary in releases

@laurentsimon
Copy link
Collaborator Author

We should verify that the the provenance generated by these e2e tests can be verified by the slsa-verifier at head and for previous versions released

@laurentsimon laurentsimon transferred this issue from slsa-framework/slsa-github-generator-go May 20, 2022
@laurentsimon laurentsimon added the area:go Issue related to the Go ecosystem label May 20, 2022
@laurentsimon laurentsimon added this to the v1 milestone May 20, 2022
@ianlewis ianlewis added the area:tests An issue with tests (unit, e2e, etc.) label May 31, 2022
@laurentsimon
Copy link
Collaborator Author

Done. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:go Issue related to the Go ecosystem area:tests An issue with tests (unit, e2e, etc.)
Projects
None yet
Development

No branches or pull requests

2 participants