Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

e2e: add matcher to validate output of a spec run #555

Closed
qweeah opened this issue Sep 12, 2022 · 0 comments · Fixed by #639
Closed

e2e: add matcher to validate output of a spec run #555

qweeah opened this issue Sep 12, 2022 · 0 comments · Fixed by #639
Assignees
Labels
E2E testing End-to-end testing related issues or pull requests
Milestone

Comments

@qweeah
Copy link
Contributor

qweeah commented Sep 12, 2022

Need to support below when running a e2e spec

  • Match full content of a output
  • Match keywords
  • Match status logs in the output
  • Use multiple matchers in one command execution

Related to #523

@qweeah qweeah changed the title test(e2e): matching output of spec runs test(e2e): add output matcher to validate output of a spec run Sep 12, 2022
@qweeah qweeah self-assigned this Sep 12, 2022
@qweeah qweeah changed the title test(e2e): add output matcher to validate output of a spec run e2e: add matcher to validate output of a spec run Sep 14, 2022
@qweeah qweeah added the E2E testing End-to-end testing related issues or pull requests label Sep 14, 2022
@shizhMSFT shizhMSFT added this to the v0.16.0 milestone Sep 17, 2022
@qweeah qweeah modified the milestones: v0.16.0, v0.17.0 Nov 2, 2022
@FeynmanZhou FeynmanZhou modified the milestones: v0.16.1, v1.0.0-rc.1 Jan 3, 2023
TerryHowe pushed a commit to TerryHowe/oras that referenced this issue Feb 2, 2023
This PR includes below changes in the E2E utility package:
1. Enables basic auth in test infra
2. Supports excuting oras or other binaries with fluent-styled runtime
configuring and result matching
3. Introduces `match` packages to validate execution results
4. Added sample test specs for `manifest fetch` command and common OCI
image scenario.

Resolves oras-project#554, resolves oras-project#555, resolves oras-project#556, resolves oras-project#565

Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
E2E testing End-to-end testing related issues or pull requests
Projects
No open projects
Status: Done
Development

Successfully merging a pull request may close this issue.

3 participants