Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

e2e: add matcher to validate output of a spec run #555

Closed
qweeah opened this issue Sep 12, 2022 · 0 comments · Fixed by #639
Closed

e2e: add matcher to validate output of a spec run #555

qweeah opened this issue Sep 12, 2022 · 0 comments · Fixed by #639
Assignees
@qweeah
Labels
E2E testing End-to-end testing related issues or pull requests
Milestone
v1.0.0-rc.1

Comments

@qweeah
Copy link
Contributor

qweeah commented Sep 12, 2022 •
edited
Loading

Need to support below when running a e2e spec

  • Match full content of a output
  • Match keywords
  • Match status logs in the output
  • Use multiple matchers in one command execution

Related to #523
@qweeah qweeah changed the title test(e2e): matching output of spec runs test(e2e): add output matcher to validate output of a spec run Sep 12, 2022
@qweeah qweeah self-assigned this Sep 12, 2022
@qweeah qweeah mentioned this issue Sep 12, 2022
Closed
@qweeah qweeah changed the title test(e2e): add output matcher to validate output of a spec run e2e: add matcher to validate output of a spec run Sep 14, 2022
@qweeah qweeah added the E2E testing End-to-end testing related issues or pull requests label Sep 14, 2022
@shizhMSFT shizhMSFT added this to the v0.16.0 milestone Sep 17, 2022
@qweeah qweeah mentioned this issue Oct 11, 2022
Merged
@qweeah qweeah modified the milestones: v0.16.0, v0.17.0 Nov 2, 2022
@FeynmanZhou FeynmanZhou modified the milestones: v0.16.1, v1.0.0-rc.1 Jan 3, 2023
TerryHowe pushed a commit to TerryHowe/oras that referenced this issue Feb 2, 2023
@qweeah
test(e2e): add E2E testing utilities and sample specs (oras-project#639)
7397155 
This PR includes below changes in the E2E utility package:
1. Enables basic auth in test infra
2. Supports excuting oras or other binaries with fluent-styled runtime
configuring and result matching
3. Introduces `match` packages to validate execution results
4. Added sample test specs for `manifest fetch` command and common OCI
image scenario.

Resolves oras-project#554, resolves oras-project#555, resolves oras-project#556, resolves oras-project#565

Signed-off-by: Billy Zha <jinzha1@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@qweeah qweeah

Labels
E2E testing End-to-end testing related issues or pull requests
Projects
No open projects
ORAS-Planning
Status: Done
Milestone
v1.0.0-rc.1
Development

Successfully merging a pull request may close this issue.

test(e2e): add E2E testing utilities and sample specs qweeah/oras
3 participants
@qweeah @shizhMSFT @FeynmanZhou