Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow to wait for test completion when running tests #8

Merged
merged 14 commits into from
Feb 24, 2023

Conversation

Luc45
Copy link
Member

@Luc45 Luc45 commented Feb 21, 2023

This PR allows running a test in sync mode by passing the --wait flag.

This is important if we want to allow users to delegate their testing execution to us in their own PRs, for instance.

How to test this:

  • Make sure you are connected to a Manager in the branch 23-01/results-iframe
  • ./qit run:activation {foo} --wait
  • See that the console waits for the completion of the test
  • In a new terminal session, run ./qit list-tests and see the last test ID
  • Go to the Manager, enter the container with make enter
  • Run wp post meta update {TEST_ID} cd_status failed
  • Go back to the terminal that was "waiting", see that it shows the completed test result.
  • Run echo $? and assert that the exit status is 1
  • Repeat the process, now setting the result of the test to success
  • Run echo $? and assert that the exit status is 0
  • If you wish, you can repeat for warning, which exits with a 2, and any other status is a 3.
  • Bonus: Do the same, but now with --json as well.

Unhappy paths:

  • Run with a timeout (minimum is 10), eg: ./qit run:activation {foo} --wait --timeout=10, see that the execution ends after the timeout is reached with a meaningful message informing that the test is still running, but we've reached the timeout for waiting for it.
  • Run without a timeout, but instead of updating the test result on the Manager, abort it with Ctrl+C, assert that you receive a meaningful message that the test is still running, but we've aborted waiting for it.

@Luc45 Luc45 self-assigned this Feb 21, 2023
Copy link
Contributor

@zhongruige zhongruige left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Thanks for adding in this wait, this is a nice addition. I left a comment for a potential wording change, but it doesn't need to hold up this PR.

Base automatically changed from 23-01/results-url to trunk February 24, 2023 18:58
@Luc45 Luc45 merged commit a7a292a into trunk Feb 24, 2023
@Luc45 Luc45 deleted the 23-01/wait-for-test-completion branch February 24, 2023 18:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants