Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GithubAction: added job to verify tests fail without src/ changes #2029

Closed
wants to merge 1 commit into from

Conversation

staabm
Copy link
Contributor

@staabm staabm commented Nov 25, 2022

@ondrejmirtes
Copy link
Member

Some suggestions:

  • Run it only if both src/ and tests/ were changed
  • Run it only if the usual tests passed
  • Somehow you need to evaluate that one of the items from the matrix failed, and consider that a success build. Otherwise the build should fail. Possibly you'll need to save the tests result in a machine-readable format, upload the artifact, and do an "evaluate" step with some custom logic written in PHP. I do a similar thing already for the issue bot.

@staabm
Copy link
Contributor Author

staabm commented Nov 25, 2022

I think it works as expected regarding

  • Run it only if the usual tests passed
  • evaluate that one of the items from the matrix failed

missing atm is

  • a check to run all these checks only if both src/ and tests/ were changed
  • a prettty action summary

(and of course the temp changes need to be reverted, as soon as we get everything running as expected)

@staabm
Copy link
Contributor Author

staabm commented Nov 26, 2022

I think we are feature complete.

the job dependency graph

grafik

if you are fine with the result, I will re-enable all lines I previously commented to speedup the feedback loop.

@staabm
Copy link
Contributor Author

staabm commented Jan 5, 2023

If we agreed on this one, I can cleanup all the test code and remove the temporary changes

@staabm staabm force-pushed the test-fail branch 2 times, most recently from e84aa7b to cce9f8e Compare January 20, 2023 21:23
@staabm
Copy link
Contributor Author

staabm commented Jan 20, 2023

squashed and rebased

@staabm staabm marked this pull request as ready for review January 20, 2023 21:23
@phpstan-bot
Copy link
Collaborator

This pull request has been marked as ready for review.

@staabm
Copy link
Contributor Author

staabm commented Apr 5, 2023

@ondrejmirtes is this PR useful? should I rebase it for 1.10.x?

@staabm staabm closed this Sep 9, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants