-
-
Notifications
You must be signed in to change notification settings - Fork 21.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Revert "Test Godot with Vulkan in CI" #48024
Conversation
This reverts commit 599d961.
If we remove the test projects, what other ways of CI testing and catching regressions do you propose? To fix the issue of "uncontrolled" & "third-party projects", I imagine the projects used to test could be transferred to godot-engine organization? |
As I wrote in the previous thread, I agree with the statement that using an external project in Godot is not a very good idea, so I would prefer Godot to create an official test project instead removing it completelly. Godot has unit tests, but their number and coverage are small because for many elements e.g. physics it is not possible to create such tests easily. The current test project tests the editor (opens and closes it), all classes and methods (also those that PR changes and adds), adding and removing all nodes etc. As for the fact that a change in a test project can trigger errors in CI, that's true and even happened once(yeah, that was my fault). Of course there is also the matter of PR's which break compatibility (at the moment I can think of only 3) and as I wrote earlier this can be handled with 1/2 find/sed commands in CI e.g. As for SwiftShader, I don't see any problem here since it works as it should(in Ci there is info which commit it uses). |
@qarmin's test projects seem quite useful even for non-official stuff, see resolved issues like goostengine/goost#61 in a custom module. I've been considering integrating something like this for the reasons above. But yeah, more reproducibility would be needed, and the approach has to mature to the point of not interfering with checks in other pull requests. An ideal solution to this would be implementing something like godotengine/godot-proposals#168, where issues could be reported outside of CI, or accompany the CI results... That said, if there's a way to make current tests to work in non-blocking manner while still being able to report issues coming from those checks, this would likely resolve the underlying problem as presented with this PR. But at the same time, I don't think the downside of CI occasionally failing to be a major problem (as long as those blockers are fixed, of course). |
Looks like current CI checks work fine, and this PR has merge conflicts now. Would you say it's still relevant? |
Indeed, IIRC qarmin made the CI project more resilient to such API changes so that it doesn't disrupt the workflow. There's still some points worth discussing (notably moving the project under @godotengine) but a revert is not needed. |
This PR reverts #47414, because there are a number of issues with #47414 the most important being:
Note: Currently #47789 is failing CI checks because of #47414, not because #47789 is flawed, but because #47414 is flawed.
While I fully support the principal behind the updates to the CI checks, I think the updates introduced in #47414 were not fully developed or thought through. Therefore this PR should be reverted until the issues identified above have been properly addressed.