Implement a test runner for GDScript

[pfertyk]

No description provided.

[meatball133]

Just to make you aware. I am rather sure that you aren't allowed to do this. I think when they run in the containers are they running in read-only environments, I will ask Jeremy or Erik but I think doing something like that aint allowed.

[pfertyk]

Hm ... that might be a bit of a problem :( Godot needs to create some extra folders (at least in ~/.local, I think there might be more) where the configuration of each individual project is stored. Without a permission to modify the filesystem, Godot fails to start. I might be able to create those folders manually in Dockerfile, I'll try to do that later.

[ErikSchierboom]

Well, we don't yet enforce read only, but we might at some point in the future. The /tmp folder is writeable, as well as the output directory passed in as the third argument.

[pfertyk]

Hm ... I'll see what I can do about Godot's limitations here, in case we decide to enforce read only in the future :)

[pfertyk]

BTW @ErikSchierboom , does the fact that readonly might be enforced in the future also mean that test runners shouldn't create any output files, and only use console output? Godot doesn't have an easy way to restrict the amount of printed information (e.g. when assertions are involved, or the greeting message when the binary starts), so I was going to experiment with JSON files for test results. Is this a bad idea? ;)

[pfertyk]

OK, I've added --read-only back, but instead I created 3 volumes for directories that Godot needs to work properly (.config, .local, and .cache). If you're OK with this solution, I will gladly keep it ;)

[ErikSchierboom]

I'm sorry, but that won't work, as all test runners are mounted in the same way (so without those three new files).

[pfertyk]

Hm ... I am not sure, but Godot might work without these folders, it will just print a lot of error messages ... I will check this. As long as the "results.json" file is produced correctly, we should be good, right?

[ErikSchierboom]

That is correct!

[pfertyk]

OK, I've removed extra volumes and I'm redirecting stderr to a file in /tmp. If there is time later, I might add a filtering of those errors, so that the expected ones are ignored and the remaining ones are printed.

[ErikSchierboom]

I'll test it functionally tomorrow!

[ErikSchierboom]

Tests are working fine locally!

[pfertyk]

@ErikSchierboom Fantastic, thanks for checking! Please don't merge this yet, I would like to apply your suggestions, add more tests, and do a bit of refactoring ;)

EDIT: I realized that this is still (correctly) marked as a draft, sending a clear message that it should not be merged yet :P

[ErikSchierboom]

Great work!