tests: Use string and subprocess for isolation#6514
Conversation
Instead of using multiprocessing, use code in a string and subprocess with sys.executable to run code in isolation. While this is requires the tested code to be in a string as opposed to a function, this avoids many issues of multiprocessing (stuck queue on failure, complexities of spawning). It replaces queue communication with JSON in stdout which requires additional care in what is actually printed. As a result, the global environment is clearly not changed (it is a subprocess) as opposed to relying on multiprocessing behavior. For C library test, the separation of processes is even more clear, while at the same time, the test itself can now set up the runtime environment for dynamic libraries instead of relying on the external environmet.
github-actions
bot
added
Python
|
A remaining issue here are the tests for GridModule which is using the C API, but I don't plan to address that in this PR and I don't have a specific plan for it. The string solution looks good for couple specialized tests, but I would prefer not to use it for all GridModule tests. Suggestions welcome. Anyway, I'm hoping this PR will help with failures in #6482. |
…odate backslashes (as a way of dealing with Windows paths).
…lying on stderr being printed through pytest (does not show up on Windows in CI)
…smoke testing. Be more consitent in mocking no global session.
|
I updated the PR description to clarify the what is included with all the additional changes. The grass_script_setup_test.py might be better reviewed as the file, not just the diff, because there is just a lot of structural changes due to switch to code in strings, passing of JSON as dict, not queue with a tuple, and a lot of new comments. However, the diff does show the related old and new lines together, so review with diff only should fine too. |
… Python (it is updated, but the getter function keeps using a cached version), so the C functions don't see GISRC.
2 participants
Instead of using multiprocessing, use code in a string and subprocess with sys.executable to run code in isolation. While this is requires the tested code to be in a string as opposed to a function, this avoids many issues of multiprocessing (stuck queue on failure, complexities of spawning). It replaces queue communication with JSON in stdout which requires additional care in what is actually printed.
As a result, the global environment is clearly not changed (it is a subprocess) as opposed to relying on multiprocessing behavior. For C library test, the separation of processes is even more clear, while at the same time, the test itself can now set up the runtime environment for dynamic libraries instead of relying on the external environment.
Renames and document tests in grass_script_setup_test.py with their current scope as their scope shifted when only one way of isolating and calling is used (no more partial vs nested function). Clarify role of setting the global capturing stderr in the tests and add more capture stderr smoke testing.
Be more consistent in mocking no global session with the fixture.
Uses capture outputs and exception to report stderr from the subprocess to ensure it is reported instead of relying on stderr being printed through pytest (stderr does not show up on Windows in CI with pytest for failures in lib_gis_env_test.py).
The constructed script needs to use raw strings for the path to accommodate backslashes (as a way of dealing with Windows paths).
The library test is skipped on Windows because the environment is not refreshed with the changes made in Python. Python updates it, but the getter function on Windows apparently keeps using a cached version, so the C functions access with ctypes don't see GISRC and fail with no session.