Add support for arranging tests in subdirectories #125
Conversation
This is embarrassing. We copy the input .c file to the Sphinx srcdir for every extension test case. However, the extension looks at hawkmoth_root configuration for the input file root, defaulting to the Sphinx confdir. Which, during test execution, is the test directory. We don't use the copied files at all. So stop copying.
Set hawkmoth_root for extension tests to the directory holding the testcase yaml. We'll want the filenames in the testcase yaml to be relative to the yaml, with hopes it's the least confusing option. We also don't want to adjust the directive during testing depending on the paths either. Because the hawkmoth_root depends on the testcase yaml location, we can't set it in conf.py. For now, these are all the same, of course, but this will change if we arrange the tests in a directory hierarchy.
Add support for organizing the files in subdirectories. Note that organizing examples in a subdirectory currently does not work, because the documentation sets hawkmoth_root to the test directory.
Be defensive against test environment bugs that cause tests to pass with empty results.
Fail if a mandatory file is None or does not exist. Be more robust against errors in the test environment.
| @@ -101,9 +101,10 @@ def get_testid(testcase): | |||
| return testcase.get_testid() | |||
There was a problem hiding this comment.
I'm not sure that the comment about the examples is warranted. hawkmoth_root simply has to be redirected to an examples folder and / or the directives must include the path of the files relative to hawkmoth_root. I.e. it does work with respect to the changes we introduce here.
Anyway, nitpicking.
There was a problem hiding this comment.
It'll work if you move all the examples to the same directory, and point hawkmoth_root there. But if you wanted to have examples in multiple directories, it'll break...
There was a problem hiding this comment.
My point is that this code is perfect as it is with regards to that. If we have hawkmoth_root=test/examples, we just need to do .. c:autodoc:: dir1/dir2/src.c, either way it's not a limitation with regards to what you changed here. We'd need only change conf.py and possibly adjust update_examples.py unless I've missed something.
Anyway, good merge ;)
|
Looks pretty good to me! |
|
Thanks for the review, merged! |
This should enable us to start arranging tests in subdirectories. I played with it a bit, and it seems to work, but I didn't actually include any rearrangement of the files. I threw in some additional asserts in case the files aren't found.
Examples work test-wise, but
make htmlwill fail if the examples are placed in a subdirectory, because ofhawkmoth_rootconfig. I think that's fine for now.