Reusing the same workspace for each test causes dependencies between tests

[slarse]

The Constants.SORALD_WORKSPACE path is used for every test, but it's not cleaned up. This means that if a source file is used for two tests, it's output path may very well be the same between the two tests. Therefore, the second test may pass without doing anything if the first test produced the correct output.

I see two reasonable solutions.

1. Use a temporary directory for the workspace
2. Clean up the workspace after or before each test

I prefer solution 1, but because of the current setup it may be less work to implement solution 2.

[fermadeiral]

Thank you for pointing that out.

We already have a solution but not exactly for the same problem, and I think this solution is only applied in one test class: see FileOutputStrategyTest, the tearDownmethod.

I prefer solution 1, but because of the current setup it may be less work to implement solution 2.

I don't have strong feelings about any of the proposed solution, so you can go with the one you prefer.

[slarse]

@fermadeiral Right, thanks. That approach can be generalized to a JUnit test listener that applies to the entire test suite, which is essentially solution 2. I'm not entirely sure how to do it for our test suite as it's currently a mix of JUnit4 and 5, and their respective test listeners are slightly different. But it should be manageable.

The reason I prefer solution 1. is that it JUnit5 has great support for automatic handling of temporary directories. It also becomes explicit that the directory used is indeed intended to be used temporarily. Again, not sure how to apply that across JUnit4+5, it may be preferable to convert the whole test suite to JUnit5 before proceeding here.