Name test suites more consistently

[ProfJanetDavis]

Test suites for specific detectors are not named consistently, and are mixed in with test suites for the framework:

tests/test_document.py 
tests/test_effort_detector.py
tests/test_flags.py 
tests/test_main_script.py 
tests/test_personal_life.py
tests/test_publication_detector.py
tests/test_report.py
tests/test_sanity.py

My instinct is to move the test suites for detectors into a subdirectory, but that doesn't correspond well to the structure of the source code. We also expect to see more active development of new detectors than the framework.

An alternative is to move all the test suites for the framework or base system into a subdirectory, and leave the tests of detectors at the top level:

tests/base/test_document.py 
tests/base/test_flags.py 
tests/base/test_main_script.py 
tests/base/test_report.py
tests/base/test_sanity.py
tests/test_effort.py
tests/test_personal_life.py
tests/test_publication.py

This reorganization would also reveal that many detectors lack automated tests.

[j6k4m8]

We're halfway between colocated unit tests (myfile.py and test_myfile.py in the same directory) and mirrored directory trees (foo/bar/myfile.py and test/foo/bar/test_myfile.py). I very much prefer the former because it puts tests directly alongside their target code (if we delete foo/bar, we also delete all tests that correspond to those units) and it eliminates the difficulty of maintaining a test directory submodule (which is a bit cumbersome for not much benefit). I think that was Neil's initial conception as well in the early days of this repository, and I'm ashamed to say I've fallen a bit out of the habit of keeping tests where they belong.

[ProfJanetDavis]

Wow - it is super confusing that the tests are in two different places! With the modular organization of the code, I can see the advantage of co-locating the tests with the source code.

The one thing I wonder about is that it seems like pytest is currently running just the tests in the test directory. Is there some configuration required for pytest to discover the other tests?

[j6k4m8]

oof. I have my head in too many codebases right now. All of the tests are indeed in /tests, which pytest does automatically discover.

I agree that we should organize them like you mention above!
I'd also like to add a handful of integration tests with letters that are Known-Good or Known-Bad so we can make sure that at least simple statistics (Bad reports tend to have more negative issues than Good reports) are true whenever we make a change. But that's perhaps a different issue...