Refactor tests to pytest

[elfjes]

Following #326 I've refactored some more tests to make better use of pytest's capabilities

• added some more global fixtures to request a project:
  • either a clean or a filled project (eg: siouxfalls)
  • with a scope of session or test (to choose between long lived and short lived projects)
• Refactored some tests files.
• IMHO unit tests should be deterministic (ie, no randomization). There's a place for randomization in testing (ie. property based testing, using hypothesis for example) but that requires a structured/systematic approach. Providing a whole bunch of randomized inputs to see if a piece of code succeeds under many different input conditions. Not just a single random value and call it a day
• I've removed TestFieldEditor.test_check_completeness as all it did was test the database (running a query and see the result). It didn't test any aequilibrae-code

Take especially note of test_field_editor.py which uses some new (ie. not previously introduced here) constructs to remove in-test looping, conditional tests and provide better test isolation:

• test parametrization: using pytest.mark.parametrize to introduce new fixtures or override existing ones (first argument) and provide a list of test cases (values of these fixtures)
• fixture parametrization: using pytest.fixture(params=[...]) to create multiple versions of a fixture. Every test is run with all versions of the fixture (unless the fixture is overridden, eg using pytest.mark.parametrize)
• @pytest.mark.usefixtures to use a fixture for a test without specifying it as an argument

After these changes running the tests/aquilibrae/project directory (on my machine) took 28 seconds, vs 48 seconds previously (~70% faster). There's room for more improvements given that I only refactored a couple of files

[pedrocamargo]

This uses stuff that @jamiecook is probably more comfortable with reviewing than me.

[jamiecook]

Would it make sense to now move this next to the project fixture in the conftest.py?

[elfjes]

I'm a little bit hesitant to do this. Having a session scoped project inside a class is one thing, having a global session scoped project is quite another. Currently, if a test behaves poorly and doesn't clean up after itself, it can only affect the other tests inside the class. In case of a global fixture that's used throughout the whole test suite there's an increased risk of tests affecting other tests in unpredictable ways.

Also, in the TestFieldEditor class we know the only part of the Project that is mutated is the sqlite database, so we also only need to restore that. For the general case, we don't know that, so we also cannot make the assumption that restoring the database is sufficient. So we'd need to restore the whole project, which is a significantly more expensive operation (and is effectively what we do with create_project_session already)

[jamiecook]

Hmmm - my understanding of scope is that if it's defined with session scope - it won't matter where it's defined - there will only be one of them for the entire test suite. Or are you relying here on the fact that each of the fixtures is namespaced to the class that is invoking it? I think that makes sense - although it could therefore be tagged as "module" level scope.

[jamiecook]

If what I said lines up with your understanding - i'm happy to resolve this conversation and approve.

[elfjes]

Yes, you're right. Since it's inside a class, it's effectively a module or even class scoped fixture. I think it makes sense to label it as such, to avoid confusion.

[elfjes]

I've process the latest comments. If there's no other remarks, I think this can be merged :)