unittest variant of our test suite
A couple of weeks ago, we started extending our test suite for our main Discord bot, @Python. I want to propose for us to switch to the
unittest framework instead of the
pytest framework we're currently using. I think that making this decision is especially important given the ambition we have with regards to requiring test coverage on
bot as well as
The main reason why I think we should switch is consistency. Both our
site repositories are central to our community and they are highly related as well. Therefore, I think it makes sense for us to use the same testing framework for both of them. Since the built-in testing utilities of Django are based on
unittest, the most logical test for that framework would be
I think that there are three major arguments in favor of that consistency:
Consistency in appearance and structure means our tests suites will be easer to read once someone gets used to that appearance and structure.
Contributors will not have to learn two frameworks in order to contribute to these two central repositories.
The output of both test suites (
bot) will be identical, making it easier to interpret the results you get back once you get used to that output.
There are other arguments as well, such reducing the number of dependencies, increasing the number of reviewers who feel confident enough with the framework to make reviews, and being able to have a similar/identical set-up for the linting & testing phase in the pipeline of both repositories.
Examples of output
When a test fails, there's a difference in output, though. However, both output the test values you put in (given that you use the
self.subTest context manager with
unittest like in the exposition file). For
unittest, it's in the
FAIL line, for
pytest, you need to look at the code context to see the test values used.
pytest gives slightly more context, but I think they both provide sufficient output to interpret the failing test: