Validation for active infractions by SebastiaanZ · Pull Request #278 · python-discord/site

SebastiaanZ · 2019-10-07T23:45:36Z

This pull request adds validation rules to the API to validate infractions based on the active field. This means that the API will now reject active infractions of types that should never be active (notes, warnings, and kicks) and only accept one active infraction per user for the other types of infractions (mute, ban, watch, superstar).

In addition, this pull requests adds a data migration to make sure the database itself is consistent with the new validation rules. Since this is a fairly complex data migration, I have added tests to make sure the migration works as expected. (Read more about that below.)

Finally, I've introduced a database-level UniqueConstraint to prevent a user from having multiple active infractions of the same type. I have also added tests to make sure this constraint works as expected.

Testing migrations

To test the effect of the migration file, I've created a subclass of django.test.TestCase, MigrationsTestCase, that reverts the test database to the migration just prior to the one of interest. We can then inject test data, apply the migration, and test if the migration had the desired effect.

Since a migration is really snapshot in time of the migration history, this also allows us to work with the database models as they were at those two time points in the migration history (directly before and directly after applying this migration), since otherwise these tests will fail in the future if we decide to change the models.

#273 This commit adds validation rules to the Infraction serializer that validate if a given infraction should be accepted based on its status of being considered `active`. If the validation fails, the API will reject the request and return a 400 status. Specifically, this validator checks that: - infractions that can never be active do not have `active=True` set; - a user can never receive a second active infraction of the same type. Tests have been added to `test_infractions.py` to ensure that the validators work as expected. This commit implements the first part of #273

#273 This commit adds a data migration to migrate active infractions that should not be active to inactive. There are two types of infractions that this migration will migrate to inactive: - Infractions of types that should never be active (e.g. notes) - Secondary active infractions if a given user already has an active infraction of the same type. Since this makes the migration file fairly complex, I have written tests to make sure the migration works as expected. In order to do this, I've subclassed `django.test.TestCase` to create a `MigrationsTestCase` that takes care of reverting the database back to a state prior to the migrations we want to test and injects test data before applying the migrations we want to test. For more information, see `pydis_site.apps.api.tests.migrations.base` This implements the last part of and closes #273

#273 This commits adds a UniqueConstraint for active infractions on a combination of the `user` and `type` field. This means that a user can only have one active infraction of a given type in the database at any time. I've also added tests to make sure that this behaves as expected.

MarkKoz

Migration testing is quite clever. I'm impressed. Nice work!
The infractions tests don't strike me as very DRY, though many other tests for the site are probably also guilty of that. Would it be feasible to come up with some functions to make it more DRY, at least for the new infraction tests you've written?
While your tests are already thorough, I noticed that MigrationsTestCase does not itself have any tests. Would it be feasible to come up with tests for that? Is it even worth it?

The migration files were generated and named before the migrations added by other pull requests. This caused the migration path to diverge. Since the migrations did not touch the same models, the solution was to rename the migration files to place them at the end of the migration history.

SebastiaanZ · 2019-10-18T14:59:04Z

The infractions tests don't strike me as very DRY, though many other tests for the site are probably also guilty of that. Would it be feasible to come up with some functions to make it more DRY, at least for the new infraction tests you've written?

(I still have to look into this.)

While your tests are already thorough, I noticed that MigrationsTestCase does not itself have any tests. Would it be feasible to come up with tests for that? Is it even worth it?

I'm not quite sure how to do that. I've thought about it, but I haven't come up with a way of testing it that isn't just "pretending to thoroughly test it". That is, testing some parts of it is fairly straight forward (the initial checks for set class variables, whether or not the hook methods are getting called) and I can add that, but that's not really testing the core of the TestCase yet.

The idea I'm playing with in my head is patching the hook methods to see whether they get called with the apps in the state we expect them to be in at that point in the migration history. That would confirm the thing does what it should: Give us states at specified points in the migration history. It's a bit of a hack, though, cause I'd need to find a migration point for which I can test "We've gone to precisely this state". Not sure if there's a better way.

#278 This commit applies the feedback given to PR #278. I have made a number of significant changes: - I have added tests for the `MigrationsTestCase` class, including a test to see if the test case travels to the specified points in the migration history. - The dictionary of infracted users in the active migration tests now uses informative strings as keys. This makes it easier to relate a specific test to the data it's testing. - I have simplified the migration query logic by replacing unneeded Q objects by simpler keyword arguments to `filter` method calls. - I have removed the custom validation logic for allowing only one active infraction of a given type per user and replaced it by a `UniqueTogetherValidator` with a custom validation error message. - `test_unique_constraint_accepts_active_infraction_after_inactive_ infraction` relied on an Error results instead of a Fail result to indicate an issue with the database constraint. Since a fail should lead to a Fail status, I've replaced it by a try-except block that signals a test failure if an `IntegrityError` was caught by it.

The last two lines in the docstring of `InfractionFactory` in the `test_active_infraction_migration.py` file were overindented by one space. I've removed the space. The docstring of the `test_loader_build_graph_gets_called_once` method in the `MigrationsTestCaseNoSideEffectsTests` calss of the `test_base.py` file was missing two words. Corrected.

The `test_returns_400_for_active_infractions_of_type_that_cannot_be_ active` test relied on the order in which the validation was done since it contained incompatible combinations of arguments. The test has been changed to make sure the data is valid except for the thing we actually want to test. I have also tried to improve the name of the test that tests the `test_unique_constraint_accepts_active_infraction_after_inactive_ infraction` test. It now includes the logic of what it does, but not the entire name of the test it's testing.

SebastiaanZ · 2019-10-25T13:51:17Z

Since other pull requests with migrations for the api app have since been merged, care should be taken when merging this. I'll resolve the created conflicts later, since it involves a bit more for this PR. In the mean time, reviews are still welcome.

Since other pull requests were merged that also included migrations for the API app, this PR needed to be updated to avoid conflicts in the migration history. In addition, the test files contained names of specific migration files that needed to be updated to the correct names after the merge resolution.

SebastiaanZ added 3 commits October 7, 2019 12:46

SebastiaanZ added area: backend Related to internal functionality and utilities area: API Related to or causes API changes status: stalled Something is blocking further progress labels Oct 7, 2019

SebastiaanZ commented Oct 7, 2019

View reviewed changes

Comment thread pydis_site/apps/api/serializers.py Outdated

Comment thread pydis_site/apps/api/serializers.py Outdated