Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
[WIP] Disable/enable constraints for required tables only #5801
The idea is to speed up tests on mssql by disabling/enabling constraints only for those tables that have actually changed.
I believe this could give roughly two orders of magnitude better performance for constraint checking. The reason is that currently mssql checks every table for every disable/enable constraint checks call. This is around 1500 tables * 225 calls, that is roughly 330 000 disable/enable cycles. With the patch, the amount of disable/enable cycles is 473. (EDIT: Actually, currently there are roughly 750 calls to disable constraints, of which around 500 are from a loop in tests.serializers.test_data).
With this approach, it is possible that a constraint in some other table might fail if we delete a row from a referenced table, but I don't believe this is a problem for Django's test suite, and it can be a problem in fixture loading only in extreme cases (where one updates an already existing row's column pointed by a row in some other table). Of course, this could be fixed simply by adding the other table to the list of tables to disable/enable constraints for.
To get the speed benefit for mssql, the mssql backend should only disable/enable constraints for the tables required. That is, to see the full speed benefit of this change, the mssql backend needs to be updated.
This is an alternate to #5438.
I am not intending to continue work on this pull request in the immediate future - others are welcome to take over my work.
I've ran the test suite with django-mssql and this patch (mssql requires a couple of minor changes for the tests to run).
The results are very promising, I got a runtime of 1800 seconds (with profiling enabled). I'm currently running the test suite without this patch to compare the runtime, I'll report the results tomorrow.
The patch requires a bit more work, as there are some new failures. I don't believe those to be anything fundamental, instead they just show a couple of places where we need to be a bit more careful and make sure the tables passed to disable_constraints() actually exist.
I think I know the real reason for the slowdown.
Django-mssql treats empty list in table_names argument of check_constraints() similarly to None. That is, when Django requests the backend to check constraints for no tables, django-mssql checks all tables. The check for empty list happens a lot in 1.8, so for that reason the tests are very slow.
I'll post a patch to django-mssql separately, and I'll open a separate PR for the work I've done on 1.8.