Checking how various object factories perform in Django with respect to creating valid instances out of the box.
See the full write-up (with pretty diagram).
Factory libraries
The following factory libraries have been explored:
Disclosure: Factory Djoy is my factory library. It's a thin wrapper around Factory Boy which does the hard work.
Models
Two factories have been created with each factory library:
ItemFactory
: to create and save instances ofplant.models.Item
, a test model defined in the 'plant' app.UserFactory
: to create and save instances of the defaultdjango.contrib.auth
User Model.
Grading
Each factory is graded based on how its default configuration behaves.
The gradings are based on the definition of "valid". Valid instances are ones which will pass Django's
full_clean
and not raise aValidationError
. For example, using theItemFactory
a generated item passes validation with:item = ItemFactory() item.full_clean()
The gradings are:
- 🔴 RED - Factory creates invalid instances of the model and saves them to the database.
- 💛 YELLOW - Factory raises an exception and does not
save any instances. Preferably this would be a
ValidationError
, but I've also allowedIntegrityError
here. - 💚 GREEN - Factory creates multiple valid instances with no
invalid instances created or skipped. Running factory
n
times generatesn
valid instances.
Library | ItemFactory | UserFactory |
---|---|---|
Django Fakery | 🔴 RED | 💛 YELLOW |
Factory Boy | 🔴 RED | 🔴 RED |
Factory Djoy | 💛 YELLOW | 💚 GREEN |
Hypothesis[django] | 🔴 RED | 🔴 RED |
Mixer | 💚 GREEN | 💚 GREEN |
Model Mommy | 💛 YELLOW | 💚 GREEN |
For more detailed notes about each factory including some of the grey areas around grading please see Notes about each library in the full write-up.
To install and run:
$ git clone git@github.com:jamescooke/factory_audit.git
$ cd factory_audit
$ make venv
$ . venv/bin/activate
$ make install
$ cd factory_audit/
$ make test
Please add your own factory library and run it through the tests - pull requests very welcome.
Please help me write a better wrapper for Hypothesis! I hypothesise that it is possible!