Testing different factory libraries with Django
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
factory_audit
requirements
.gitignore
.travis.yml
LICENSE
Makefile
README.rst
test_mixer.sh

README.rst

Factory Audit

Checking how various object factories perform in Django with respect to creating valid instances out of the box.

See the full write-up (with pretty diagram).

https://travis-ci.org/jamescooke/factory_audit.svg?branch=master

Definitions

  • Factory libraries

    The following factory libraries have been explored:

    Disclosure: Factory Djoy is my factory library. It's a thin wrapper around Factory Boy which does the hard work.

  • Models

    Two factories have been created with each factory library:

    • ItemFactory: to create and save instances of plant.models.Item, a test model defined in the 'plant' app.
    • UserFactory: to create and save instances of the default django.contrib.auth User Model.
  • Grading

    Each factory is graded based on how its default configuration behaves.

    The gradings are based on the definition of "valid". Valid instances are ones which will pass Django's full_clean and not raise a ValidationError. For example, using the ItemFactory a generated item passes validation with:

    item = ItemFactory()
    item.full_clean()

    The gradings are:

    • πŸ”΄ RED - Factory creates invalid instances of the model and saves them to the database.
    • πŸ’› YELLOW - Factory raises an exception and does not save any instances. Preferably this would be a ValidationError, but I've also allowed IntegrityError here.
    • πŸ’š GREEN - Factory creates multiple valid instances with no invalid instances created or skipped. Running factory n times generates n valid instances.

Results

Library ItemFactory UserFactory
Django Fakery πŸ”΄ RED πŸ’› YELLOW
Factory Boy πŸ”΄ RED πŸ”΄ RED
Factory Djoy πŸ’› YELLOW πŸ’š GREEN
Hypothesis[django] πŸ”΄ RED πŸ”΄ RED
Mixer πŸ’š GREEN πŸ’š GREEN
Model Mommy πŸ’› YELLOW πŸ’š GREEN

For more detailed notes about each factory including some of the grey areas around grading please see Notes about each library in the full write-up.

Development

To install and run:

$ git clone git@github.com:jamescooke/factory_audit.git
$ cd factory_audit
$ make venv
$ . venv/bin/activate
$ make install
$ cd factory_audit/
$ make test

Contributions

Please add your own factory library and run it through the tests - pull requests very welcome.

Please help me write a better wrapper for Hypothesis! I hypothesise that it is possible!