django-roesti

django-roesti provides a HashedModel, a Django model whose primary key is a hash of its contents. This enables efficient use of the database when you need to reference data by its identity while minimizing round trips to the database, or encounter the same data often, and need to ensure that it exists in your database without duplicating it.

Hashes are generated by converting values into Python immutable types and then generating the md5 hash of its pickled value.

HashedModels that maintain references to other HashedModels are supported.

django-roesti has been tested with Python 3.6 and Django 1.9.

Pull requests welcome.

Roesti?

A Rösti is a Swiss potato pancake, similar to hash browns. Get it?

Installation

Install from PyPI:

pip install django-roesti

... and add to your INSTALLED_APPS:

INSTALLED_APPS = [
    # ...
    'roesti',
]

Usage Examples

Hashes are calculated from the fields specified in hash_fields. Foreign keys should be hashed on the ID value.

For example, given these models:

from roesti.models import HashedModel


class TestModel(HashedModel):
    hash_fields = ['char_field_1', 'integer_field_1']

    char_field_1 = models.CharField(max_length=32)
    integer_field_1 = models.IntegerField()


class TestReferencesModel(HashedModel):
    hash_fields = ['test_model_1_id', 'test_model_2_id', 'integer_field_1']

    test_model_1 = models.ForeignKey(TestModel)
    test_model_2 = models.ForeignKey(TestModel)
    integer_field_1 = models.IntegerField()

You may create new or retrieve existing model instances with the ensure function. ensure accepts either dictionaries or model instances.

my_models = TestModel.objects.ensure([{
    'char_field_1': 'field 1 value 1',
    'integer_field_1': 1
}, {
    'char_field_1': 'field 1 value 2',
    'integer_field_1': 2
}, {
    'char_field_1': 'field 1 value 3',
    'integer_field_1': 3
}])

my_reference_models = TestReferencesModel.objects.ensure({
    'test_model_1': {
        'char_field_1': 'field 1 value 1',
        'integer_field_1': 1
    },
    'test_model_2': {
        'char_field_2': 'field 1 value 2',
        'integer_field_1': 2
    },
    'integer_field_1': 3
})

Reverse relationships may be used to calculate the item's hash, and passed to the ensure method. For instance, this implements an ordered list of items. Note the use of items in hash_fields for TestOrderedList:

class TestOrderedList(HashedModel):
    hash_fields = ('name', 'items')
    name = models.TextField()


class TestItemDetails(HashedModel):
    hash_fields = ('text',)
    text = models.TextField()


class TestOrderedListItem(HashedModel):
    hash_fields = ('lst_id', 'order', 'details',)
    lst = models.ForeignKey(TestOrderedList, related_name='items')
    order = models.IntegerField()
    details = models.ForeignKey(TestItemDetails)

    instances = TestOrderedList.objects.ensure([{
        'name': 'My list 1',
        'items': [{
            'order': index,
            'details': {
                'text': '1 Item %d' % index
            }
        } for index in range(10)]
    }, {
        'name': 'My list 2',
        'items': [{
            'order': index,
            'details': {
                'text': '2 Item %d' % index
            }
        } for index in range(10)]
    }, {
        'name': 'My list 3',
        'items': [{
            'order': index,
            'details': {
                'text': '3 Item %d' % index
            }
        } for index in range(10)]
    }])

There is also a HashedList model for managing ordered lists of hashed items. To use, create the corresponding HashedModel and a mapping table. Note that if you need to store references to the list in a HasedModel, as in the above example, you should not used HashedList.

class TestItem(HashedModel):
    hash_fields = ('text',)
    text = models.TextField(blank=True, null=True)


class TestListItem(HashedListItemModel):
    item = models.ForeignKey(TestItem)


items = [
    TestItem(text='Item %d' % index)
    for index in range(10)
]
my_list = HashedList.objects.ensure_list(TestItem, items)