django-fast-sync

A Django app which helps you to import a lot of data with raw SQL

Important: This library only supports Postgres (common for Django projects)!

Example

Let's say you have this model in the app 'booking':

class Category(models.Model):
  category_id = models.IntegerField()
  name = models.CharField(max_length=255)

and you want to import about 1000000 categories fast. You get a list of dictionaries from you REST API or different database which looks like this:

[
  {
    'category_id': 2,
    'name': 'light'
  },
  {
    'category_id': 3,
    'name': 'heavy'
  },
  ...
]

Then you can import my package and just use:

from django_fast_sync.fast_sync import FastSync

fast_sync = FastSync(my_data_list, ['category_id'], 'booking_category')
# FastSync(list_of_data, list_of_primary_field_names, table_name)
and then run
fast_sync.start_sync()

Note: You can also do it in a transaction (which makes it more faster)

from django.db.transaction import *

from django_fast_sync.fast_sync import FastSync

fast_sync = FastSync(my_data_list, ['category_id'], 'booking_category')
# FastSync(list_of_data, list_of_primary_field_names, table_name)
and then run
set_autocommit(False)
fast_sync.start_sync()
commit()
set_autocommit(True)