From 1f79f23f7551c9d1f49b9a74d8816dff7cb0b7cb Mon Sep 17 00:00:00 2001 From: Marc Tamlyn Date: Sun, 14 Sep 2014 11:41:08 +0100 Subject: [PATCH 1/2] Add a draft for indexes. --- drafts/indexes.rst | 116 +++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 116 insertions(+) create mode 100644 drafts/indexes.rst diff --git a/drafts/indexes.rst b/drafts/indexes.rst new file mode 100644 index 0000000..ac25b69 --- /dev/null +++ b/drafts/indexes.rst @@ -0,0 +1,116 @@ +Advanced indexes +================ + +:Created: 2014-09-14 +:Author: Marc Tamlyn +:Status: Draft + +Abstract +======== + +This DEP aims to rework how Django handles index definitions. This will allow +Django to support more complex indexes on the relevant backends (most notably +PostgreSQL), such as functional indexes and indexes with different algorithms +(e.g. ``GiST``). + +Current Implementation +====================== + +At present, Django has several APIs for creating indexes, some of which are +implicit: ``primary_key``, ``db_index`` and ``unique`` on a ``Field()``, +``unqiue_together`` and ``index_together`` in ``class Meta``, and +``spatial_index`` on a ``Field()`` in GIS. + +There is additional complexity in that handling of these APIs is done in both +``DatabaseCreation`` and in ``SchemaEditor`` at present. Given that +``DatabaseCreation`` is deprecated, and set to be removed in Django 2.0, we +will only consider how ``SchemaEditor`` handles indexes for now. + +For the ``primary_key`` and ``unique`` on fields it is not necessary to create +indexes by hand on any of the supported backends as the database handles these +itself. + +``unique`` and ``unique_together`` are handled by using the SQL ``CONSTRAINT`` +rather than ``INDEX``, or by using the ``UNIQUE`` keyword when creating the +column or table at the same time. + +So that leaves ``db_index`` and ``index_together`` which use a shared API on +``SchemaEditor``, encapsulated in the ``_create_index_sql`` statement. + +``spatial_index`` in gis duplicates similar code in customised ``SchemaEditor`` +subclasses for the Postgis backend, and uses somewhat different syntax for the +sqlite backend. As far as I can tell ``spatial_index`` is currently not handled +by any ``SchemaEditor`` for MySQL or Oracle, despite the old +``DatabaseCreation`` classes handling it correctly (and very similarly to +Postgis. + +Motivation +========== + +There are new fields being introduced in ``contrib.postgres`` which have +similar issues to the rationale for ``spatial_index`` - a ``BTree`` index is +either insufficent, inefficient or does not work and an alternative index type +is needed for that field. A field should be able to declare the type of index +which is appropriate to it. + +In addition, both postgres and oracle support powerful functional indexes (and +mysql support indexing on the first few characters of a string), which allow +significant performance benefits for certain queries. Of course, it is possible +to create these indexes by hand if you know what you are doing using ``RunSQL`` +based migrations. + +PostgreSQL also support partial indexes. + +Specification +============= + +Introduce ``django.db.models.indexes`` with deconstructable ``Index`` classes. +These will include different index algorithms such as ``BTree`` and ``Hash``, +and also ``FunctionalIndex(expression)``. + +``Index()`` classes will be able to define their own SQL using the ``as_sql`` +pattern similar to that used by ``Lookup()``, but being passed the +``SchemaEditor`` class instead of a compiler. By default, this will defer to +the ``SchemaEditor`` class, but it allows non-standard index creation such as +that for spatialite (using a ``SELECT`` statement and a function) without +having to subclass ``SchemaEditor``. + +``Index()`` subclasses will also have a ``supports(backend)`` method which +raises an error if the backend does not support that index type. + +``Field(db_index=)`` will support either the value ``True``, or an index class +(e.g. ``IntegerField(db_index=indexes.Hash)``). + +``Field`` will gain a method ``get_default_index`` which will be called when +``db_index=True`` is passed to the ``__init__``. This will be used in place of +``spatial_index`` in gis, which will be supported with deprecation warnings. + +Introduce ``Meta.indexes`` for other indexes. This will contain a list of +``Index()`` instances. ``index_together`` will continue to be supported but +will be inserted into ``Meta.indexes`` and handled only there by +``SchemaEditor``. + +Example:: + + class River(models.Model): + name = models.CharField(max_length=255) + slug = models.SlugField(unique=True) + route = gis.LineStringField(db_index=True) + length = models.FloatField() + average_capacity = models.FloatField() + + class Meta: + indexes = [ + models.BTree(fields=['slug', 'name']), + models.FunctionalIndex(expression(F('length') * F('average_capacity')), + type=models.BTree), + ] + +Backwards compatibility +======================= + +``spatial_index`` will be deprecated as setting ``db_index`` on a spatial field +will be sufficient. + +The main concern is whether to fully support the new API on +``DatabaseCreation`` as well as in ``SchemaEditor``. From 8000663433f559bb2e79ac637d1e4a342cd192a8 Mon Sep 17 00:00:00 2001 From: Marc Tamlyn Date: Tue, 16 Sep 2014 09:48:49 +0100 Subject: [PATCH 2/2] Fix typo, change spec of `supports(backend)`. --- drafts/indexes.rst | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/drafts/indexes.rst b/drafts/indexes.rst index ac25b69..ea8afe5 100644 --- a/drafts/indexes.rst +++ b/drafts/indexes.rst @@ -76,7 +76,8 @@ that for spatialite (using a ``SELECT`` statement and a function) without having to subclass ``SchemaEditor``. ``Index()`` subclasses will also have a ``supports(backend)`` method which -raises an error if the backend does not support that index type. +returns ``True`` or ``False`` depending on whether the backend supports that +index type. ``Field(db_index=)`` will support either the value ``True``, or an index class (e.g. ``IntegerField(db_index=indexes.Hash)``). @@ -102,7 +103,7 @@ Example:: class Meta: indexes = [ models.BTree(fields=['slug', 'name']), - models.FunctionalIndex(expression(F('length') * F('average_capacity')), + models.FunctionalIndex(expression=(F('length') * F('average_capacity')), type=models.BTree), ]