Skip to content
This repository

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Browse code

[1.6.x] Fixed #20842 and #20845 - Added a note on order_by() and impr…

…oved prefetch_related() docs.

Backport of e8183a8 from master
  • Loading branch information...
commit 74205c4a3c587a9eedceef1b98cd4b0d529a7510 1 parent 77293f9
Daniele Procida authored August 02, 2013 timgraham committed August 04, 2013

Showing 1 changed file with 66 additions and 35 deletions. Show diff stats Hide diff stats

  1. 101  docs/ref/models/querysets.txt
101  docs/ref/models/querysets.txt
@@ -110,7 +110,7 @@ described here.
110 110
 
111 111
 .. admonition:: You can't share pickles between versions
112 112
 
113  
-    Pickles of QuerySets are only valid for the version of Django that
  113
+    Pickles of ``QuerySets`` are only valid for the version of Django that
114 114
     was used to generate them. If you generate a pickle using Django
115 115
     version N, there is no guarantee that pickle will be readable with
116 116
     Django version N+1. Pickles should not be used as part of a long-term
@@ -300,14 +300,30 @@ Be cautious when ordering by fields in related models if you are also using
300 300
 :meth:`distinct()`. See the note in :meth:`distinct` for an explanation of how
301 301
 related model ordering can change the expected results.
302 302
 
303  
-It is permissible to specify a multi-valued field to order the results by (for
304  
-example, a :class:`~django.db.models.ManyToManyField` field). Normally
305  
-this won't be a sensible thing to do and it's really an advanced usage
306  
-feature. However, if you know that your queryset's filtering or available data
307  
-implies that there will only be one ordering piece of data for each of the main
308  
-items you are selecting, the ordering may well be exactly what you want to do.
309  
-Use ordering on multi-valued fields with care and make sure the results are
310  
-what you expect.
  303
+.. note::
  304
+    It is permissible to specify a multi-valued field to order the results by
  305
+    (for example, a :class:`~django.db.models.ManyToManyField` field, or the
  306
+    reverse relation of a :class:`~django.db.models.ForeignKey` field).
  307
+
  308
+    Consider this case::
  309
+
  310
+         class Event(Model):
  311
+            parent = models.ForeignKey('self', related_name='children')
  312
+            date = models.DateField()
  313
+
  314
+         Event.objects.order_by('children__date')
  315
+
  316
+    Here, there could potentially be multiple ordering data for each ``Event``;
  317
+    each ``Event`` with multiple ``children`` will be returned multiple times
  318
+    into the new ``QuerySet`` that ``order_by()`` creates. In other words,
  319
+    using ``order_by()`` on the ``QuerySet`` could return more items than you
  320
+    were working on to begin with - which is probably neither expected nor
  321
+    useful.
  322
+
  323
+    Thus, take care when using multi-valued field to order the results. **If**
  324
+    you can be sure that there will only be one ordering piece of data for each
  325
+    of the items you're ordering, this approach should not present problems. If
  326
+    not, make sure the results are what you expect.
311 327
 
312 328
 There's no way to specify whether ordering should be case sensitive. With
313 329
 respect to case-sensitivity, Django will order results however your database
@@ -388,7 +404,7 @@ field names, the database will only compare the specified field names.
388 404
 
389 405
 .. note::
390 406
     When you specify field names, you *must* provide an ``order_by()`` in the
391  
-    QuerySet, and the fields in ``order_by()`` must start with the fields in
  407
+    ``QuerySet``, and the fields in ``order_by()`` must start with the fields in
392 408
     ``distinct()``, in the same order.
393 409
 
394 410
     For example, ``SELECT DISTINCT ON (a)`` gives you the first row for each
@@ -805,8 +821,8 @@ stop the deluge of database queries that is caused by accessing related objects,
805 821
 but the strategy is quite different.
806 822
 
807 823
 ``select_related`` works by creating a SQL join and including the fields of the
808  
-related object in the SELECT statement. For this reason, ``select_related`` gets
809  
-the related objects in the same database query. However, to avoid the much
  824
+related object in the ``SELECT`` statement. For this reason, ``select_related``
  825
+gets the related objects in the same database query. However, to avoid the much
810 826
 larger result set that would result from joining across a 'many' relationship,
811 827
 ``select_related`` is limited to single-valued relationships - foreign key and
812 828
 one-to-one.
@@ -835,39 +851,54 @@ For example, suppose you have these models::
835 851
             return u"%s (%s)" % (self.name, u", ".join([topping.name
836 852
                                                         for topping in self.toppings.all()]))
837 853
 
838  
-and run this code::
  854
+and run::
839 855
 
840 856
     >>> Pizza.objects.all()
841 857
     [u"Hawaiian (ham, pineapple)", u"Seafood (prawns, smoked salmon)"...
842 858
 
843  
-The problem with this code is that it will run a query on the Toppings table for
844  
-**every** item in the Pizza ``QuerySet``.  Using ``prefetch_related``, this can
845  
-be reduced to two:
  859
+The problem with this is that every time ``Pizza.__unicode__()`` asks for
  860
+``self.toppings.all()`` it has to query the database, so
  861
+``Pizza.objects.all()`` will run a query on the Toppings table for **every**
  862
+item in the Pizza ``QuerySet``.
  863
+
  864
+We can reduce to just two queries using ``prefetch_related``:
846 865
 
847 866
     >>> Pizza.objects.all().prefetch_related('toppings')
848 867
 
849  
-All the relevant toppings will be fetched in a single query, and used to make
850  
-``QuerySets`` that have a pre-filled cache of the relevant results. These
851  
-``QuerySets`` are then used in the ``self.toppings.all()`` calls.
  868
+This implies a ``self.toppings.all()`` for each ``Pizza``; now each time
  869
+``self.toppings.all()`` is called, instead of having to go to the database for
  870
+the items, it will find them in a prefetched ``QuerySet`` cache that was
  871
+populated in a single query.
  872
+
  873
+That is, all the relevant toppings will have been fetched in a single query,
  874
+and used to make ``QuerySets`` that have a pre-filled cache of the relevant
  875
+results; these ``QuerySets`` are then used in the ``self.toppings.all()`` calls.
  876
+
  877
+The additional queries in ``prefetch_related()`` are executed after the
  878
+``QuerySet`` has begun to be evaluated and the primary query has been executed.
852 879
 
853  
-The additional queries are executed after the QuerySet has begun to be evaluated
854  
-and the primary query has been executed. Note that the result cache of the
855  
-primary QuerySet and all specified related objects will then be fully loaded
856  
-into memory, which is often avoided in other cases - even after a query has been
857  
-executed in the database, QuerySet normally tries to make uses of chunking
858  
-between the database to avoid loading all objects into memory before you need
859  
-them.
  880
+Note that the result cache of the primary ``QuerySet`` and all specified related
  881
+objects will then be fully loaded into memory. This changes the typical
  882
+behavior of ``QuerySets``, which normally try to avoid loading all objects into
  883
+memory before they are needed, even after a query has been executed in the
  884
+database.
  885
+
  886
+.. note::
860 887
 
861  
-Also remember that, as always with QuerySets, any subsequent chained methods
862  
-which imply a different database query will ignore previously cached results,
863  
-and retrieve data using a fresh database query. So, if you write the following:
  888
+    Remember that, as always with ``QuerySets``, any subsequent chained methods
  889
+    which imply a different database query will ignore previously cached
  890
+    results, and retrieve data using a fresh database query. So, if you write
  891
+    the following:
864 892
 
865  
-    >>> pizzas = Pizza.objects.prefetch_related('toppings')
866  
-    >>> [list(pizza.toppings.filter(spicy=True)) for pizza in pizzas]
  893
+        >>> pizzas = Pizza.objects.prefetch_related('toppings')
  894
+        >>> [list(pizza.toppings.filter(spicy=True)) for pizza in pizzas]
867 895
 
868  
-...then the fact that ``pizza.toppings.all()`` has been prefetched will not help
869  
-you - in fact it hurts performance, since you have done a database query that
870  
-you haven't used. So use this feature with caution!
  896
+    ...then the fact that ``pizza.toppings.all()`` has been prefetched will not
  897
+    help you. The ``prefetch_related('toppings')`` implied
  898
+    ``pizza.toppings.all()``, but ``pizza.toppings.filter()`` is a new and
  899
+    different query. The prefetched cache can't help here; in fact it hurts
  900
+    performance, since you have done a database query that you haven't used. So
  901
+    use this feature with caution!
871 902
 
872 903
 You can also use the normal join syntax to do related fields of related
873 904
 fields. Suppose we have an additional model to the example above::
@@ -920,7 +951,7 @@ additional queries on the ``ContentType`` table if the relevant rows have not
920 951
 already been fetched.
921 952
 
922 953
 ``prefetch_related`` in most cases will be implemented using a SQL query that
923  
-uses the 'IN' operator. This means that for a large QuerySet a large 'IN' clause
  954
+uses the 'IN' operator. This means that for a large ``QuerySet`` a large 'IN' clause
924 955
 could be generated, which, depending on the database, might have performance
925 956
 problems of its own when it comes to parsing or executing the SQL query. Always
926 957
 profile for your use case!

0 notes on commit 74205c4

Please sign in to comment.
Something went wrong with that request. Please try again.