Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Ticket 20877 performance documentation (work in progress) #1463

Merged
merged 1 commit into from

10 participants

@evildmp
Collaborator

Work in progress. Please comment here, or at https://code.djangoproject.com/ticket/20877, or on the developers email list.

docs/index.txt
@@ -228,6 +228,16 @@ regions:
* :doc:`"Local flavor" <topics/localflavor>`
* :doc:`Time zones </topics/i18n/timezones>`
+Performance and optimization
+============================
+
+There are a variety of approaches, techniques, tools and helpers that can help
+get your code running more efficiently - faster, and using fewer system
+resources.
+
+* :doc:`Performance and optimization overview <topics/performance>`
@evildmp Collaborator
evildmp added a note

Why is Sphinx saying:

checking consistency... /home/daniele/django/docs/topics/performance.txt:: WARNING: document isn't included in any toctree

when it is clearly included here?

@MarkusH Collaborator
MarkusH added a note

You should add the file to docs/topics/index.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((63 lines not shown))
+Databases
+=========
+
+:doc:`Database optimization </topics/db/optimization>`
+------------------------------------------------------
+
+Django’s database layer provides various ways to help developers get the most
+out of their databases. The database optimisation document gathers together
+links to the relevant documentation, and adds various tips, organized under a
+number of headings that outline the steps to take when attempting to optimize
+your database usage.
+
+Other database-related tips
+---------------------------
+
+:ref:`Persistent database connections <persistent-database-connections>`
@evildmp Collaborator
evildmp added a note

Also in howto/deployment/checklist as a potential performance enhancer - is it required in both places?

@MarkusH Collaborator
MarkusH added a note

In this case I rather go with both links (on the deployment checklist and on this optimization overview page). First, because configuring persistent connections is most likely done on production or staging systems. Second, the overview page should not miss such an important and powerful topic.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((146 lines not shown))
+Alternative template systems vary in the extent to which they share Django's
+templating language. If the bottlenecks in your Django project seem to lie in
+the template system, a third-party alternative may be the answer.
+
+Alternative versions of third-party libraries
+---------------------------------------------
+
+When you are using a Python library, it's always worth checking whether a
+version has been provided in another, faster language.
+
+For example `ElementTree <http://effbot.org/zone/element-index.htm>`_ - a
+library for handling data trees, such as HTML or XML documents - is also
+implemented in C as `cElementTree <http://effbot.org/zone/celementtree.htm`_;
+it provides exactly the same API, but it can be many times faster. So::
+
+ try:
@MarkusH Collaborator
MarkusH added a note

That's an indention error

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/index.txt
@@ -228,6 +228,16 @@ regions:
* :doc:`"Local flavor" <topics/localflavor>`
* :doc:`Time zones </topics/i18n/timezones>`
+Performance and optimization
+============================
+
+There are a variety of approaches, techniques, tools and helpers that can help
+get your code running more efficiently - faster, and using fewer system
@carljm Owner
carljm added a note

Grammatically, the dash here should be a colon instead: "... more efficiently: faster, and..."

@evildmp Collaborator
evildmp added a note

If it's good enough for Jane Austen, it's good enough for me. Joking aside, it's a fairly standard usage - but if you really object I'll change it.

@carljm Owner
carljm added a note

The general rule is to use a colon when you're filling in fairly mundane details to clarify or extend what came before, whereas the dash is more dramatic, used to set off a surprising or important bit of additional information. (See http://www.quickanddirtytips.com/education/grammar/dashes-versus-colons?page=all and http://www.fas.harvard.edu/~wricntr/documents/GP.html). I think this is pretty clearly the former case, and the dash reads badly to me - but it's your bikeshed, paint it as you like! :-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
@@ -0,0 +1,204 @@
+===========================
+Perfomance and optimization
+===========================
+
+This document provides an overview of approaches, techniques, tools and helpers
+that can help get your Django code running more efficiently - faster, and using
@carljm Owner
carljm added a note

colon instead of dash here as well

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
@@ -0,0 +1,204 @@
+===========================
+Perfomance and optimization
+===========================
+
+This document provides an overview of approaches, techniques, tools and helpers
+that can help get your Django code running more efficiently - faster, and using
+fewer system resources.
+
+Introduction
+============
+
+Generally one's first concern when writing to code is to write code that
@carljm Owner
carljm added a note

s/writing to code/writing code/

Though I think you could just as well leave out the entire phrase "when writing [to] code" and you wouldn't lose anything.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((22 lines not shown))
+=======
+
+Often it is expensive - resource-hungry and slow - to compute a value, so there
+can be huge benefit in saving the value to a quickly accessible cache ready for
+the next time it's required.
+
+It's a sufficiently significant and powerful technique that Django includes a
+comprehensive caching framework, as well as numerous other opportunities to
+make use of caching.
+
+:doc:`The caching framework </topics/cache>`
+--------------------------------------------
+
+Django comes with a robust cache system that lets you save dynamic pages so
+they don't have to be calculated for each request. For convenience, Django
+offers different levels of cache granularity: You can cache the output of
@carljm Owner
carljm added a note

s/You/you/

@timgraham Owner

self-defeating

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((44 lines not shown))
+Beyond the caching framework, Django offers other smaller pieces of caching functionality.
+
+:class:`django.utils.functional.cached_property`
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+It's common to have to call a class instances's method more than once. If
+that function is expensive, then doing so is wasteful.
+
+Using the ``@cached_property`` decorator saves the value returned by a
+property; the next time the function is called on that instance, it will return
+the saved value rather than re-computing it.
+
+:class:`django.contrib.staticfiles.storage.CachedStaticFilesStorage`
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Caches :doc:`static files </ref/contrib/staticfiles>`.
@carljm Owner
carljm added a note

I think this description is misleading (though I see that unfortunately similar wording is used at https://docs.djangoproject.com/en/1.5/ref/contrib/staticfiles/#django.contrib.staticfiles.storage.CachedStaticFilesStorage). CachedStaticFilesStorage doesn't really cache anything itself in any usual sense of the term; what it does is append a content-dependent tag to the filename so that you can safely set far-futures expires (i.e. allow clients to cache the files long-term) with confidence that when the file contents change, the filename will change, thus automatically expiring the cache.

For brief wording here, I'd suggest something like "CachedStaticFilesStorage appends a content-dependent tag to the filenames of static assets to make it safe for browsers to cache them long-term without missing future changes."

@timgraham Owner

-1 on including specific recommendations here (although I'd agree django-debug-toolbar is the defacto standard here). I'd prefer to mention the topic in general and link to a wiki page with any specific recommendations. Historically Django has taken a strong stance against endorsing or blessing any 3rd party apps.

@akaariai Collaborator

There is a tool that should be mentioned:
python -m cProfile -s cumulative manage.py runserver --nothreading > prof.txt

Hit the server multiple times (with web benchmarking tool, or just browser + reload).

This is easily the most important tool for me.

@evildmp Collaborator
evildmp added a note

Comments on this @timgraham?

@timgraham Owner

I'm not familiar with it. maybe we can get @akaariai to draft some text. Not a blocker for committing a first version of this I think.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((55 lines not shown))
+
+:class:`django.contrib.staticfiles.storage.CachedStaticFilesStorage`
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Caches :doc:`static files </ref/contrib/staticfiles>`.
+
+# other examples?
+
+Databases
+=========
+
+:doc:`Database optimization </topics/db/optimization>`
+------------------------------------------------------
+
+Django’s database layer provides various ways to help developers get the most
+out of their databases. The database optimisation document gathers together
@carljm Owner
carljm added a note

I think Django's docs generally use American spelling? Thus "optimization"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((139 lines not shown))
+---------------------------------
+
+Most developers find Django's template system perfectly adequate. Some consider
+third-party alternatives, such as `Jinja2 <http://jinja.pocoo.org/docs/>`_ or
+`Cheetah <http://www.cheetahtemplate.org/>`_ to be an improvement, particularly
+when it comes to speed.
+
+Alternative template systems vary in the extent to which they share Django's
+templating language. If the bottlenecks in your Django project seem to lie in
+the template system, a third-party alternative may be the answer.
+
+Alternative versions of third-party libraries
+---------------------------------------------
+
+When you are using a Python library, it's always worth checking whether a
+version has been provided in another, faster language.
@carljm Owner
carljm added a note

I think @alex would probably point out that language implementations, not languages, are faster or slower :-) (i.e. PyPy can sometimes make Python code run faster than equivalent C code). Maybe just say "...worth checking whether another implementation is available that might be faster."

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((216 lines not shown))
+The result is that often you'll find that you can simply swap out one component
+for another.
+
+Alternatives to Django components
+---------------------------------
+
+Most developers find Django's template system perfectly adequate. Some consider
+third-party alternatives, such as `Jinja2 <http://jinja.pocoo.org/docs/>`_ or
+`Cheetah <http://www.cheetahtemplate.org/>`_ to be an improvement, particularly
+when it comes to speed.
+
+Alternative template systems vary in the extent to which they share Django's
+templating language. If the bottlenecks in your Django project seem to lie in
+the template system, a third-party alternative may be the answer.
+
+Alternative versions of third-party libraries
@timgraham Owner

I think we should keep this limited to Django specific issues as much as possible and not veer off into general Python performance or endorsing 3rd party libraries unless it's a defacto standard (+1 to django-debug toolbar).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((53 lines not shown))
+It's no good just guessing or assuming where the inefficiencies lie in your
+code.
+
+`django-debug-toolbar
+<https://github.com/django-debug-toolbar/django-debug-toolbar/>`_ is a very
+handy tool that provides insights into what your code is doing and how much
+time it spends doing it.
+
+Third-party plugins are also available for the toolbar; particularly useful for
+studying the performance of your code are the `Cache Panel
+<https://github.com/lincolnloop/django-cache-panel>`_ and `Template Timings
+<https://github.com/orf/django-debug-toolbar-template-timings>`_ modules.
+
+# other tools
+
+# should we mention front-end performance analysis tools too?

I think this would be cool. I'd include NewRelic as well. It has amazing profiling capabilities. And it's awesome to debug frontend performance issues.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@catalanojuan catalanojuan commented on the diff
docs/topics/performance.txt
((62 lines not shown))
+studying the performance of your code are the `Cache Panel
+<https://github.com/lincolnloop/django-cache-panel>`_ and `Template Timings
+<https://github.com/orf/django-debug-toolbar-template-timings>`_ modules.
+
+# other tools
+
+# should we mention front-end performance analysis tools too?
+# * pingdom
+# * Yahoo's Yslow
+# * Google PageSpeed
+
+Get things right from the start
+-------------------------------
+
+Some work in optimization involves tackling performance shortcomings, but some
+of the work can simply be built in to what you'd do anyway, as part of the good

shouldn't it be 'built-in'? That way sounds like "built into"

@timgraham Owner

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((80 lines not shown))
+
+In this respect Python is an excellent language to work with, because solutions
+that look elegant and feel right usually are the best performing ones. As with
+most skills, learning what 'looks right' takes practice, but some useful
+guidelines are:
+
+Work at the appropriate level
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Django offers many possible different ways of approaching things, but don't do
+things just because you can. For example, you might find that you could
+calculate the same thing - the number of items in a collection, perhaps - in a
+``QuerySet``, in Python or in a template.
+
+However, almost always it will be faster to do this work closer to the bottom
+of the system, where these things are dealt with in their rawest states rather

with in == within?

@shaib Collaborator
shaib added a note

No. (things are dealt with) (in their rawest states). But this whole para looks a little awkward to me -- I'd try something like

However, it will almost always be faster to do such work closer to the source of data, where these things are dealt with in their rawest form rather than through multiple levels of abstraction

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((145 lines not shown))
+
+For convenience, Django offers different levels of cache granularity: you can
+cache the output of specific views, you can cache only the pieces that are
+difficult to produce, or even an entire site.
+
+Other opportunities for caching
+-------------------------------
+
+Beyond the caching framework, Django offers other smaller pieces of caching
+functionality.
+
+:class:`django.utils.functional.cached_property`
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+It's common to have to call a class instances's method more than once. If
+that function is expensive, then doing so is wasteful.
@funkybob
funkybob added a note

"can be" wasteful

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((147 lines not shown))
+cache the output of specific views, you can cache only the pieces that are
+difficult to produce, or even an entire site.
+
+Other opportunities for caching
+-------------------------------
+
+Beyond the caching framework, Django offers other smaller pieces of caching
+functionality.
+
+:class:`django.utils.functional.cached_property`
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+It's common to have to call a class instances's method more than once. If
+that function is expensive, then doing so is wasteful.
+
+Using the ``@cached_property`` decorator saves the value returned by a
@funkybob
funkybob added a note

Perhaps best to mention explicitly it works just like the property decorator -- that is, your method is no longer a method, and can take no arguments.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((111 lines not shown))
+
+Generally speaking, the most appropriate level for the job is the lowest-level one that it is comfortable to code for.
+
+.. note::
+
+ The example above is merely illustrative.
+
+ Firstly, in a real-life case you need to consider what is happening before
+ and after your count to work out what's an optimal way of doing it *in that
+ particular context*. The database optimization documents describes :ref:`a
+ case where counting in the template would be better
+ <overuse_of_count_and_exists>`.
+
+ Secondly, there are other options to consider: in a real-life case, ``{{
+ my_bicycles.count }}``, which invokes the ``QuerySet`` ``count()`` method
+ directly from the template, might be the appropriate choice.
@shaib Collaborator
shaib added a note

I find it hard to imagine where it is better to invoke operations from the template rather than from Python code; and I think as a general rule, we should advocate against invoking non-trivial operations (especially database queries) from the templates. This seems like a direct violation of MVC principles.

@evildmp Collaborator
evildmp added a note

In this case, we have a fairly efficient way for a template to invoke a fast database operation directly. We don't always know in a view which attributes of an object a template might want to make use of, and here the easy access to count() in the template - possibly by the template author rather than the person responsible for creating the view - might make it appropriate to do it this with rather than through re-writing the view.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((144 lines not shown))
+doesn't need to be calculated for each request.
+
+For convenience, Django offers different levels of cache granularity: you can
+cache the output of specific views, you can cache only the pieces that are
+difficult to produce, or even an entire site.
+
+Other opportunities for caching
+-------------------------------
+
+Beyond the caching framework, Django offers other smaller pieces of caching
+functionality.
+
+:class:`django.utils.functional.cached_property`
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+It's common to have to call a class instances's method more than once. If
@shaib Collaborator
shaib added a note

Doesn't apply to general methods, only to properties.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((134 lines not shown))
+
+It's a sufficiently significant and powerful technique that Django includes a
+comprehensive caching framework, as well as numerous other opportunities to
+make use of caching.
+
+:doc:`The caching framework </topics/cache>`
+--------------------------------------------
+
+Django's :doc:`caching framework </topics/cache>` offers very significant
+opportunities for performance gains, by saving dynamic content so that it
+doesn't need to be calculated for each request.
+
+For convenience, Django offers different levels of cache granularity: you can
+cache the output of specific views, you can cache only the pieces that are
+difficult to produce, or even an entire site.
+
@shaib Collaborator
shaib added a note

It may (or may not) be useful to mention some 3rd-party uses of the cache framework, such as http://djangosnippets.org/snippets/492/ and other snippets referred from there. I have a client using a mini-framework inspired by those to great advantage (sadly, I am not at liberty to open-source it...)

@timgraham Owner

so it's ready

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((177 lines not shown))
+----------------------
+
+*Laziness* is the strategy complementary to caching. Caching avoids
+recomputation by saving results; laziness saves effort trying to avoid work in
+the first place. That is, laziness means not doing anything until it has to be
+done, because it may not turn out to be necessary after all, and if something
+does have to be done, it will do its best to do it only once.
+
+Python makes very good use of this principle. In fact Python is quite a lazy
+language, and one of the best ways to make use of laziness in Python is a
+negative one: don't do things that prevent Python being lazy. Similarly, Django
+likes to be lazy.
+
+A typical Django example can be found in the evaluation of ``QuerySets``.
+
+# other examples in Django?
@shaib Collaborator
shaib added a note

Some, but I'm not sure they're relevant to performance -- settings and translations come to mind.

However, translations bring to mind a note which may be worth making -- laziness is not just about performance; it is an important abstraction mechanism. See http://www.cse.chalmers.se/~rjmh/Papers/whyfp.html

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/index.txt
@@ -232,6 +232,16 @@ regions:
* :doc:`"Local flavor" <topics/localflavor>`
* :doc:`Time zones </topics/i18n/timezones>`
+Performance and optimization
+============================
+
+There are a variety of approaches, techniques, tools and helpers that can help
@timgraham Owner

this is a bit abstract -- what's the difference between "approaches" and "techniques" as well as "tools" and "helpers"? missing a comma before "and" in the list in any case.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/db/optimization.txt
@@ -88,7 +88,8 @@ of parentheses, but will call callables automatically, hiding the above
distinction.
Be careful with your own custom properties - it is up to you to implement
-caching.
+caching when required, for example using the
+:class:`django.utils.functional.cached_property` decorator.
@timgraham Owner

I'd do a short reference with ~ here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/db/optimization.txt
@@ -111,8 +112,8 @@ For instance:
* At the most basic level, use :ref:`filter and exclude <queryset-api>` to do
filtering in the database.
-* Use :class:`F expressions <django.db.models.F>` to do filtering
- against other fields within the same model.
+* Use :class:`F expressions <django.db.models.F>` to do filter
@timgraham Owner

I wouldn't change filtering -> filter

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
@@ -0,0 +1,405 @@
+============================
+Performance and optimization
+============================
+
+This document provides an overview of approaches, techniques, tools and helpers
@timgraham Owner

same comments as above plus dash -> colon as Carl mentioned

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
@@ -0,0 +1,405 @@
+============================
+Performance and optimization
+============================
+
+This document provides an overview of approaches, techniques, tools and helpers
+that can help get your Django code running more efficiently - faster, and using
+fewer system resources.
+
+Introduction
+============
+
+Generally one's first concern is to write code that *works*, whose logic
+functions as required to produce the expected output. Sometimes it will turn
@timgraham Owner

I'd chop "it will turn out to be the case though that" , i.e. Sometimes this is not enough...

@timgraham Owner

add commas: Sometimes, however,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((4 lines not shown))
+
+This document provides an overview of approaches, techniques, tools and helpers
+that can help get your Django code running more efficiently - faster, and using
+fewer system resources.
+
+Introduction
+============
+
+Generally one's first concern is to write code that *works*, whose logic
+functions as required to produce the expected output. Sometimes it will turn
+out to be the case though that this is not enough to make the code work as
+*efficiently* as one would like.
+
+In this case, what's needed is something - and in practice, often a collection
+of things - to improve the code's performance without, or only minimally,
+affecting its behaviour.
@timgraham Owner

behavior

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((12 lines not shown))
+Generally one's first concern is to write code that *works*, whose logic
+functions as required to produce the expected output. Sometimes it will turn
+out to be the case though that this is not enough to make the code work as
+*efficiently* as one would like.
+
+In this case, what's needed is something - and in practice, often a collection
+of things - to improve the code's performance without, or only minimally,
+affecting its behaviour.
+
+General approaches
+==================
+
+What are you optimizing *for*?
+------------------------------
+
+It's important to have a clear idea what you mean by improved performance -
@timgraham Owner

dash -> as?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((16 lines not shown))
+
+In this case, what's needed is something - and in practice, often a collection
+of things - to improve the code's performance without, or only minimally,
+affecting its behaviour.
+
+General approaches
+==================
+
+What are you optimizing *for*?
+------------------------------
+
+It's important to have a clear idea what you mean by improved performance -
+there is not just one marker of performance.
+
+Improved speed might be the most obvious aim for a program, but sometimes the
+performance improvements sought might be in another direction, such as lower
@timgraham Owner

direction -> area?

@timgraham Owner

not sure if you missed the comments here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((17 lines not shown))
+In this case, what's needed is something - and in practice, often a collection
+of things - to improve the code's performance without, or only minimally,
+affecting its behaviour.
+
+General approaches
+==================
+
+What are you optimizing *for*?
+------------------------------
+
+It's important to have a clear idea what you mean by improved performance -
+there is not just one marker of performance.
+
+Improved speed might be the most obvious aim for a program, but sometimes the
+performance improvements sought might be in another direction, such as lower
+memory consumption, or fewer demands on the database or network.
@timgraham Owner

chop comma?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((19 lines not shown))
+affecting its behaviour.
+
+General approaches
+==================
+
+What are you optimizing *for*?
+------------------------------
+
+It's important to have a clear idea what you mean by improved performance -
+there is not just one marker of performance.
+
+Improved speed might be the most obvious aim for a program, but sometimes the
+performance improvements sought might be in another direction, such as lower
+memory consumption, or fewer demands on the database or network.
+
+Improvements in one area will often bring about improved performance in other,
@timgraham Owner

others? ... in others, but not always. Sometimes an improvement in one area can be at the expense of performance in another area.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((28 lines not shown))
+there is not just one marker of performance.
+
+Improved speed might be the most obvious aim for a program, but sometimes the
+performance improvements sought might be in another direction, such as lower
+memory consumption, or fewer demands on the database or network.
+
+Improvements in one area will often bring about improved performance in other,
+not always; sometimes one can even be at the expense of another. For example,
+an improvement in a program's speed might cause it to use more memory. Even
+worse, it can be self defeating - if the speed improvement is so memory-hungry
+that the system starts to run out of memory, you'll have done more harm than
+good.
+
+There are other trade-offs to bear in mind. Your own time is a valuable
+resource, more precious than CPU time. Some improvements might be too difficult
+to be worth attaining, or might affect the portability or maintainability of
@timgraham Owner

too difficult to implement?

@timgraham Owner

chop "of achieving them"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((29 lines not shown))
+
+Improved speed might be the most obvious aim for a program, but sometimes the
+performance improvements sought might be in another direction, such as lower
+memory consumption, or fewer demands on the database or network.
+
+Improvements in one area will often bring about improved performance in other,
+not always; sometimes one can even be at the expense of another. For example,
+an improvement in a program's speed might cause it to use more memory. Even
+worse, it can be self defeating - if the speed improvement is so memory-hungry
+that the system starts to run out of memory, you'll have done more harm than
+good.
+
+There are other trade-offs to bear in mind. Your own time is a valuable
+resource, more precious than CPU time. Some improvements might be too difficult
+to be worth attaining, or might affect the portability or maintainability of
+the code. They are not all worth it.
@timgraham Owner

"They are not all worth it." -> "Not all optimization techniques are worthwhile in every situation."?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((54 lines not shown))
+code.
+
+Django tools
+^^^^^^^^^^^^
+
+`django-debug-toolbar
+<https://github.com/django-debug-toolbar/django-debug-toolbar/>`_ is a very
+handy tool that provides insights into what your code is doing and how much
+time it spends doing it.
+
+Third-party plugins are also available for the toolbar; particularly useful for
+studying the performance of your code are the `Cache Panel
+<https://github.com/lincolnloop/django-cache-panel>`_ and `Template Timings
+<https://github.com/orf/django-debug-toolbar-template-timings>`_ modules.
+
+Third-party services
@timgraham Owner

-0 on this section. It isn't Django specific and the Django docs don't need to teach web development in general.

@evildmp Collaborator
evildmp added a note

I take your points about mentioning third-party packages, and about keeping things Django-specific. However, since this is a topic rather than a reference document, I feel there is scope for presenting a slightly wider overview. People need somewhere to start, and to see examples of the kind of tool that's being described.

@timgraham Owner

We can put examples on the wiki, can't we? If you want to bring it up on django-developers, I'll be happy to defer if I'm in the minority. I'd also be concerned we'd get a lot of tickets like "please add the debugging tool I developed!!" A wiki page eliminates this problem.

@evildmp Collaborator
evildmp added a note

I've removed references to the specific plugins, but perhaps a reference to django-debug-toolbar is worth keeping (it's mentioned elsewhere in the docs).

@timgraham Owner

I'm ok with it. In fact there's a ticket suggesting we may include it in contrib at some point - https://code.djangoproject.com/ticket/11698

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((82 lines not shown))
+
+There are also several paid-for services that perform a similar analysis,
+including some that are Django-aware and can integrate with your codebase to
+profile its performance far more comprehensively.
+
+Get things right from the start
+-------------------------------
+
+Some work in optimization involves tackling performance shortcomings, but some
+of the work can simply be built in to what you'd do anyway, as part of the good
+practices you should adopt even before you start thinking about improving
+performance.
+
+In this respect Python is an excellent language to work with, because solutions
+that look elegant and feel right usually are the best performing ones. As with
+most skills, learning what 'looks right' takes practice, but some useful
@timgraham Owner

double quotes?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((88 lines not shown))
+-------------------------------
+
+Some work in optimization involves tackling performance shortcomings, but some
+of the work can simply be built in to what you'd do anyway, as part of the good
+practices you should adopt even before you start thinking about improving
+performance.
+
+In this respect Python is an excellent language to work with, because solutions
+that look elegant and feel right usually are the best performing ones. As with
+most skills, learning what 'looks right' takes practice, but some useful
+guidelines are:
+
+Work at the appropriate level
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Django offers many possible different ways of approaching things, but don't do
@timgraham Owner

"possible different" - redundant words?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((91 lines not shown))
+of the work can simply be built in to what you'd do anyway, as part of the good
+practices you should adopt even before you start thinking about improving
+performance.
+
+In this respect Python is an excellent language to work with, because solutions
+that look elegant and feel right usually are the best performing ones. As with
+most skills, learning what 'looks right' takes practice, but some useful
+guidelines are:
+
+Work at the appropriate level
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Django offers many possible different ways of approaching things, but don't do
+things just because you can. For example, you might find that you could
+calculate the same thing - the number of items in a collection, perhaps - in a
+``QuerySet``, in Python or in a template.
@timgraham Owner

Oxford comma

@timgraham Owner

I wouldn't have a new paragraph here -- the sentence is directly tied to the previous one.

@timgraham Owner

also "it will almost always"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@timgraham timgraham commented on the diff
docs/topics/performance.txt
((105 lines not shown))
+calculate the same thing - the number of items in a collection, perhaps - in a
+``QuerySet``, in Python or in a template.
+
+However, almost always it will be faster to do this work at lower rather than
+higher levels, where these things are dealt with in their rawer states rather
+than through multiple levels of abstraction.
+
+That is, the database can typically do things faster than Python can, which can
+do them faster than the template language can::
+
+ # QuerySet operation on the database
+ # fast, because that's what databases are good at
+ my_bicycles.count()
+
+ # counting Python objects
+ # slower, because it requires a database query anyway, and processing
@timgraham Owner

chop "anyway,"?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((114 lines not shown))
+
+ # QuerySet operation on the database
+ # fast, because that's what databases are good at
+ my_bicycles.count()
+
+ # counting Python objects
+ # slower, because it requires a database query anyway, and processing
+ # of the Python objects
+ len(my_bicycles)
+
+ # Django template filter
+ # slower still, because it will have to count them in Python anyway,
+ # and because of template language overheads
+ {{ my_bicycles|length }}
+
+Generally speaking, the most appropriate level for the job is the lowest-level one that it is comfortable to code for.
@timgraham Owner

long line

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@timgraham timgraham commented on the diff
docs/topics/performance.txt
((148 lines not shown))
+Often it is expensive - resource-hungry and slow - to compute a value, so there
+can be huge benefit in saving the value to a quickly accessible cache ready for
+the next time it's required.
+
+It's a sufficiently significant and powerful technique that Django includes a
+comprehensive caching framework, as well as numerous other opportunities to
+make use of caching.
+
+:doc:`The caching framework </topics/cache>`
+--------------------------------------------
+
+Django's :doc:`caching framework </topics/cache>` offers very significant
+opportunities for performance gains, by saving dynamic content so that it
+doesn't need to be calculated for each request.
+
+For convenience, Django offers different levels of cache granularity: you can
@timgraham Owner

structure of the list is off: "you can cache..., you can cache...., or even an entire site." If you're going to repeat "you" it should be repeated the 3rd time as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((171 lines not shown))
+Other opportunities for caching
+-------------------------------
+
+Beyond the caching framework, Django offers other smaller pieces of caching
+functionality.
+
+:class:`django.utils.functional.cached_property`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+It's common to have to call a class instances's method more than once. If
+that function is expensive, then doing so can be wasteful.
+
+Using the ``@cached_property`` decorator saves the value returned by a
+property; the next time the function is called on that instance, it will return
+the saved value rather than re-computing it. Note that this only works on
+methods that take a single argument, and changes the method to a property.
@timgraham Owner

no comma

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((176 lines not shown))
+
+:class:`django.utils.functional.cached_property`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+It's common to have to call a class instances's method more than once. If
+that function is expensive, then doing so can be wasteful.
+
+Using the ``@cached_property`` decorator saves the value returned by a
+property; the next time the function is called on that instance, it will return
+the saved value rather than re-computing it. Note that this only works on
+methods that take a single argument, and changes the method to a property.
+
+:class:`django.contrib.staticfiles.storage.CachedStaticFilesStorage`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+CachedStaticFilesStorage appends a content-dependent tag to the filenames of
@timgraham Owner

add ``

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((178 lines not shown))
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+It's common to have to call a class instances's method more than once. If
+that function is expensive, then doing so can be wasteful.
+
+Using the ``@cached_property`` decorator saves the value returned by a
+property; the next time the function is called on that instance, it will return
+the saved value rather than re-computing it. Note that this only works on
+methods that take a single argument, and changes the method to a property.
+
+:class:`django.contrib.staticfiles.storage.CachedStaticFilesStorage`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+CachedStaticFilesStorage appends a content-dependent tag to the filenames of
+:doc:`static files </ref/contrib/staticfiles>` to make it safe for browsers to
+cache them long-term without missing future changes - when a file changes, so
@timgraham Owner

I use a separate sentence rather than dash.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((181 lines not shown))
+that function is expensive, then doing so can be wasteful.
+
+Using the ``@cached_property`` decorator saves the value returned by a
+property; the next time the function is called on that instance, it will return
+the saved value rather than re-computing it. Note that this only works on
+methods that take a single argument, and changes the method to a property.
+
+:class:`django.contrib.staticfiles.storage.CachedStaticFilesStorage`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+CachedStaticFilesStorage appends a content-dependent tag to the filenames of
+:doc:`static files </ref/contrib/staticfiles>` to make it safe for browsers to
+cache them long-term without missing future changes - when a file changes, so
+will the tag, so browsers will reload the asset automatically.
+
+.. todo::
@timgraham Owner

Feel free to commit this as v1 and add others later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((188 lines not shown))
+:class:`django.contrib.staticfiles.storage.CachedStaticFilesStorage`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+CachedStaticFilesStorage appends a content-dependent tag to the filenames of
+:doc:`static files </ref/contrib/staticfiles>` to make it safe for browsers to
+cache them long-term without missing future changes - when a file changes, so
+will the tag, so browsers will reload the asset automatically.
+
+.. todo::
+
+ document other cache resources (such as memoize) in django.utils.functional
+
+Understanding laziness
+======================
+
+*Laziness* is the strategy complementary to caching. Caching avoids
@timgraham Owner

a strategy?

@timgraham Owner

the strategy -> a strategy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((193 lines not shown))
+cache them long-term without missing future changes - when a file changes, so
+will the tag, so browsers will reload the asset automatically.
+
+.. todo::
+
+ document other cache resources (such as memoize) in django.utils.functional
+
+Understanding laziness
+======================
+
+*Laziness* is the strategy complementary to caching. Caching avoids
+recomputation by saving results; laziness delays computation until it's
+actually required.
+
+Laziness allows us to refer to things before they are instantiated, or even
+before it's possible to instantiate them, which has numerous uses (for example,
@timgraham Owner

it's initially unclear what part of the sentence "which has numerous uses" refers to. I'd probably break this up a bit so it's less ambiguous.

@timgraham Owner

"For example, " as a separate sentence instead of parenthetical?

For example, lazy translation can be used to mark a string for translation before the target language is known. Translation will take place when the translated string is actually required, such as in a rendered template.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((194 lines not shown))
+will the tag, so browsers will reload the asset automatically.
+
+.. todo::
+
+ document other cache resources (such as memoize) in django.utils.functional
+
+Understanding laziness
+======================
+
+*Laziness* is the strategy complementary to caching. Caching avoids
+recomputation by saving results; laziness delays computation until it's
+actually required.
+
+Laziness allows us to refer to things before they are instantiated, or even
+before it's possible to instantiate them, which has numerous uses (for example,
+:ref:`lazy-translations` can be used before the target language is even known.
@timgraham Owner

chop "even"?

@timgraham Owner

period -> comma?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((200 lines not shown))
+Understanding laziness
+======================
+
+*Laziness* is the strategy complementary to caching. Caching avoids
+recomputation by saving results; laziness delays computation until it's
+actually required.
+
+Laziness allows us to refer to things before they are instantiated, or even
+before it's possible to instantiate them, which has numerous uses (for example,
+:ref:`lazy-translations` can be used before the target language is even known.
+because it doesn't take place until the translated string is actually required,
+such as in a rendered template).
+
+Laziness is also a way to save effort by trying to avoid work in the first
+place. That is, if laziness means not doing anything until it has to be done,
+it may not turn out to be necessary after all, and if something does have to be
@timgraham Owner

it->the work in question (otherwise "it" refers to "laziness" in the first clause I think).. but this sentence is overly complex and confusing and should be rewritten I think.

@timgraham Owner

commas around therefore?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((205 lines not shown))
+actually required.
+
+Laziness allows us to refer to things before they are instantiated, or even
+before it's possible to instantiate them, which has numerous uses (for example,
+:ref:`lazy-translations` can be used before the target language is even known.
+because it doesn't take place until the translated string is actually required,
+such as in a rendered template).
+
+Laziness is also a way to save effort by trying to avoid work in the first
+place. That is, if laziness means not doing anything until it has to be done,
+it may not turn out to be necessary after all, and if something does have to be
+done, it will have avoided doing it unecessarily. Laziness therefore can
+obviously have performance implications, and the more expensive the work
+concerned, the more there is to gain through laziness.
+
+Python makes very good use of laziness. In fact Python is quite a lazy
@timgraham Owner

is there any link in the official Python documentation regarding laziness? after reading this, I'm left wondering what I should or shouldn't do in order not to disrupt Python's laziness?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((222 lines not shown))
+negative one: don't do things that prevent Python being lazy. Similarly, Django
+likes to be lazy, so the first thing to do should simply be not to disrupt that
+laziness.
+
+Laziness in Django
+------------------
+
+A good example of this in Django can be found in the evaluation of
+``QuerySets``. :ref:`QuerySets are lazy <querysets-are-lazy>`, but
+:ref:`certain operations will force the evaluation of a QuerySet
+<when-querysets-are-evaluated>`. Avoiding the premature evaluation of a
+``QuerySet`` can save on an expensive and unnecessary trip to the database.
+
+Django also offers an :meth:`~django.utils.functional.allow_lazy` decorator,
+that allows a function that has been called with a lazy argument to behave
+lazily itself, only evaluating when required. This is an opportunity therefore
@timgraham Owner

evaluating -> being evaluated? or "being called"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((223 lines not shown))
+likes to be lazy, so the first thing to do should simply be not to disrupt that
+laziness.
+
+Laziness in Django
+------------------
+
+A good example of this in Django can be found in the evaluation of
+``QuerySets``. :ref:`QuerySets are lazy <querysets-are-lazy>`, but
+:ref:`certain operations will force the evaluation of a QuerySet
+<when-querysets-are-evaluated>`. Avoiding the premature evaluation of a
+``QuerySet`` can save on an expensive and unnecessary trip to the database.
+
+Django also offers an :meth:`~django.utils.functional.allow_lazy` decorator,
+that allows a function that has been called with a lazy argument to behave
+lazily itself, only evaluating when required. This is an opportunity therefore
+not to waste the benefit of a lazy function.
@timgraham Owner

how would I waste the benefit?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((238 lines not shown))
+not to waste the benefit of a lazy function.
+
+.. todo::
+
+ document other laziness resources in django.utils.functional
+
+Databases
+=========
+
+:doc:`Database optimization </topics/db/optimization>`
+------------------------------------------------------
+
+Django’s database layer provides various ways to help developers get the most
+out of their databases. The :doc:`database optimization documentation
+</topics/db/optimization>` gathers together links to the relevant
+documentation, and adds various tips, organized under a number of headings that
@timgraham Owner

chop "organized under a number of headings"? remove commas

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((235 lines not shown))
+Django also offers an :meth:`~django.utils.functional.allow_lazy` decorator,
+that allows a function that has been called with a lazy argument to behave
+lazily itself, only evaluating when required. This is an opportunity therefore
+not to waste the benefit of a lazy function.
+
+.. todo::
+
+ document other laziness resources in django.utils.functional
+
+Databases
+=========
+
+:doc:`Database optimization </topics/db/optimization>`
+------------------------------------------------------
+
+Django’s database layer provides various ways to help developers get the most
@timgraham Owner

"get the most from" -> "maximize performance of database operations"?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((259 lines not shown))
+:ref:`Persistent database connections <persistent-database-connections>`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Enabling persistent database connections can result in a nice speed-up when
+connecting to the database accounts for a significant part of the request
+processing time.
+
+This helps a lot on virtualized hosts with limited network performance, for example.
+
+HTTP performance
+================
+
+Django comes with a few other pieces of middleware that can help optimize your
+site's performance :
+
+:class:`django.middleware.http.ConditionalGetMiddleware`
@timgraham Owner

want to remove these items from "topics/cache". I added a sentence re: security of GzipMiddleware that can be moved here.

@evildmp Collaborator
evildmp added a note

I know GZipMiddleware has lost some friends, but are you suggesting that both it and ConditionalGetMiddleware should be removed altogether, or simply moved?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((257 lines not shown))
+---------------------------
+
+:ref:`Persistent database connections <persistent-database-connections>`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Enabling persistent database connections can result in a nice speed-up when
+connecting to the database accounts for a significant part of the request
+processing time.
+
+This helps a lot on virtualized hosts with limited network performance, for example.
+
+HTTP performance
+================
+
+Django comes with a few other pieces of middleware that can help optimize your
+site's performance :
@timgraham Owner

extra space before :

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((272 lines not shown))
+site's performance :
+
+:class:`django.middleware.http.ConditionalGetMiddleware`
+--------------------------------------------------------
+
+Adds support for modern browsers to conditionally GET responses based on the
+``ETag`` and ``Last-Modified`` headers.
+
+:class:`django.middleware.gzip.GZipMiddleware`
+----------------------------------------------
+
+Compresses responses for all modern browsers, saving bandwidth and transfer
+time. Note that `GZipMiddleware is currently considered a security risk
+<https://www.djangoproject.com/weblog/2013/aug/06/breach-and-django/>`_.
+
+Third-party tools for improving performance
@timgraham Owner

Not Django specific

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((284 lines not shown))
+time. Note that `GZipMiddleware is currently considered a security risk
+<https://www.djangoproject.com/weblog/2013/aug/06/breach-and-django/>`_.
+
+Third-party tools for improving performance
+-------------------------------------------
+
+There are numerous third-party tools and packages available, notably ones that
+are able to 'minify' and compress HTML, CSS and JavaScript.
+
+Template performance
+====================
+
+Note that:
+
+* blocks are faster than includes
+* heavily fragmented templates can affect performance
@timgraham Owner

what does "heavily fragmented" mean?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((292 lines not shown))
+
+Template performance
+====================
+
+Note that:
+
+* blocks are faster than includes
+* heavily fragmented templates can affect performance
+
+:class:`django.template.loaders.cached.Loader`
+----------------------------------------------
+
+Enabling the cached template loader often improves performance drastically, as
+it avoids compiling each template every time it needs to be rendered.
+
+Using faster versions of available software
@timgraham Owner

unsure about everything from here down, not really Django specific.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((264 lines not shown))
+:ref:`Persistent database connections <persistent-database-connections>`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Enabling persistent database connections can result in a nice speed-up when
+connecting to the database accounts for a significant part of the request
+processing time.
+
+This helps a lot on virtualized hosts with limited network performance, for example.
+
+HTTP performance
+================
+
+Django comes with a few other pieces of middleware that can help optimize your
+site's performance:
+
+:class:`~django.middleware.http.ConditionalGetMiddleware`
@timgraham Owner

re: "I know GZipMiddleware has lost some friends, but are you suggesting that both it and ConditionalGetMiddleware should be removed altogether, or simply moved?"

I'm suggesting remove these items from topics/cache since they aren't part of the cache framework which is more the focus of that doc.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@evildmp
Collaborator

@timgraham if you're happy with this rewording I will reconcile merge conflicts and rebase

docs/topics/performance.txt
((87 lines not shown))
+
+Some work in optimization involves tackling performance shortcomings, but some
+of the work can simply be built in to what you'd do anyway, as part of the good
+practices you should adopt even before you start thinking about improving
+performance.
+
+In this respect Python is an excellent language to work with, because solutions
+that look elegant and feel right usually are the best performing ones. As with
+most skills, learning what "looks right" takes practice, but one of the most
+useful guidelines is:
+
+Work at the appropriate level
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Django offers many different ways of approaching things, but don't do things
+just because you can. For example, you might find that you could calculate the
@timgraham Owner

I'm not sure that "don't do things just because you can" articulates the topic very well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((132 lines not shown))
+ The example above is merely illustrative.
+
+ Firstly, in a real-life case you need to consider what is happening before
+ and after your count to work out what's an optimal way of doing it *in that
+ particular context*. The database optimization documents describes :ref:`a
+ case where counting in the template would be better
+ <overuse_of_count_and_exists>`.
+
+ Secondly, there are other options to consider: in a real-life case, ``{{
+ my_bicycles.count }}``, which invokes the ``QuerySet`` ``count()`` method
+ directly from the template, might be the most appropriate choice.
+
+Caching
+=======
+
+Often it is expensive - resource-hungry and slow - to compute a value, so there
@timgraham Owner

I guess it's partly personal preference but more often I've seen parentheses used for this type of remark. When I'm reading I tend to find dashes interruptive.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((152 lines not shown))
+comprehensive caching framework, as well as numerous other opportunities to
+make use of caching.
+
+:doc:`The caching framework </topics/cache>`
+--------------------------------------------
+
+Django's :doc:`caching framework </topics/cache>` offers very significant
+opportunities for performance gains, by saving dynamic content so that it
+doesn't need to be calculated for each request.
+
+For convenience, Django offers different levels of cache granularity: you can
+cache the output of specific views, or only the pieces that are difficult to
+produce, or even an entire site.
+
+Implementing caching should not be regarded as an alternative to improving
+poorly-performing code. It's one of the final steps towards producing
@timgraham Owner

I think you mean "poorly-performing code that's also poorly written"?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@timgraham timgraham commented on the diff
docs/topics/performance.txt
((212 lines not shown))
+
+Laziness is also a way to save effort by trying to avoid work in the first
+place. That is, one aspect of laziness is not doing anything until it has to be
+done, because it may not turn out to be necessary after all. Laziness therefore
+can obviously have performance implications, and the more expensive the work
+concerned, the more there is to gain through laziness.
+
+Python provides a number of tools for lazy evaluation, particularly through the
+:py:term:`generator` and :py:term:`generator expression` constructs. It's worth
+reading up on laziness in Python to discover opportunities for making use of
+lazy patterns in your code.
+
+Laziness in Django
+------------------
+
+Django is itself quite lazy. A good example of this can be found in the
@timgraham Owner

these two paragraphs are quite good

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((302 lines not shown))
+* using ``{% block %}`` is faster than using ``{% include %}``
+* heavily-fragmented templates, assembled from many small pieces, can affect
+ performance
+
+The cached template loader
+--------------------------
+
+Enabling the :class:`cached template loader
+<django.template.loaders.cached.Loader>` often improves performance
+drastically, as it avoids compiling each template every time it needs to be
+rendered.
+
+Using different versions of available software
+==============================================
+
+It can sometimes be worth checking whether different - better-performing -
@timgraham Owner

is "different" necessary? i.e. whether better-performing versions? if yes, then I'd say "different, better-performing" rather than dashes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((333 lines not shown))
+while being aware that newer versions are likely to perform better, don't
+simply assume that they always will.
+
+This is true of Django itself. Successive release notes show a number of
+improvements across the system to be enjoyed just by using a newer version, but
+you should still check the real-world performance of your application, because
+in some cases you may find that changes mean it performs less well rather than
+better.
+
+Newer versions of Python, and also of Python packages, will often perform
+better too - but measure, rather than assume.
+
+.. note::
+
+ Unless you've encountered an unusual performance problem in a particular
+ version, you'll generally find better features, reliability and security in
@timgraham Owner

oxford comma

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((322 lines not shown))
+
+.. note::
+
+ It's worth repeating: **reaching for alternatives to software you're
+ already using is very rarely the answer to performance problems**.
+
+Newer is often - but not always - better
+----------------------------------------
+
+It's fairly rare for a new release of well-maintained software to be less
+efficient, but the maintainers can't anticipate every possible use-case - so
+while being aware that newer versions are likely to perform better, don't
+simply assume that they always will.
+
+This is true of Django itself. Successive release notes show a number of
+improvements across the system to be enjoyed just by using a newer version, but
@timgraham Owner

Successive release notes show -> Successive releases have included

chop: "to be enjoyed just by using a newer version"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/index.txt
@@ -232,6 +232,15 @@ regions:
* :doc:`"Local flavor" <topics/localflavor>`
* :doc:`Time zones </topics/i18n/timezones>`
+Performance and optimization
+============================
+
+There are a variety of techniques and tools that can help get your code running
+more efficiently - faster, and using fewer system resources.
+
+* :doc:`Performance and optimization overview <topics/performance>`
+
+
@timgraham Owner

one space between sections

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((92 lines not shown))
+
+In this respect Python is an excellent language to work with, because solutions
+that look elegant and feel right usually are the best performing ones. As with
+most skills, learning what "looks right" takes practice, but one of the most
+useful guidelines is:
+
+Work at the appropriate level
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Django offers many different ways of approaching things, but don't do things
+just because you can. For example, you might find that you could calculate the
+same thing - the number of items in a collection, perhaps - in a ``QuerySet``,
+in Python, or in a template.
+
+However, almost always it will be faster to do this work at lower rather than
+higher levels, where these things are dealt with in their rawer states rather
@shaib Collaborator
shaib added a note

"states"? not "forms" or something? English is not my native tongue, so I may be missing something.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((47 lines not shown))
+also need to know that you have a good reason for aiming in that direction -
+and for that you need:
+
+Performance benchmarking
+------------------------
+
+It's no good just guessing or assuming where the inefficiencies lie in your
+code.
+
+Django tools
+^^^^^^^^^^^^
+
+`django-debug-toolbar
+<https://github.com/django-debug-toolbar/django-debug-toolbar/>`_ is a very
+handy tool that provides insights into what your code is doing and how much
+time it spends doing it.

If we're mentioning django debug toolbar, I'd like to make it something like:

".. that shows many details, including all SQL queries executed for that page, and how long each one took. ..."

as that is the crucial thing it does from the performance point of view. The other stuff it shows is useful for debugging what went wrong, but not so useful for performance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((274 lines not shown))
+
+Django comes with a few helpful pieces of :doc:`middleware </ref/middleware>`
+that can help optimize your site's performance. They include:
+
+:class:`~django.middleware.http.ConditionalGetMiddleware`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Adds support for modern browsers to conditionally GET responses based on the
+``ETag`` and ``Last-Modified`` headers.
+
+:class:`~django.middleware.gzip.GZipMiddleware`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Compresses responses for all modern browsers, saving bandwidth and transfer
+time. Note that `GZipMiddleware is currently considered a security risk
+<https://www.djangoproject.com/weblog/2013/aug/06/breach-and-django/>`_.

It is only a security risk when used with TLS/SSL, so maybe this could be:

Note that GZipMiddleware is considered a security risk on sites that use HTTPS

@timgraham Owner

If you're not using https, then you've always been subject to the attacks (man in the middle), right? I just wouldn't want to give the impression that if you use GzipMiddleware without SSL you are somehow more secure than https + gzip.

@evildmp You'll notice this when you rebase, but I added a warning in da843e7 (referencing the docs rather than the blog post)

Fair point, how about:

Note that using GZipMiddleware allows attacks that would nullify the security provided by SSL/TLS.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@timgraham timgraham commented on the diff
docs/topics/performance.txt
((48 lines not shown))
+and for that you need:
+
+Performance benchmarking
+------------------------
+
+It's no good just guessing or assuming where the inefficiencies lie in your
+code.
+
+Django tools
+^^^^^^^^^^^^
+
+`django-debug-toolbar
+<https://github.com/django-debug-toolbar/django-debug-toolbar/>`_ is a very
+handy tool that provides insights into what your code is doing and how much
+time it spends doing it. In particular it can show you all the SQL queries your
+page is generating, and how long each one has taken.
@timgraham Owner

no comma

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((50 lines not shown))
+Performance benchmarking
+------------------------
+
+It's no good just guessing or assuming where the inefficiencies lie in your
+code.
+
+Django tools
+^^^^^^^^^^^^
+
+`django-debug-toolbar
+<https://github.com/django-debug-toolbar/django-debug-toolbar/>`_ is a very
+handy tool that provides insights into what your code is doing and how much
+time it spends doing it. In particular it can show you all the SQL queries your
+page is generating, and how long each one has taken.
+
+Third-party panels are also available for the toolbar, that can - for example -
@timgraham Owner

I'd drop the first comma and replace the dashes by commas

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((184 lines not shown))
+Using the ``@cached_property`` decorator saves the value returned by a
+property; the next time the function is called on that instance, it will return
+the saved value rather than re-computing it. Note that this only works on
+methods that take ``self`` as their only argument and that it changes the
+method to a property.
+
+:class:`~django.contrib.staticfiles.storage.CachedStaticFilesStorage`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+:class:`~django.contrib.staticfiles.storage.CachedStaticFilesStorage` appends a
+content-dependent tag to the filenames of :doc:`static files
+</ref/contrib/staticfiles>` to make it safe for browsers to cache them
+long-term without missing future changes - when a file changes, so will the
+tag, so browsers will reload the asset automatically.
+
+.. todo::
@timgraham Owner

please remove before committing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((234 lines not shown))
+``QuerySets``, without actually incurring any trips to the database to fetch
+the items it describes. What gets passed around is the ``QuerySet`` object, not
+the collection of items that - eventually - will be required from the database.
+
+On the other hand, :ref:`certain operations will force the evaluation of a
+QuerySet <when-querysets-are-evaluated>`. Avoiding the premature evaluation of
+a ``QuerySet`` can save making an expensive and unnecessary trip to the
+database.
+
+Django also offers an :meth:`~django.utils.functional.allow_lazy` decorator.
+This allows a function that has been called with a lazy argument to behave
+lazily itself, only being evaluated when it needs to be. Thus the lazy argument
+- which could be an expensive one - will not be called upon for evaluation
+until it's strictly required.
+
+.. todo::
@timgraham Owner

please chop before committing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((278 lines not shown))
+
+Django comes with a few helpful pieces of :doc:`middleware </ref/middleware>`
+that can help optimize your site's performance. They include:
+
+:class:`~django.middleware.http.ConditionalGetMiddleware`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Adds support for modern browsers to conditionally GET responses based on the
+``ETag`` and ``Last-Modified`` headers.
+
+:class:`~django.middleware.gzip.GZipMiddleware`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Compresses responses for all modern browsers, saving bandwidth and transfer
+time. Note that `GZipMiddleware is currently considered a security risk
+<https://www.djangoproject.com/weblog/2013/aug/06/breach-and-django/>`_, and is
@timgraham Owner

could we please use something like this text: "See the warning in :class:~django.middleware.gzip.GZipMiddleware for details." rather than linking to the blog post which may become out of update at some point?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((326 lines not shown))
+You won't usually gain performance advantages that are better than marginal.
+
+.. note::
+
+ It's worth repeating: **reaching for alternatives to software you're
+ already using is very rarely the answer to performance problems**.
+
+Newer is often - but not always - better
+----------------------------------------
+
+It's fairly rare for a new release of well-maintained software to be less
+efficient, but the maintainers can't anticipate every possible use-case - so
+while being aware that newer versions are likely to perform better, don't
+simply assume that they always will.
+
+This is true of Django itself. Successive release notes show a number of
@timgraham Owner

not sure if you missed or ignored by previous comment here but I think it's weird to reference the release notes. I'd say: "Successive releases have included a number of... "

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((329 lines not shown))
+
+ It's worth repeating: **reaching for alternatives to software you're
+ already using is very rarely the answer to performance problems**.
+
+Newer is often - but not always - better
+----------------------------------------
+
+It's fairly rare for a new release of well-maintained software to be less
+efficient, but the maintainers can't anticipate every possible use-case - so
+while being aware that newer versions are likely to perform better, don't
+simply assume that they always will.
+
+This is true of Django itself. Successive release notes show a number of
+improvements across the system, but you should still check the real-world
+performance of your application, because in some cases you may find that
+changes mean it performs less well rather than better.
@timgraham Owner

less well -> worse?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
docs/topics/performance.txt
((338 lines not shown))
+while being aware that newer versions are likely to perform better, don't
+simply assume that they always will.
+
+This is true of Django itself. Successive release notes show a number of
+improvements across the system, but you should still check the real-world
+performance of your application, because in some cases you may find that
+changes mean it performs less well rather than better.
+
+Newer versions of Python, and also of Python packages, will often perform
+better too - but measure, rather than assume.
+
+.. note::
+
+ Unless you've encountered an unusual performance problem in a particular
+ version, you'll generally find better features, reliability, and security in
+ a new release and that these are far more significant than any performance
@timgraham Owner

these benefits are

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
@evildmp evildmp merged commit a991b01 into from
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Sep 20, 2013
  1. @evildmp
This page is out of date. Refresh to see the latest.
View
8 docs/index.txt
@@ -233,6 +233,14 @@ regions:
* :doc:`"Local flavor" <topics/localflavor>`
* :doc:`Time zones </topics/i18n/timezones>`
+Performance and optimization
+============================
+
+There are a variety of techniques and tools that can help get your code running
+more efficiently - faster, and using fewer system resources.
+
+* :doc:`Performance and optimization overview <topics/performance>`
+
Python compatibility
====================
View
4 docs/ref/utils.txt
@@ -495,8 +495,8 @@ Atom1Feed
For cases like this, use the ``django.utils.functional.allow_lazy()``
decorator. It modifies the function so that *if* it's called with a lazy
- translation as the first argument, the function evaluation is delayed until it
- needs to be converted to a string.
+ translation as one of its arguments, the function evaluation is delayed
+ until it needs to be converted to a string.
For example::
View
16 docs/topics/cache.txt
@@ -1162,22 +1162,6 @@ Example::
.. _`Cache-Control spec`: http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9
-Other optimizations
-===================
-
-Django comes with a few other pieces of middleware that can help optimize your
-site's performance:
-
-* ``django.middleware.http.ConditionalGetMiddleware`` adds support for
- modern browsers to conditionally GET responses based on the ``ETag``
- and ``Last-Modified`` headers.
-
-* :class:`django.middleware.gzip.GZipMiddleware` compresses responses for all
- modern browsers, saving bandwidth and transfer time. Be warned, however,
- that compression techniques like ``GZipMiddleware`` are subject to attacks.
- See the warning in :class:`~django.middleware.gzip.GZipMiddleware` for
- details.
-
Order of MIDDLEWARE_CLASSES
===========================
View
12 docs/topics/db/optimization.txt
@@ -88,7 +88,8 @@ of parentheses, but will call callables automatically, hiding the above
distinction.
Be careful with your own custom properties - it is up to you to implement
-caching.
+caching when required, for example using the
+:class:`~django.utils.functional.cached_property` decorator.
Use the ``with`` template tag
-----------------------------
@@ -111,10 +112,11 @@ For instance:
* At the most basic level, use :ref:`filter and exclude <queryset-api>` to do
filtering in the database.
-* Use :class:`F expressions <django.db.models.F>` to do filtering
- against other fields within the same model.
+* Use :class:`F expressions <django.db.models.F>` to filter
+ based on other fields within the same model.
-* Use :doc:`annotate to do aggregation in the database </topics/db/aggregation>`.
+* Use :doc:`annotate to do aggregation in the database
+ </topics/db/aggregation>`.
If these aren't enough to generate the SQL you need:
@@ -233,6 +235,8 @@ queryset``.
But:
+.. _overuse_of_count_and_exists:
+
Don't overuse ``count()`` and ``exists()``
------------------------------------------
View
1  docs/topics/index.txt
@@ -26,6 +26,7 @@ Introductions to all the key parts of Django you'll need to know:
pagination
python3
security
+ performance
serialization
settings
signals
View
384 docs/topics/performance.txt
@@ -0,0 +1,384 @@
+============================
+Performance and optimization
+============================
+
+This document provides an overview of techniques and tools that can help get
+your Django code running more efficiently - faster, and using fewer system
+resources.
+
+Introduction
+============
+
+Generally one's first concern is to write code that *works*, whose logic
+functions as required to produce the expected output. Sometimes, however, this
+will not be enough to make the code work as *efficiently* as one would like.
+
+In this case, what's needed is something - and in practice, often a collection
+of things - to improve the code's performance without, or only minimally,
+affecting its behavior.
+
+General approaches
+==================
+
+What are you optimizing *for*?
+------------------------------
+
+It's important to have a clear idea what you mean by 'performance'. There is
+not just one metric of it.
+
+Improved speed might be the most obvious aim for a program, but sometimes other
+performance improvements might be sought, such as lower memory consumption or
+fewer demands on the database or network.
+
+Improvements in one area will often bring about improved performance in
+another, but not always; sometimes one can even be at the expense of another.
+For example, an improvement in a program's speed might cause it to use more
+memory. Even worse, it can be self-defeating - if the speed improvement is so
+memory-hungry that the system starts to run out of memory, you'll have done
+more harm than good.
+
+There are other trade-offs to bear in mind. Your own time is a valuable
+resource, more precious than CPU time. Some improvements might be too difficult
+to be worth implementing, or might affect the portability or maintainability of
+the code. Not all performance improvements are worth the effort.
+
+So, you need to know what performance improvements you are aiming for, and you
+also need to know that you have a good reason for aiming in that direction -
+and for that you need:
+
+Performance benchmarking
+------------------------
+
+It's no good just guessing or assuming where the inefficiencies lie in your
+code.
+
+Django tools
+^^^^^^^^^^^^
+
+`django-debug-toolbar
@timgraham Owner

-1 on including specific recommendations here (although I'd agree django-debug-toolbar is the defacto standard here). I'd prefer to mention the topic in general and link to a wiki page with any specific recommendations. Historically Django has taken a strong stance against endorsing or blessing any 3rd party apps.

@akaariai Collaborator

There is a tool that should be mentioned:
python -m cProfile -s cumulative manage.py runserver --nothreading > prof.txt

Hit the server multiple times (with web benchmarking tool, or just browser + reload).

This is easily the most important tool for me.

@evildmp Collaborator
evildmp added a note

Comments on this @timgraham?

@timgraham Owner

I'm not familiar with it. maybe we can get @akaariai to draft some text. Not a blocker for committing a first version of this I think.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+<https://github.com/django-debug-toolbar/django-debug-toolbar/>`_ is a very
+handy tool that provides insights into what your code is doing and how much
+time it spends doing it. In particular it can show you all the SQL queries your
+page is generating, and how long each one has taken.
@timgraham Owner

no comma

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+
+Third-party panels are also available for the toolbar, that can (for example)
+report on cache performance and template rendering times.
+
+Third-party services
+^^^^^^^^^^^^^^^^^^^^
+
+There are a number of free services that will analyse and report on the
+performance of your site's pages from the perspective of a remote HTTP client,
+in effect simulating the experience of an actual user.
+
+These can't report on the internals of your code, but can provide a useful
+insight into your site's overall performance, including aspects that can't be
+adequately measured from within Django environment. Examples include:
+
+* `Yahoo's Yslow <http://developer.yahoo.com/yslow/>`_
+* `Google PageSpeed <https://developers.google.com/speed/pagespeed/>`_
+
+There are also several paid-for services that perform a similar analysis,
+including some that are Django-aware and can integrate with your codebase to
+profile its performance far more comprehensively.
+
+Get things right from the start
+-------------------------------
+
+Some work in optimization involves tackling performance shortcomings, but some
+of the work can simply be built in to what you'd do anyway, as part of the good

shouldn't it be 'built-in'? That way sounds like "built into"

@timgraham Owner

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+practices you should adopt even before you start thinking about improving
+performance.
+
+In this respect Python is an excellent language to work with, because solutions
+that look elegant and feel right usually are the best performing ones. As with
+most skills, learning what "looks right" takes practice, but one of the most
+useful guidelines is:
+
+Work at the appropriate level
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Django offers many different ways of approaching things, but just because it's
+possible to do something in a certain way doesn't mean that it's the most
+appropriate way to do it. For example, you might find that you could calculate
+the same thing - the number of items in a collection, perhaps - in a
+``QuerySet``, in Python, or in a template.
+
+However, it will almost always be faster to do this work at lower rather than
+higher levels. At higher levels the system has to deal with objects through
+multiple levels of abstraction and layers of machinery.
+
+That is, the database can typically do things faster than Python can, which can
+do them faster than the template language can::
+
+ # QuerySet operation on the database
+ # fast, because that's what databases are good at
+ my_bicycles.count()
+
+ # counting Python objects
+ # slower, because it requires a database query anyway, and processing
@timgraham Owner

chop "anyway,"?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+ # of the Python objects
+ len(my_bicycles)
+
+ # Django template filter
+ # slower still, because it will have to count them in Python anyway,
+ # and because of template language overheads
+ {{ my_bicycles|length }}
+
+Generally speaking, the most appropriate level for the job is the lowest-level
+one that it is comfortable to code for.
+
+.. note::
+
+ The example above is merely illustrative.
+
+ Firstly, in a real-life case you need to consider what is happening before
+ and after your count to work out what's an optimal way of doing it *in that
+ particular context*. The database optimization documents describes :ref:`a
+ case where counting in the template would be better
+ <overuse_of_count_and_exists>`.
+
+ Secondly, there are other options to consider: in a real-life case, ``{{
+ my_bicycles.count }}``, which invokes the ``QuerySet`` ``count()`` method
+ directly from the template, might be the most appropriate choice.
+
+Caching
+=======
+
+Often it is expensive (that is, resource-hungry and slow) to compute a value,
+so there can be huge benefit in saving the value to a quickly accessible cache,
+ready for the next time it's required.
+
+It's a sufficiently significant and powerful technique that Django includes a
+comprehensive caching framework, as well as numerous other opportunities to
+make use of caching.
+
+:doc:`The caching framework </topics/cache>`
+--------------------------------------------
+
+Django's :doc:`caching framework </topics/cache>` offers very significant
+opportunities for performance gains, by saving dynamic content so that it
+doesn't need to be calculated for each request.
+
+For convenience, Django offers different levels of cache granularity: you can
@timgraham Owner

structure of the list is off: "you can cache..., you can cache...., or even an entire site." If you're going to repeat "you" it should be repeated the 3rd time as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+cache the output of specific views, or only the pieces that are difficult to
+produce, or even an entire site.
+
+Implementing caching should not be regarded as an alternative to improving code
+that's performing poorly because it has been written badly. It's one of the
+final steps towards producing well-performing code, not a shortcut.
+
+Other opportunities for caching
+-------------------------------
+
+Beyond the caching framework, Django offers other smaller pieces of caching
+functionality.
+
+:class:`~django.utils.functional.cached_property`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+It's common to have to call a class instances's method more than once. If
+that function is expensive, then doing so can be wasteful.
+
+Using the ``@cached_property`` decorator saves the value returned by a
+property; the next time the function is called on that instance, it will return
+the saved value rather than re-computing it. Note that this only works on
+methods that take ``self`` as their only argument and that it changes the
+method to a property.
+
+:class:`~django.contrib.staticfiles.storage.CachedStaticFilesStorage`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+:class:`~django.contrib.staticfiles.storage.CachedStaticFilesStorage` appends a
+content-dependent tag to the filenames of :doc:`static files
+</ref/contrib/staticfiles>` to make it safe for browsers to cache them
+long-term without missing future changes - when a file changes, so will the
+tag, so browsers will reload the asset automatically.
+
+Understanding laziness
+======================
+
+*Laziness* is a strategy complementary to caching. Caching avoids
+recomputation by saving results; laziness delays computation until it's
+actually required.
+
+Laziness allows us to refer to things before they are instantiated, or even
+before it's possible to instantiate them. This has numerous uses.
+
+For example, :ref:`lazy translation <lazy-translations>` can be used before the
+target language is even known, because it doesn't take place until the
+translated string is actually required, such as in a rendered template.
+
+Laziness is also a way to save effort by trying to avoid work in the first
+place. That is, one aspect of laziness is not doing anything until it has to be
+done, because it may not turn out to be necessary after all. Laziness can
+therefore have performance implications, and the more expensive the work
+concerned, the more there is to gain through laziness.
+
+Python provides a number of tools for lazy evaluation, particularly through the
+:py:term:`generator` and :py:term:`generator expression` constructs. It's worth
+reading up on laziness in Python to discover opportunities for making use of
+lazy patterns in your code.
+
+Laziness in Django
+------------------
+
+Django is itself quite lazy. A good example of this can be found in the
@timgraham Owner

these two paragraphs are quite good

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
+evaluation of ``QuerySets``. :ref:`QuerySets are lazy <querysets-are-lazy>`.
+Thus a ``QuerySet`` can be created, passed around and combined with other
+``QuerySets``, without actually incurring any trips to the database to fetch
+the items it describes. What gets passed around is the ``QuerySet`` object, not
+the collection of items that - eventually - will be required from the database.
+
+On the other hand, :ref:`certain operations will force the evaluation of a
+QuerySet <when-querysets-are-evaluated>`. Avoiding the premature evaluation of
+a ``QuerySet`` can save making an expensive and unnecessary trip to the
+database.
+
+Django also offers an :meth:`~django.utils.functional.allow_lazy` decorator.
+This allows a function that has been called with a lazy argument to behave
+lazily itself, only being evaluated when it needs to be. Thus the lazy argument
+- which could be an expensive one - will not be called upon for evaluation
+until it's strictly required.
+
+Databases
+=========
+
+:doc:`Database optimization </topics/db/optimization>`
+------------------------------------------------------
+
+Django’s database layer provides various ways to help developers get the best
+performance from their databases. The :doc:`database optimization documentation
+</topics/db/optimization>` gathers together links to the relevant
+documentation and adds various tips that outline the steps to take when
+attempting to optimize your database usage.
+
+Other database-related tips
+---------------------------
+
+Enabling :ref:`persistent-database-connections` can speed up connections to the
+database accounts for a significant part of the request processing time.
+
+This helps a lot on virtualized hosts with limited network performance, for example.
+
+HTTP performance
+================
+
+Middleware
+----------
+
+Django comes with a few helpful pieces of :doc:`middleware </ref/middleware>`
+that can help optimize your site's performance. They include:
+
+:class:`~django.middleware.http.ConditionalGetMiddleware`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Adds support for modern browsers to conditionally GET responses based on the
+``ETag`` and ``Last-Modified`` headers.
+
+:class:`~django.middleware.gzip.GZipMiddleware`
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Compresses responses for all modern browsers, saving bandwidth and transfer
+time. Note that GZipMiddleware is currently considered a security risk, and is
+vulnerable to attacks that nullify the protection provided by TLS/SSL. See the
+warning in :class:`~django.middleware.gzip.GZipMiddleware` for more information.
+
+Third-party HTTP tools
+----------------------
+
+There are numerous third-party Django tools and packages available, notably
+ones that are able to "minify" and compress HTML, CSS and JavaScript.
+
+Template performance
+====================
+
+Note that:
+
+* using ``{% block %}`` is faster than using ``{% include %}``
+* heavily-fragmented templates, assembled from many small pieces, can affect
+ performance
+
+The cached template loader
+--------------------------
+
+Enabling the :class:`cached template loader
+<django.template.loaders.cached.Loader>` often improves performance
+drastically, as it avoids compiling each template every time it needs to be
+rendered.
+
+Using different versions of available software
+==============================================
+
+It can sometimes be worth checking whether different and better-performing
+versions of the software that you're using are available.
+
+This may be helpful, but is unlikely to solve a serious performance problem.
+You won't usually gain performance advantages that are better than marginal.
+
+.. note::
+
+ It's worth repeating: **reaching for alternatives to software you're
+ already using is very rarely the answer to performance problems**.
+
+Newer is often - but not always - better
+----------------------------------------
+
+It's fairly rare for a new release of well-maintained software to be less
+efficient, but the maintainers can't anticipate every possible use-case - so
+while being aware that newer versions are likely to perform better, don't
+simply assume that they always will.
+
+This is true of Django itself. Successive releases have offered a number of
+improvements across the system, but you should still check the real-world
+performance of your application, because in some cases you may find that
+changes mean it performs worse rather than better.
+
+Newer versions of Python, and also of Python packages, will often perform
+better too - but measure, rather than assume.
+
+.. note::
+
+ Unless you've encountered an unusual performance problem in a particular
+ version, you'll generally find better features, reliability, and security
+ in a new release and that these benefits are far more significant than any
+ performance you might win or lose.
+
+Alternatives to Django's template language
+------------------------------------------
+
+For nearly all cases, Django's built-in template language is perfectly
+adequate. However, if the bottlenecks in your Django project seem to lie in the
+template system and you have exhausted other opportunities to remedy this, a
+third-party alternative may be the answer.
+
+`Jinja2 <http://jinja.pocoo.org/docs/>`_ can offer performance improvements,
+particularly when it comes to speed.
+
+Alternative template systems vary in the extent to which they share Django's
+templating language.
+
+.. note::
+
+ *If* you experience performance issues in templates, the first thing to do
+ is to understand exactly why. Using an alternative template system may
+ prove faster, but the same gains may also be available without going to
+ that trouble - for example, expensive processing and logic in your
+ templates could be done more efficiently in your views.
+
+Alternative software implementations
+------------------------------------
+
+It *may* be worth checking whether Python software you're using has been
+provided in a different implementation that can execute the same code faster.
+
+However, most Django performance problems in well-written code are typically
+not to be found at the Python execution level, but rather in inefficient
+database querying, caching, and templates (and if you're relying on
+poorly-written Python code, your performance problems are very unlikely to be
+solved by having it execute faster).
+
+Avoid using C implementations of Python libraries or non-standard Python
+implementations like `PyPy <http://pypy.org/>`_ in search of performance gains,
+unless you are sure they are appropriate for your application. Any gains are
+likely to be small, and compatibility issues are common.
Something went wrong with that request. Please try again.