Skip to content
Newer
Older
100644 1096 lines (821 sloc) 44.4 KB
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
1 ============================
2 Chapter 12: Deploying Django
3 ============================
4
5 This chapter covers the last essential step of building a Django application:
6 deploying it to a production server.
7
8 If you've been following along with our ongoing examples, you've likely been
9 using the ``runserver``, which makes things very easy -- with ``runserver``,
10 you don't have to worry about Web server setup. But ``runserver`` is intended
11 only for development on your local machine, not for exposure on the public Web.
12 To deploy your Django application, you'll need to hook it into an
13 industrial-strength Web server such as Apache. In this chapter, we'll show you
14 how to do that -- but, first, we'll give you a checklist of things to do in
15 your codebase before you go live.
16
17 Preparing Your Codebase for Production
18 ======================================
19
20 Fortunately, the ``runserver`` approximates a "real" Web server closely enough
21 that not very many changes need to be made to a Django application in order to
22 make it production-ready. But there are a few *essential things* you should do
23 before you turn the switch.
24
25 Turning Off Debug Mode
26 ----------------------
27
28 When we created a project in Chapter 2, the command
29 ``django-admin.py startproject`` created a ``settings.py`` file with ``DEBUG``
30 set to ``True``. Many internal parts of Django check this setting and change
31 their behavior if ``DEBUG`` mode is on. For example, if ``DEBUG`` is set to
32 ``True``, then:
33
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
34 * All database queries will be saved in memory as the object
35 ``django.db.connection.queries``. As you can imagine, this eats up
36 memory!
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
37
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
38 * Any 404 error will be rendered by Django's special 404 error page
39 (covered in Chapter 3) rather than returning a proper 404 response. This
40 page contains potentially sensitive information and should *not* be
41 exposed to the public Internet.
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
42
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
43 * Any uncaught exception in your Django application -- from basic Python
44 syntax errors to database errors to template syntax errors -- will be
45 rendered by the Django pretty error page that you've likely come to know
46 and love. This page contains even *more* sensitive information than the
47 404 page and should *never* be exposed to the public.
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
48
49 In short, setting ``DEBUG`` to ``True`` tells Django to assume only trusted
50 developers are using your site. The Internet is full of untrustworthy
51 hooligans, and the first thing you should do when you're preparing your
52 application for deployment is set ``DEBUG`` to ``False``.
53
54 Turning Off Template Debug Mode
55 -------------------------------
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
56
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
57 Similarly, you should set ``TEMPLATE_DEBUG`` to ``False`` in production. If
58 ``True``, this setting tells Django's template system to save some extra
59 information about every template, for use on the pretty error pages.
60
61 Implementing a 404 Template
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
62 ---------------------------
63
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
64 If ``DEBUG`` is ``True``, Django displays the useful 404 error page. But if
65 ``DEBUG`` is ``False``, then it does something different: it renders a template
66 called ``404.html`` in your root template directory. So, when you're ready to
67 deploy, you'll need to create this template and put a useful "Page not found"
68 message in it.
69
70 Here's a sample ``404.html`` you can use as a starting point. It assumes you're
71 using template inheritance and have defined a ``base.html`` with blocks called
72 ``title`` and ``content``.
73
74 ::
75
76 {% extends "base.html" %}
77
78 {% block title %}Page not found{% endblock %}
79
80 {% block content %}
81 <h1>Page not found</h1>
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
82
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
83 <p>Sorry, but the requested page could not be found.</p>
84 {% endblock %}
85
86 To test that your ``404.html`` is working, just change ``DEBUG`` to ``False``
87 and visit a nonexistent URL. (This works on the ``runserver`` just as well as
88 it works on a production server.)
89
90 Implementing a 500 Template
91 ---------------------------
92
93 Similarly, if ``DEBUG`` is ``False``, then Django no longer displays its useful
94 error/traceback pages in case of an unhandled Python exception. Instead, it
95 looks for a template called ``500.html`` and renders it. Like ``404.html``,
96 this template should live in your root template directory.
97
98 There's one slightly tricky thing about ``500.html``. You can never be sure
99 *why* this template is being rendered, so it shouldn't do anything that
100 requires a database connection or relies on any potentially broken part of your
101 infrastructure. (For example, it should not use custom template tags.) If it
102 uses template inheritance, then the parent template(s) shouldn't rely on
103 potentially broken infrastructure, either. Therefore, the best approach is to
104 avoid template inheritance and use something very simple. Here's an example
105 ``500.html`` as a starting point::
106
107 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN"
108 "http://www.w3.org/TR/html4/strict.dtd">
109 <html lang="en">
110 <head>
111 <title>Page unavailable</title>
112 </head>
113 <body>
114 <h1>Page unavailable</h1>
115
116 <p>Sorry, but the requested page is unavailable due to a
117 server hiccup.</p>
118
119 <p>Our engineers have been notified, so check back later.</p>
120 </body>
121 </html>
122
123 Setting Up Error Alerts
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
124 -----------------------
125
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
126 When your Django-powered site is running and an exception is raised, you'll
127 want to know about it, so you can fix it. By default, Django is configured to
128 send an e-mail to the site developers whenever your code raises an unhandled
129 exception -- but you need to do two things to set it up.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
130
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
131 First, change your ``ADMINS`` setting to include your e-mail address, along
132 with the e-mail addresses of any other people who need to be notified. This
133 setting takes a tuple of ``(name, email)`` tuples, like this::
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
134
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
135 ADMINS = (
136 ('John Lennon', 'jlennon@example.com'),
137 ('Paul McCartney', 'pmacca@example.com'),
138 )
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
139
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
140 Second, make sure your server is configured to send e-mail. Setting up
141 ``postfix``, ``sendmail`` or any other mail server is outside the scope of this
142 book, but on the Django side of things, you'll want to make sure your
143 ``EMAIL_HOST`` setting is set to the proper hostname for your mail server.
144 It's set to ``'localhost'`` by default, which works out of the box for most
145 shared-hosting environments. You might also need to set ``EMAIL_HOST_USER``,
146 ``EMAIL_HOST_PASSWORD``, ``EMAIL_PORT`` or ``EMAIL_USE_TLS``, depending on the
147 complexity of your arrangement.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
148
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
149 Also, you can set ``EMAIL_SUBJECT_PREFIX`` to control the prefix Django uses
150 in front of its error e-mails. It's set to ``'[Django] '`` by default.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
151
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
152 Setting Up Broken Link Alerts
153 -----------------------------
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
154
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
155 If you have the ``CommonMiddleware`` installed (e.g., if your
156 ``MIDDLEWARE_CLASSES`` setting includes
157 ``'django.middleware.common.CommonMiddleware'``, which it does by default),
158 then you have the option of receiving an e-mail any time somebody visits a page
159 on your Django-powered site that raises 404 with a non-empty referrer -- that
160 is, every broken link. If you want to activate this feature, set
161 ``SEND_BROKEN_LINK_EMAILS`` to ``True`` (it's ``False`` by default), and set
162 your ``MANAGERS`` setting to a person or people who will receive these
163 broken-link e-mails. ``MANAGERS`` uses the same syntax as ``ADMINS``. For
164 example::
165
166 MANAGERS = (
167 ('George Harrison', 'gharrison@example.com'),
168 ('Ringo Starr', 'ringo@example.com'),
169 )
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
170
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
171 Note that error e-mails can get annoying; they're not for everybody.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
172
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
173 Using Different Settings for Production
174 =======================================
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
175
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
176 So far in this book, we've dealt with only a single settings file: the
177 ``settings.py`` generated by ``django-admin.py startproject``. But as you get
178 ready to deploy, you'll likely find yourself needing multiple settings files to
179 keep your development environment isolated from your production environment.
180 (For example, you probably won't want to change ``DEBUG`` from ``False`` to
181 ``True`` whenever you want to test code changes on your local machine.) Django
182 makes this very easy by allowing you to use multiple settings files.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
183
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
184 If you'd like to organize your settings files into "production" and
185 "development" settings, you can accomplish this in one of three ways:
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
186
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
187 * Set up two full-blown, independent settings files.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
188
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
189 * Set up a "base" settings file (say, for development) and a second (say,
190 production) settings file that merely imports from the first one and
191 defines whatever overrides it needs to define.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
192
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
193 * Use only a single settings file that has Python logic to change the
194 settings based on context.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
195
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
196 We'll take these one at a time.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
197
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
198 First, the most basic approach is to define two separate settings files. If
199 you're following along, you've already got ``settings.py``. Now, just make a
200 copy of it called ``settings_production.py``. (We made this name up; you can
201 call it whatever you want.) In this new file, change ``DEBUG``, etc.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
202
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
203 The second approach is similar but cuts down on redundancy. Instead of having
204 two settings files whose contents are mostly similar, you can treat one as the
205 "base" file and create another file that imports from it. For example::
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
206
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
207 # settings.py
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
208
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
209 DEBUG = True
210 TEMPLATE_DEBUG = DEBUG
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
211
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
212 DATABASE_ENGINE = 'postgresql_psycopg2'
213 DATABASE_NAME = 'devdb'
214 DATABASE_USER = ''
215 DATABASE_PASSWORD = ''
216 DATABASE_PORT = ''
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
217
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
218 # ...
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
219
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
220 # settings_production.py
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
221
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
222 from settings import *
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
223
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
224 DEBUG = TEMPLATE_DEBUG = False
225 DATABASE_NAME = 'production'
226 DATABASE_USER = 'app'
227 DATABASE_PASSWORD = 'letmein'
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
228
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
229 Here, ``settings_production.py`` imports everything from ``settings.py`` and
230 just redefines the settings that are particular to production. In this case,
231 ``DEBUG`` is set to ``False``, but we've also set different database access
232 parameters for the production setting. (The latter goes to show that you can
233 redefine *any* setting, not just the basic ones like ``DEBUG``.)
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
234
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
235 Finally, the most concise way of accomplishing two settings environments is to
236 use a single settings file that branches based on the environment. One way to
237 do this is to check the current hostname. For example::
238
239 # settings.py
240
241 import socket
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
242
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
243 if socket.gethostname() == 'my-laptop':
244 DEBUG = TEMPLATE_DEBUG = True
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
245 else:
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
246 DEBUG = TEMPLATE_DEBUG = False
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
247
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
248 # ...
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
249
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
250 Here, we import the ``socket`` module from Python's standard library and use it
251 to check the current system's hostname. We can check the hostname to determine
252 whether the code is being run on the production server.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
253
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
254 A core lesson here is that settings files are *just Python code*. They can
255 import from other files, they can execute arbitrary logic, etc. Just make sure
256 that, if you go down this road, the Python code in your settings files is
257 bulletproof. If it raises any exceptions, Django will likely crash badly.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
258
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
259 .. admonition:: Renaming settings.py
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
260
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
261 Feel free to rename your ``settings.py`` to ``settings_dev.py`` or
262 ``settings/dev.py`` or ``foobar.py`` -- Django doesn't care, as long as
263 you tell it what settings file you're using.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
264
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
265 But if you *do* rename the ``settings.py`` file that is generated by
266 ``django-admin.py startproject``, you'll find that ``manage.py`` will give
267 you an error message saying that it can't find the settings. That's because
268 it tries to import a module called ``settings``. You can fix this either by
269 editing ``manage.py`` to change ``settings`` to the name of your module, or
270 by using ``django-admin.py`` instead of ``manage.py``. In the latter case,
271 you'll need to set the ``DJANGO_SETTINGS_MODULE`` environment variable to
272 the Python path to your settings file (e.g., ``'mysite.settings'``).
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
273
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
274 DJANGO_SETTINGS_MODULE
275 ======================
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
276
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
277 With those code changes out of the way, the next part of this chapter will
278 focus on deployment instructions for specific environments, such as Apache.
279 The instructions are different for each environment, but one thing remains the
280 same: in each case, you will have to tell the Web server your
281 ``DJANGO_SETTINGS_MODULE``. This is the entry point into your Django
282 application. The ``DJANGO_SETTINGS_MODULE`` points to your settings file, which
283 points to your ``ROOT_URLCONF``, which points to your views, and so on.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
284
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
285 ``DJANGO_SETTINGS_MODULE`` is the Python path to your settings file. For
286 example, assuming the ``mysite`` directory is on your Python path, the
287 ``DJANGO_SETTINGS_MODULE`` for our ongoing example is ``'mysite.settings'``.
288
289 Using Django with Apache and mod_python
290 =======================================
291
292 Apache with mod_python historically has been the suggested setup for using
293 Django on a production server.
294
295 mod_python (http://www.djangoproject.com/r/mod_python/) is an Apache plug-in
296 that embeds Python within Apache and loads Python code into memory when the
297 server starts. Code stays in memory throughout the life of an Apache process,
298 which leads to significant performance gains over other server arrangements.
299
300 Django requires Apache 2.x and mod_python 3.x.
301
302 .. note::
303
304 Configuring Apache is *well* beyond the scope of this book, so
305 we'll simply mention details as needed. Luckily, many great resources are
306 available if you need to learn more about Apache. A few of them we like
307 are:
308
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
309 * The free online Apache documentation, available via
310 http://www.djangoproject.com/r/apache/docs/
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
311
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
312 * *Pro Apache, Third Edition* (Apress, 2004) by Peter Wainwright,
313 available via http://www.djangoproject.com/r/books/pro-apache/
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
314
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
315 * *Apache: The Definitive Guide, Third Edition* (O'Reilly, 2002) by Ben
316 Laurie and Peter Laurie, available via
317 http://www.djangoproject.com/r/books/pro-apache/
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
318
319 Basic Configuration
320 -------------------
321
322 To configure Django with mod_python, first make sure you have Apache installed
323 with the mod_python module activated. This usually means having a
324 ``LoadModule`` directive in your Apache configuration file. It will look something
325 like this::
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
326
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
327 LoadModule python_module /usr/lib/apache2/modules/mod_python.so
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
328
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
329 Then, edit your Apache configuration file and add a ``<Location>`` directive
330 that ties a specific URL path to a specific Django installation. For example::
331
332 <Location "/">
333 SetHandler python-program
334 PythonHandler django.core.handlers.modpython
335 SetEnv DJANGO_SETTINGS_MODULE mysite.settings
336 PythonDebug Off
337 </Location>
338
339 Make sure to replace ``mysite.settings`` with the appropriate
340 ``DJANGO_SETTINGS_MODULE`` for your site.
341
342 This tells Apache, "Use mod_python for any URL at or under '/', using the
343 Django mod_python handler." It passes the value of ``DJANGO_SETTINGS_MODULE``
344 so mod_python knows which settings to use.
345
346 Note that we're using the ``<Location>`` directive, not the ``<Directory>``
347 directive. The latter is used for pointing at places on your filesystem,
348 whereas ``<Location>`` points at places in the URL structure of a Web site.
349 ``<Directory>`` would be meaningless here.
350
351 Apache likely runs as a different user than your normal login and may have a
352 different path and sys.path. You may need to tell mod_python how to find your
353 project and Django itself. ::
354
355 PythonPath "['/path/to/project', '/path/to/django'] + sys.path"
356
357 You can also add directives such as ``PythonAutoReload Off`` for performance.
358 See the mod_python documentation for a full list of options.
359
360 Note that you should set ``PythonDebug Off`` on a production server. If you
361 leave ``PythonDebug On``, your users will see ugly (and revealing) Python
362 tracebacks if something goes wrong within mod_python.
363
364 Restart Apache, and any request to your site (or virtual host if you've put
365 this directive inside a ``<VirtualHost>`` block) will be served by Django.
366
367 Running Multiple Django Installations on the Same Apache Instance
368 -----------------------------------------------------------------
369
370 It's entirely possible to run multiple Django installations on the same Apache
371 instance. You might want to do this if you're an independent Web developer with
372 multiple clients but only a single server.
373
374 To accomplish this, just use ``VirtualHost`` like so::
375
376 NameVirtualHost *
377
378 <VirtualHost *>
379 ServerName www.example.com
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
380 # ...
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
381 SetEnv DJANGO_SETTINGS_MODULE mysite.settings
382 </VirtualHost>
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
383
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
384 <VirtualHost *>
385 ServerName www2.example.com
386 # ...
387 SetEnv DJANGO_SETTINGS_MODULE mysite.other_settings
388 </VirtualHost>
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
389
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
390 If you need to put two Django installations within the same ``VirtualHost``,
391 you'll need to take a special precaution to ensure mod_python's code cache
392 doesn't mess things up. Use the ``PythonInterpreter`` directive to give
393 different ``<Location>`` directives separate interpreters::
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
394
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
395 <VirtualHost *>
396 ServerName www.example.com
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
397 # ...
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
398 <Location "/something">
399 SetEnv DJANGO_SETTINGS_MODULE mysite.settings
400 PythonInterpreter mysite
401 </Location>
402
403 <Location "/otherthing">
404 SetEnv DJANGO_SETTINGS_MODULE mysite.other_settings
405 PythonInterpreter mysite_other
406 </Location>
407 </VirtualHost>
408
409 The values of ``PythonInterpreter`` don't really matter, as long as they're
410 different between the two ``Location`` blocks.
411
412 Running a Development Server with mod_python
413 --------------------------------------------
414
415 Because mod_python caches loaded Python code, when deploying Django sites on
416 mod_python you'll need to restart Apache each time you make changes to your
417 code. This can be a hassle, so here's a quick trick to avoid it: just add
418 ``MaxRequestsPerChild 1`` to your config file to force Apache to reload
419 everything for each request. But don't do that on a production server, or we'll
420 revoke your Django privileges.
421
422 If you're the type of programmer who debugs using scattered ``print``
423 statements (we are), note that ``print`` statements have no effect in
424 mod_python; they don't appear in the Apache log, as you might expect. If you
425 have the need to print debugging information in a mod_python setup, you'll
426 probably want to use Python's standard logging package. More information is
427 available at http://docs.python.org/lib/module-logging.html.
428
429 Serving Django and Media Files from the Same Apache Instance
430 ------------------------------------------------------------
431
432 Django should not be used to serve media files itself; leave that job to
433 whichever Web server you choose. We recommend using a separate Web server
434 (i.e., one that's not also running Django) for serving media. For more
435 information, see the "Scaling" section.
436
437 If, however, you have no option but to serve media files on the same Apache
438 ``VirtualHost`` as Django, here's how you can turn off mod_python for a
439 particular part of the site::
440
441 <Location "/media/">
442 SetHandler None
443 </Location>
444
445 Change ``Location`` to the root URL of your media files.
446
447 You can also use ``<LocationMatch>`` to match a regular expression. For
448 example, this sets up Django at the site root but explicitly disables Django
449 for the ``media`` subdirectory and any URL that ends with ``.jpg``, ``.gif``,
450 or ``.png``::
451
452 <Location "/">
453 SetHandler python-program
454 PythonHandler django.core.handlers.modpython
455 SetEnv DJANGO_SETTINGS_MODULE mysite.settings
456 </Location>
457
458 <Location "/media/">
459 SetHandler None
460 </Location>
461
462 <LocationMatch "\.(jpg|gif|png)$">
463 SetHandler None
464 </LocationMatch>
465
466 In all of these cases, you'll need to set the ``DocumentRoot`` directive so
467 Apache knows where to find your static files.
468
469 Error Handling
470 --------------
471
472 When you use Apache/mod_python, errors will be caught by Django -- in other
473 words, they won't propagate to the Apache level and won't appear in the Apache
474 ``error_log``.
475
476 The exception to this is if something is really messed up in your Django
477 setup. In that case, you'll see an ominous "Internal Server Error" page in your
478 browser and the full Python traceback in your Apache ``error_log`` file. The
479 ``error_log`` traceback is spread over multiple lines. (Yes, this is ugly and
480 rather hard to read, but it's how mod_python does things.)
481
482 Handling a Segmentation Fault
483 -----------------------------
484
485 Sometimes, Apache segfaults when you install Django. When this happens, it's
486 almost *always* one of two causes mostly unrelated to Django itself:
487
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
488 * It may be that your Python code is importing the ``pyexpat`` module
489 (used for XML parsing), which may conflict with the version embedded in
490 Apache. For full information, see "Expat Causing Apache Crash" at
491 http://www.djangoproject.com/r/articles/expat-apache-crash/.
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
492
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
493 * It may be because you're running mod_python and mod_php in the same
494 Apache instance, with MySQL as your database backend. In some cases, this
495 causes a known mod_python issue due to version conflicts in PHP and the
496 Python MySQL back-end. There's full information in a mod_python FAQ entry,
497 accessible via http://www.djangoproject.com/r/articles/php-modpython-faq/.
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
498
499 If you continue to have problems setting up mod_python, a good thing to do is
500 get a bare-bones mod_python site working, without the Django framework. This is
501 an easy way to isolate mod_python-specific problems. The article "Getting mod_python
502 Working" details this procedure:
503 http://www.djangoproject.com/r/articles/getting-modpython-working/.
504
505 The next step should be to edit your test code and add an import of any
506 Django-specific code you're using -- your views, your models, your URLconf,
507 your RSS configuration, and so forth. Put these imports in your test handler function
508 and access your test URL in a browser. If this causes a crash, you've
509 confirmed it's the importing of Django code that causes the problem. Gradually
510 reduce the set of imports until it stops crashing, so as to find the specific
511 module that causes the problem. Drop down further into modules and look into
512 their imports as necessary. For more help, system tools like ``ldconfig`` on
513 Linux, ``otool`` on Mac OS, and ``ListDLLs`` (from SysInternals) on Windows
514 can help you identify shared dependencies and possible version conflicts.
515
516 An Alternative: mod_wsgi
517 ------------------------
518
519 As an alternative to mod_python, you might consider using mod_wsgi
520 (http://code.google.com/p/modwsgi/), which has been developed more recently
521 than mod_python and is getting some traction in the Django community. A full
522 overview is outside the scope of this book, but see the official Django
523 documentation for more information.
524
525 Using Django with FastCGI
526 =========================
527
528 Although Django under Apache and mod_python is the most robust deployment
529 setup, many people use shared hosting, on which FastCGI is the only available
530 deployment option.
531
532 Additionally, in some situations, FastCGI allows better security and possibly
533 better performance than mod_python. For small sites, FastCGI can also be more
534 lightweight than Apache.
535
536 FastCGI Overview
537 ----------------
538
539 FastCGI is an efficient way of letting an external application serve pages to
540 a Web server. The Web server delegates the incoming Web requests (via a
541 socket) to FastCGI, which executes the code and passes the response back to
542 the Web server, which, in turn, passes it back to the client's Web browser.
543
544 Like mod_python, FastCGI allows code to stay in memory, allowing requests to
545 be served with no startup time. Unlike mod_python, a FastCGI process doesn't
546 run inside the Web server process, but in a separate, persistent process.
547
548 .. admonition:: Why Run Code in a Separate Process?
549
550 The traditional ``mod_*`` arrangements in Apache embed various scripting
551 languages (most notably PHP, Python/mod_python, and Perl/mod_perl) inside
552 the process space of your Web server. Although this lowers startup time
553 (because code doesn't have to be read off disk for every request), it comes
554 at the cost of memory use.
555
556 Each Apache process gets a copy of the Apache engine, complete with all
557 the features of Apache that Django simply doesn't take advantage of.
558 FastCGI processes, on the other hand, only have the memory overhead of
559 Python and Django.
560
561 Due to the nature of FastCGI, it's also possible to have processes that
562 run under a different user account than the Web server process. That's a
563 nice security benefit on shared systems, because it means you can secure
564 your code from other users.
565
566 Before you can start using FastCGI with Django, you'll need to install ``flup``,
567 a Python library for dealing with FastCGI. Some users have reported
568 stalled pages with older ``flup`` versions, so you may want to use the latest
569 SVN version. Get ``flup`` at http://www.djangoproject.com/r/flup/.
570
571 Running Your FastCGI Server
572 ---------------------------
573
574 FastCGI operates on a client/server model, and in most cases you'll be
575 starting the FastCGI server process on your own. Your Web server (be it
576 Apache, lighttpd, or otherwise) contacts your Django-FastCGI process only when
577 the server needs a dynamic page to be loaded. Because the daemon is already
578 running with the code in memory, it's able to serve the response very quickly.
579
580 .. admonition:: Note
581
582 If you're on a shared hosting system, you'll probably be forced to use Web
583 server-managed FastCGI processes. If you're in this situation, you should
584 read the section titled "Running Django on a Shared-Hosting Provider with
585 Apache," below.
586
587 A Web server can connect to a FastCGI server in one of two ways: it can use
588 either a Unix domain socket (a *named pipe* on Win32 systems) or a
589 TCP socket. What you choose is a manner of preference; a TCP socket is usually
590 easier due to permissions issues.
591
592 To start your server, first change into the directory of your project
593 (wherever your ``manage.py`` is), and then run ``manage.py`` with the
594 ``runfcgi`` command::
595
596 ./manage.py runfcgi [options]
597
598 If you specify ``help`` as the only option after ``runfcgi``, a
599 list of all the available options will display.
600
601 You'll need to specify either a ``socket`` or both ``host`` and ``port``.
602 Then, when you set up your Web server, you'll just need to point it at the
603 socket or host/port you specified when starting the FastCGI server.
604
605 A few examples should help explain this:
606
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
607 * Running a threaded server on a TCP port::
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
608
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
609 ./manage.py runfcgi method=threaded host=127.0.0.1 port=3033
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
610
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
611 * Running a preforked server on a Unix domain socket::
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
612
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
613 ./manage.py runfcgi method=prefork socket=/home/user/mysite.sock pidfile=django.pid
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
614
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
615 * Run without daemonizing (backgrounding) the process (good for
616 debugging)::
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
617
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
618 ./manage.py runfcgi daemonize=false socket=/tmp/mysite.sock
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
619
620 Stopping the FastCGI Daemon
621 ```````````````````````````
622
623 If you have the process running in the foreground, it's easy enough to stop
624 it: simply press Ctrl+C to stop and quit the FastCGI server. However,
625 when you're dealing with background processes, you'll need to resort to the
626 Unix ``kill`` command.
627
628 If you specify the ``pidfile`` option to your ``manage.py runfcgi``, you can
629 kill the running FastCGI daemon like this::
630
631 kill `cat $PIDFILE`
632
633 where ``$PIDFILE`` is the ``pidfile`` you specified.
634
635 To easily restart your FastCGI daemon on Unix, you can use this small shell
636 script::
637
638 #!/bin/bash
639
640 # Replace these three settings.
641 PROJDIR="/home/user/myproject"
642 PIDFILE="$PROJDIR/mysite.pid"
643 SOCKET="$PROJDIR/mysite.sock"
644
645 cd $PROJDIR
646 if [ -f $PIDFILE ]; then
647 kill `cat -- $PIDFILE`
648 rm -f -- $PIDFILE
649 fi
650
651 exec /usr/bin/env - \
652 PYTHONPATH="../python:.." \
653 ./manage.py runfcgi socket=$SOCKET pidfile=$PIDFILE
654
655 Using Django with Apache and FastCGI
656 ------------------------------------
657
658 To use Django with Apache and FastCGI, you'll need Apache installed and
659 configured, with mod_fastcgi installed and enabled. Consult the Apache and
660 mod_fastcgi documentation for instructions:
661 http://www.djangoproject.com/r/mod_fastcgi/.
662
663 Once you've completed the setup, point Apache at your Django FastCGI instance by
664 editing the ``httpd.conf`` (Apache configuration) file. You'll need to do two
665 things:
666
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
667 * Use the ``FastCGIExternalServer`` directive to specify the location of
668 your FastCGI server.
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
669
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
670 * Use ``mod_rewrite`` to point URLs at FastCGI as appropriate.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
671
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
672 Specifying the Location of the FastCGI Server
673 `````````````````````````````````````````````
674
675 The ``FastCGIExternalServer`` directive tells Apache how to find your FastCGI
676 server. As the FastCGIExternalServer docs
677 (http://www.djangoproject.com/r/mod_fastcgi/FastCGIExternalServer/) explain, you
678 can specify either a ``socket`` or a ``host``. Here are examples of both::
679
680 # Connect to FastCGI via a socket/named pipe:
681 FastCGIExternalServer /home/user/public_html/mysite.fcgi -socket /home/user/mysite.sock
682
683 # Connect to FastCGI via a TCP host/port:
684 FastCGIExternalServer /home/user/public_html/mysite.fcgi -host 127.0.0.1:3033
685
686 In either case, the the directory /home/user/public_html/ should exist,
687 though the file ``/home/user/public_html/mysite.fcgi`` doesn't
688 actually have to exist. It's just a URL used by the Web server internally -- a
689 hook for signifying which requests at a URL should be handled by FastCGI.
690 (More on this in the next section.)
691
692 Using mod_rewrite to Point URLs at FastCGI
693 ``````````````````````````````````````````
694
695 The second step is telling Apache to use FastCGI for URLs that match a certain
696 pattern. To do this, use the mod_rewrite module and rewrite URLs to
697 ``mysite.fcgi`` (or whatever you specified in the ``FastCGIExternalServer``
698 directive, as explained in the previous section).
699
700 In this example, we tell Apache to use FastCGI to handle any request that
701 doesn't represent a file on the filesystem and doesn't start with ``/media/``.
702 This is probably the most common case, if you're using Django's admin site::
703
704 <VirtualHost 12.34.56.78>
705 ServerName example.com
706 DocumentRoot /home/user/public_html
707 Alias /media /home/user/python/django/contrib/admin/media
708 RewriteEngine On
709 RewriteRule ^/(media.*)$ /$1 [QSA,L]
710 RewriteCond %{REQUEST_FILENAME} !-f
711 RewriteRule ^/(.*)$ /mysite.fcgi/$1 [QSA,L]
712 </VirtualHost>
713
714 FastCGI and lighttpd
715 --------------------
716
717 lighttpd (http://www.djangoproject.com/r/lighttpd/) is a lightweight Web server
718 commonly used for serving static files. It supports FastCGI natively and thus
719 is also an ideal choice for serving both static and dynamic pages, if your site
720 doesn't have any Apache-specific needs.
721
722 Make sure ``mod_fastcgi`` is in your modules list, somewhere after
723 ``mod_rewrite`` and ``mod_access``, but not after ``mod_accesslog``. You'll
724 probably want ``mod_alias`` as well, for serving admin media.
725
726 Add the following to your lighttpd config file::
727
728 server.document-root = "/home/user/public_html"
729 fastcgi.server = (
730 "/mysite.fcgi" => (
731 "main" => (
732 # Use host / port instead of socket for TCP fastcgi
733 # "host" => "127.0.0.1",
734 # "port" => 3033,
735 "socket" => "/home/user/mysite.sock",
736 "check-local" => "disable",
737 )
738 ),
739 )
740 alias.url = (
741 "/media/" => "/home/user/django/contrib/admin/media/",
742 )
743
744 url.rewrite-once = (
745 "^(/media.*)$" => "$1",
746 "^/favicon\.ico$" => "/media/favicon.ico",
747 "^(/.*)$" => "/mysite.fcgi$1",
748 )
749
750 Running Multiple Django Sites on One lighttpd Instance
751 ``````````````````````````````````````````````````````
752
753 lighttpd lets you use "conditional configuration" to allow configuration to be
754 customized per host. To specify multiple FastCGI sites, just add a conditional
755 block around your FastCGI config for each site::
756
757 # If the hostname is 'www.example1.com'...
758 $HTTP["host"] == "www.example1.com" {
759 server.document-root = "/foo/site1"
760 fastcgi.server = (
761 ...
762 )
763 ...
764 }
765
766 # If the hostname is 'www.example2.com'...
767 $HTTP["host"] == "www.example2.com" {
768 server.document-root = "/foo/site2"
769 fastcgi.server = (
770 ...
771 )
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
772 ...
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
773 }
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
774
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
775 You can also run multiple Django installations on the same site simply by
776 specifying multiple entries in the ``fastcgi.server`` directive. Add one
777 FastCGI host for each.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
778
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
779 Running Django on a Shared-Hosting Provider with Apache
780 -------------------------------------------------------
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
781
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
782 Many shared-hosting providers don't allow you to run your own server daemons
783 or edit the ``httpd.conf`` file. In these cases, it's still possible to run
784 Django using Web server-spawned processes.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
785
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
786 .. admonition:: Note
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
787
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
788 If you're using Web server-spawned processes, as explained in this
789 section, there's no need for you to start the FastCGI server on your own.
790 Apache will spawn a number of processes, scaling as it needs to.
791
792 In your Web root directory, add this to a file named ``.htaccess`` ::
793
794 AddHandler fastcgi-script .fcgi
795 RewriteEngine On
796 RewriteCond %{REQUEST_FILENAME} !-f
797 RewriteRule ^(.*)$ mysite.fcgi/$1 [QSA,L]
798
799 Then, create a small script that tells Apache how to spawn your FastCGI
800 program. Create a file, ``mysite.fcgi``, and place it in your Web directory, and
801 be sure to make it executable ::
802
803 #!/usr/bin/python
804 import sys, os
805
806 # Add a custom Python path.
807 sys.path.insert(0, "/home/user/python")
808
809 # Switch to the directory of your project. (Optional.)
810 # os.chdir("/home/user/myproject")
811
812 # Set the DJANGO_SETTINGS_MODULE environment variable.
813 os.environ['DJANGO_SETTINGS_MODULE'] = "myproject.settings"
814
815 from django.core.servers.fastcgi import runfastcgi
816 runfastcgi(method="threaded", daemonize="false")
817
818 Restarting the Spawned Server
819 `````````````````````````````
820
821 If you change any Python code on your site, you'll need to tell FastCGI the
822 code has changed. But there's no need to restart Apache in this case. Rather,
823 just reupload ``mysite.fcgi`` -- or edit the file -- so that the timestamp
824 on the file changes. When Apache sees the file has been updated, it will
825 restart your Django application for you.
826
827 If you have access to a command shell on a Unix system, you can accomplish
828 this easily by using the ``touch`` command::
829
830 touch mysite.fcgi
831
832 Scaling
833 =======
834
835 Now that you know how to get Django running on a single server, let's look at
836 how you can scale out a Django installation. This section walks through how
837 a site might scale from a single server to a large-scale cluster that could
838 serve millions of hits an hour.
839
840 It's important to note, however, that nearly every large site is large in
841 different ways, so scaling is anything but a one-size-fits-all operation. The
842 following coverage should suffice to show the general principle, and whenever
843 possible we'll try to point out where different choices could be made.
844
845 First off, we'll make a pretty big assumption and exclusively talk about
846 scaling under Apache and mod_python. Though we know of a number of successful
847 medium- to large-scale FastCGI deployments, we're much more familiar with
848 Apache.
849
850 Running on a Single Server
851 --------------------------
852
853 Most sites start out running on a single server, with an architecture that
854 looks something like Figure 12-1.
855
856 .. figure:: graphics/chapter12/scaling-1.png
857
858 Figure 12-1: a single server Django setup.
859
860 This works just fine for small- to medium-sized sites, and it's relatively cheap -- you
861 can put together a single-server site designed for Django for well under $3,000.
862
863 However, as traffic increases you'll quickly run into *resource contention*
864 between the different pieces of software. Database servers and Web servers
865 *love* to have the entire server to themselves, so when run on the same server
866 they often end up "fighting" over the same resources (RAM, CPU) that they'd
867 prefer to monopolize.
868
869 This is solved easily by moving the database server to a second machine,
870 as explained in the following section.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
871
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
872 Separating Out the Database Server
873 ----------------------------------
874
875 As far as Django is concerned, the process of separating out the database server
876 is extremely easy: you'll simply need to change the ``DATABASE_HOST``
877 setting to the IP or DNS name of your database server. It's probably a good idea
878 to use the IP if at all possible, as relying on DNS for the connection between
879 your Web server and database server isn't recommended.
880
881 With a separate database server, our architecture now looks like Figure 12-2.
882
883 .. figure:: graphics/chapter12/scaling-2.png
884
885 Figure 12-2: Moving the database onto a dedicated server.
886
887 Here we're starting to move into what's usually called *n-tier*
888 architecture. Don't be scared by the buzzword -- it just refers to the fact that
889 different "tiers" of the Web stack get separated out onto different physical
890 machines.
891
892 At this point, if you anticipate ever needing to grow beyond a single database
893 server, it's probably a good idea to start thinking about connection pooling
894 and/or database replication. Unfortunately, there's not nearly enough space to do
895 those topics justice in this book, so you'll need to consult your database's
896 documentation and/or community for more information.
897
898 Running a Separate Media Server
899 -------------------------------
900
901 We still have a big problem left over from the single-server setup:
902 the serving of media from the same box that handles dynamic content.
903
904 Those two activities perform best under different circumstances, and by smashing
905 them together on the same box you end up with neither performing particularly
906 well. So the next step is to separate out the media -- that is, anything *not*
907 generated by a Django view -- onto a dedicated server (see Figure 12-3).
908
909 .. figure:: graphics/chapter12/scaling-3.png
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
910
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
911 Figure 12-3: Separating out the media server.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
912
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
913 Ideally, this media server should run a stripped-down Web server optimized for
914 static media delivery. lighttpd and tux (http://www.djangoproject.com/r/tux/)
915 are both excellent choices here, but a heavily stripped down Apache could work,
916 too.
917
918 For sites heavy in static content (photos, videos, etc.), moving to a
919 separate media server is doubly important and should likely be the *first*
920 step in scaling up.
921
922 This step can be slightly tricky, however. If your application involves file
923 uploads, Django needs to be able to write uploaded media to the media server.
924 If media lives on another server, you'll need to arrange a way for that write
925 to happen across the network.
926
927 Implementing Load Balancing and Redundancy
928 ------------------------------------------
929
930 At this point, we've broken things down as much as possible. This
931 three-server setup should handle a very large amount of traffic -- we served
932 around 10 million hits a day from an architecture of this sort -- so if you
933 grow further, you'll need to start adding redundancy.
934
935 This is a good thing, actually. One glance at Figure 12-3 shows you that
936 if even a single one of your three servers fails, you'll bring down your
937 entire site. So as you add redundant servers, not only do you increase capacity,
938 but you also increase reliability.
939
940 For the sake of this example, let's assume that the Web server hits capacity
941 first. It's relatively easy to get multiple copies of a Django site running on
942 different hardware -- just copy all the code onto multiple machines, and start
943 Apache on both of them.
944
945 However, you'll need another piece of software to distribute traffic over your
946 multiple servers: a *load balancer*. You can buy expensive and proprietary
947 hardware load balancers, but there are a few high-quality open source software
948 load balancers out there.
949
950 Apache's ``mod_proxy`` is one option, but we've found Perlbal
951 (http://www.djangoproject.com/r/perlbal/) to be fantastic. It's a load
952 balancer and reverse proxy written by the same folks who wrote ``memcached``
953 (see `Chapter 15`_).
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
954
955 .. note::
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
956
957 If you're using FastCGI, you can accomplish this same distribution/load
958 balancing step by separating your front-end Web servers and back-end
959 FastCGI processes onto different machines. The front-end server
960 essentially becomes the load balancer, and the back-end FastCGI processes
961 replace the Apache/mod_python/Django servers.
962
963 With the Web servers now clustered, our evolving architecture starts to look
964 more complex, as shown in Figure 12-4.
965
966 .. figure:: graphics/chapter12/scaling-4.png
967
968 Figure 12-4: A load-balanced, redundant server setup.
969
970 Notice that in the diagram the Web servers are referred to as a "cluster" to
971 indicate that the number of servers is basically variable. Once you have a
972 load balancer out front, you can easily add and remove back-end Web servers
973 without a second of downtime.
974
975 Going Big
976 ---------
977
978 At this point, the next few steps are pretty much derivatives of the last one:
979
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
980 * As you need more database performance, you might want to add replicated
981 database servers. MySQL includes built-in replication; PostgreSQL
982 users should look into Slony (http://www.djangoproject.com/r/slony/)
983 and pgpool (http://www.djangoproject.com/r/pgpool/) for replication and
984 connection pooling, respectively.
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
985
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
986 * If the single load balancer isn't enough, you can add more load
987 balancer machines out front and distribute among them using
988 round-robin DNS.
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
989
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
990 * If a single media server doesn't suffice, you can add more media
991 servers and distribute the load with your load-balancing cluster.
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
992
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
993 * If you need more cache storage, you can add dedicated cache servers.
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
994
fdc4800 @jacobian Fixed a bunch of reST markup errors.
jacobian authored
995 * At any stage, if a cluster isn't performing well, you can add more
996 servers to the cluster.
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
997
998 After a few of these iterations, a large-scale architecture might look like Figure 12-5.
999
1000 .. figure:: graphics/chapter12/scaling-5.png
1001
1002 Figure 12-5. An example large-scale Django setup.
1003
1004 Though we've shown only two or three servers at each level, there's no
1005 fundamental limit to how many you can add.
1006
1007 Performance Tuning
1008 ==================
1009
1010 If you have huge amount of money, you can just keep throwing hardware at
1011 scaling problems. For the rest of us, though, performance tuning is a must.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
1012
1013 .. note::
1014
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
1015 Incidentally, if anyone with monstrous gobs of cash is actually reading
1016 this book, please consider a substantial donation to the Django Foundation.
1017 We accept uncut diamonds and gold ingots, too.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
1018
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
1019 Unfortunately, performance tuning is much more of an art than a science, and it
1020 is even more difficult to write about than scaling. If you're serious about
1021 deploying a large-scale Django application, you should spend a great deal of
1022 time learning how to tune each piece of your stack.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
1023
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
1024 The following sections, though, present a few Django-specific tuning tips we've
1025 discovered over the years.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
1026
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
1027 There's No Such Thing As Too Much RAM
1028 -------------------------------------
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
1029
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
1030 Even the really expensive RAM is relatively affordable these days. Buy as much
1031 RAM as you can possibly afford, and then buy a little bit more.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
1032
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
1033 Faster processors won't improve performance all that much; most Web
1034 servers spend up to 90% of their time waiting on disk I/O. As soon as you start
1035 swapping, performance will just die. Faster disks might help slightly, but
1036 they're much more expensive than RAM, such that it doesn't really matter.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
1037
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
1038 If you have multiple servers, the first place to put your RAM is in the
1039 database server. If you can afford it, get enough RAM to get fit your entire
1040 database into memory. This shouldn't be too hard; we've developed a site
1041 with more than half a million newspaper articles, and it took under 2GB of
1042 space.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
1043
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
1044 Next, max out the RAM on your Web server. The ideal situation is one where
1045 neither server swaps -- ever. If you get to that point, you should be able to
1046 withstand most normal traffic.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
1047
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
1048 Turn Off Keep-Alive
1049 -------------------
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
1050
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
1051 ``Keep-Alive`` is a feature of HTTP that allows multiple HTTP requests to be
1052 served over a single TCP connection, avoiding the TCP setup/teardown overhead.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
1053
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
1054 This looks good at first glance, but it can kill the performance of a Django
1055 site. If you're properly serving media from a separate server, each user
1056 browsing your site will only request a page from your Django server every ten
1057 seconds or so. This leaves HTTP servers waiting around for the next
1058 keep-alive request, and an idle HTTP server just consumes RAM that an active one
1059 should be using.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
1060
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
1061 Use memcached
1062 -------------
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
1063
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
1064 Although Django supports a number of different cache back-ends, none of them
1065 even come *close* to being as fast as memcached. If you have a high-traffic
1066 site, don't even bother with the other backends -- go straight to memcached.
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
1067
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
1068 Use memcached Often
1069 -------------------
acc918f @jacobian Initial import of djangobook from private SVN repo.
jacobian authored
1070
d40cfe7 @jacobian Restored *2.0* version of the book, not 1.0!
jacobian authored
1071 Of course, selecting memcached does you no good if you don't actually use it.
1072 `Chapter 15`_ is your best friend here: learn how to use Django's cache
1073 framework, and use it everywhere possible. Aggressive, preemptive caching is
1074 usually the only thing that will keep a site up under major traffic.
1075
1076 .. _Chapter 15: ../chapter15/
1077
1078 Join the Conversation
1079 ---------------------
1080
1081 Each piece of the Django stack -- from Linux to Apache to PostgreSQL or MySQL
1082 -- has an awesome community behind it. If you really want to get that last 1%
1083 out of your servers, join the open source communities behind your software and
1084 ask for help. Most free-software community members will be happy to help.
1085
1086 And also be sure to join the Django community. Your humble authors are only two
1087 members of an incredibly active, growing group of Django developers. Our
1088 community has a huge amount of collective experience to offer.
1089
1090 What's Next?
1091 ============
1092
1093 The remaining chapters focus on other Django features that you may or may not
1094 need, depending on your application. Feel free to read them in any order you
1095 choose.
Something went wrong with that request. Please try again.