static/index.html

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
	"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">

<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
  <title>Testing and Django</title>

  <meta name="viewport" content="width=device-width"/>

  <link rel="stylesheet" href="./css/reset.css" type="text/css"/>
  <link rel="stylesheet" href="./css/showoff.css" type="text/css"/>

  <script type="text/javascript" src="./js/jquery-1.4.2.min.js"></script>
  <script type="text/javascript" src="./js/jquery.cycle.all.js"></script>
	<script type="text/javascript" src="./js/jquery-print.js"></script>
  <script type="text/javascript" src="./js/jquery.batchImageLoad.js"></script>

  <script type="text/javascript" src="./js/jquery.doubletap-0.1.js"></script>

  <script type="text/javascript" src="./js/fg.menu.js"></script>
  <script type="text/javascript" src="./js/showoff.js"></script>
  <script type="text/javascript" src="./js/jTypeWriter.js"> </script>
  <script type="text/javascript" src="./js/sh_main.min.js"></script>
  <script type="text/javascript" src="./js/core.js"></script>
  <script type="text/javascript" src="./js/showoffcore.js"></script>
  <script type="text/javascript" src="./js/coffee-script.js"></script>

  <link type="text/css" href="./css/fg.menu.css" media="screen" rel="stylesheet" />
  <link type="text/css" href="./css/theme/ui.all.css" media="screen" rel="stylesheet" />
  <link type="text/css" href="./css/sh_style.css" rel="stylesheet" >

  
    <link rel="stylesheet" href="file/style.css" type="text/css"/>
  

    <script type="text/javascript" src="file/antipattern.js"></script>
  
    <script type="text/javascript" src="file/timer.js"></script>
  

  <script type="text/javascript">
  $(function(){
      setupPreso(false, './');
  });
  </script>

</head>

<body>

<a tabindex="0" href="#search-engines" class="fg-button fg-button-icon-right ui-widget ui-state-default ui-corner-all" id="navmenu"><span class="ui-icon ui-icon-triangle-1-s"></span>slides</a>
<div id="navigation" class="hidden"></div>

<div id="help">
  <table>
    <tr><td class="key">z, ?</td><td>toggle help (this)</td></tr>
    <tr><td class="key">space, &rarr;</td><td>next slide</td></tr>
    <tr><td class="key">shift-space, &larr;</td><td>previous slide</td></tr>
    <tr><td class="key">d</td><td>toggle debug mode</td></tr>
    <tr><td class="key">## &lt;ret&gt;</td><td>go to slide #</td></tr>
    <tr><td class="key">c, t</td><td>table of contents (vi)</td></tr>
    <tr><td class="key">f</td><td>toggle footer</td></tr>
    <tr><td class="key">r</td><td>reload slides</td></tr>
    <tr><td class="key">n</td><td>toggle notes</td></tr>
    <tr><td class="key">p</td><td>run preshow</td></tr>
  </table>
</div>

<div class="buttonNav">
  <input type="submit" onClick="prevStep();" value="prev"/>
  <input type="submit" onClick="nextStep();" value="next"/>
</div>

<div id="preso">loading presentation...</div>
<div id="footer">
  <span id="slideInfo"></span>
  <span id="debugInfo"></span>
  <span id="notesInfo"></span>
</div>

<div id="slides" class="offscreen" style="display:none;">
<div class="slide" data-transition="none"><div class="content smaller intro" ref="title/01_title">
<h1>Testing and Django</h1>

<p><img src="./file/title/logo.png" alt="OddBird"/></p>

<h2>Carl J Meyer</h2>

<h2>@carljm</h2>

<h2>carl@oddbird.net</h2>

<h2>github.com/carljm/django-testing-slides</h2>

<h2>github.com/carljm/django-testing-slides/code</h2></div>
</div><div class="slide" data-transition="none"><div class="content commandline incremental" ref="whichtests/10_not_created_equal/1">
<h1>Let's start a project.</h1>

<pre><code><code class="command">$ django-admin.py startproject testing .</code>
<code class="result">
</code><code class="command">$ sed -i -e 's/backends\./backends.sqlite3/;' testing/settings.py</code>
<code class="result">
</code><code class="command">$ ./manage.py test</code>
<code class="result">[snip]
----------------------------------------------------------------------
Ran 412 tests in 14.235s

FAILED (errors=2, skipped=1)
Destroying test database for alias 'default'...
</code></code></pre>

<p class="notes">(Oops, guess we're not quite ready to release 1.4.)
But 412 tests? 14 extra seconds?</p></div>
</div><div class="slide" data-transition="none"><div class="content" ref="whichtests/10_not_created_equal/2">
<h2>I'll never get those 14 seconds back.</h2>

<p class="notes">Does anyone still need convincing that fast test suites matter?</p></div>
</div><div class="slide" data-transition="none"><div class="content" ref="whichtests/10_not_created_equal/3">
<h2>Not all apps are created equal.</h2>

<p class="notes">Non-isolated tests break, isolated tests are pointless to run.
Integration tests should be written by the integrator.</p></div>
</div><div class="slide" data-transition="none"><div class="content" ref="whichtests/10_not_created_equal/4">
<h2>No problem.</h2>

<h3><code>./manage.py test just my apps please</code></h3>

<p class="notes">Easy to solve with a shell script. But there's more...</p></div>
</div><div class="slide" data-transition="none"><div class="content antipattern" ref="whichtests/20_lame_discovery">
<h1><code>tests/__init__.py</code></h1>

<pre class="sh_python"><code>from .test_forms import QuoteFormTest
from .test_models import (
    QuoteTest, SourceTest)
from .test_views import (
    AddQuoteTest, EditQuoteTest,
    ListQuotesTest
    )</code></pre>

<p class="notes">Django made me do it. (Or worse yet, "import *".)</p></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="whichtests/30_problem">
<h1>Django's test discovery</h1>

<ul>
<li>Wastes my time with tests I don't care about.</li>
<li>Requires app tests to be in a single module (resulting in boilerplate
imports).</li>
<li>Forces intermingling of tests and non-test code.</li>
</ul>


<p class="notes">But there's good news...</p></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="whichtests/40_solution/1">
<h1>It's easy to change.</h1>

<ul>
<li>unittest2 discovery</li>
<li><code>TEST_RUNNER</code> setting</li>
</ul>


<p class="notes">The hipsters like nose or py.test, but unittest2 gets the job done.</p></div>
</div><div class="slide" data-transition="none"><div class="content small" ref="whichtests/40_solution/2">
<pre class="sh_python"><code>class DiscoveryRunner(DjangoTestSuiteRunner):
    """A test suite runner using unittest2 discovery."""
    def build_suite(self, test_labels, **kwargs):

        suite = None
        discovery_root = settings.TEST_DISCOVERY_ROOT

        if test_labels:
            suite = defaultTestLoader.loadTestsFromNames(
                test_labels)

        if suite is None:
            suite = defaultTestLoader.discover(
                discovery_root,
                top_level_dir=settings.BASE_PATH,
                )

        return reorder_suite(suite, (TestCase,))</code></pre>

<p class="notes">(Enhanced version in the code online with the slides.)</p></div>
</div><div class="slide" data-transition="none"><div class="content small" ref="whichtests/40_solution/3">
<h1><code>settings.py</code></h1>

<pre class="sh_python"><code>import os.path

BASE_PATH = os.path.dirname(os.path.dirname(__file__))

TEST_DISCOVERY_ROOT = os.path.join(BASE_PATH, "tests")

TEST_RUNNER = "tests.runner.DiscoveryRunner"</code></pre></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="whichtests/40_solution/4">
<h1>\o/</h1>

<ul>
<li>Discovers tests wherever you want them.</li>
<li>Doesn't run tests from external apps by default.</li>
<li>Flexible specification of specific tests to run: Python dotted path to test
module, not Django app label.</li>
<li><code>./manage.py test tests.quotes.test_views</code></li>
</ul>
</div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="whichtests/40_solution/5">
<h1>Maybe in 1.5?</h1>

<ul>
<li>https://code.djangoproject.com/ticket/17365</li>
</ul>


<p class="notes">django-nose is good too, but this could actually go in Django.</p></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="levels/10_intro/1">
<h1>Types of test</h1>

<ul>
<li><p>unit</p></li>
<li><p>system / integration / functional</p></li>
</ul>


<p class="notes">Go see the video of Gary's "Fast test, slow test" talk.</p></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="levels/10_intro/2">
<h1>Unit tests</h1>

<ul>
<li><p>Test one unit of code (a function or method) in something approaching
isolation.</p></li>
<li><p>Fast, focused (useful failures).</p></li>
<li><p>Help you structure your code better.</p></li>
</ul>
</div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="levels/10_intro/3">
<h1>Integration tests</h1>

<ul>
<li><p>Test that the whole integrated system works; catch regressions.</p></li>
<li><p>Slow.</p></li>
<li><p>Less useful failures.</p></li>
<li><p>Write fewer.</p></li>
</ul>


<p class="notes">Summary: both are useful, write more unit tests.</p></div>
</div><div class="slide" data-transition="none"><div class="content section" ref="models/05_intro">
<h1>Testing models</h1></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="models/10_dbsetup/1">
<h2>The database makes your tests slow.</h2>

<ul>
<li>Try to write tests that don't hit it at all.</li>
<li>Separate db-independent model-layer functionality from db-dependent
functionality.</li>
<li>But you'll still have a lot of tests that do.</li>
<li>Mocking the database usually isn't worth it.</li>
</ul>
</div>
</div><div class="slide" data-transition="none"><div class="content" ref="models/10_dbsetup/2">
<pre class="sh_python"><code>class Thing(models.Model):
  def frobnicate(self):
    """Frobnicate and save the thing."""
    # ... do something complicated
    self.save()</code></pre></div>
</div><div class="slide" data-transition="none"><div class="content" ref="models/10_dbsetup/3">
<pre class="sh_python"><code>def frobnicate_thing(thing):
  # ... do something complicated
  return thing


class Thing(models.Model):
  def frobnicate(self):
    """Frobnicate and save the thing."""
    frobnicate_thing(self)
    self.save()</code></pre></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="models/10_dbsetup/4">
<h1><code>django.test.TestCase</code></h1>

<ul>
<li>Runs each test within a transaction.</li>
<li>Rolls back the transaction at the end of the test.</li>
<li>Monkeypatches transaction functions to be no-ops.</li>
</ul>


<p class="notes">Django tries to make them fast...</p></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="models/10_dbsetup/5">
<h1><code>TransactionTestCase</code></h1>

<ul>
<li>Lets you test transactions in your code.</li>
<li>Has to flush every database table after every test.</li>
<li>Makes your tests extra super bonus slow.</li>
</ul>


<p class="notes">This part of the talk is boring because I have no complaints.</p></div>
</div><div class="slide" data-transition="none"><div class="content antipattern" ref="models/20_fixtures/1">
<pre class="sh_python"><code>{
    "pk": 4,
    "model": "auth.user",
    "fields": {
        "username": "manager",
        "first_name": "",
        "last_name": "",
        "is_active": true,
        "is_superuser": false,
        "is_staff": false,
        "last_login": "2012-02-06 15:06:44",</code></pre>

<p class="notes">Do you have these in your tests? BURN THEM!</p></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="models/20_fixtures/2">
<h1>Just say no.</h1>

<ul>
<li>Hard to maintain and update.</li>
<li>Increase test interdependence.</li>
<li>Slow to load.</li>
</ul>
</div>
</div><div class="slide" data-transition="none"><div class="content" ref="models/20_fixtures/3">
<h1>Model factories!</h1>

<pre class="sh_python"><code>def create_profile(**kwargs):
    defaults = {
        "likes_cheese": True,
        "age": 32,
        "address": "3815 Brookside Dr",
    }
    defaults.update(kwargs)
    if "user" not in defaults:
        defaults["user"] = create_user()
    return Profile.objects.create(
        **defaults)</code></pre>

<p class="notes">You can write simple factory functions like this (key benefit is prefilling FKs).</p></div>
</div><div class="slide" data-transition="none"><div class="content" ref="models/20_fixtures/4">
<h1>Using a factory</h1>

<pre class="sh_python"><code>def test_can_vote(self):
    """A user age 18+ can vote in the US."""
    profile = create_profile(age=18)
    self.assertTrue(profile.can_vote)</code></pre>

<p class="notes">BAD example. This test shouldn't touch the DB. So you want a smarter factory that can also build objects without saving.</p></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="models/20_fixtures/5">
<h1>Or use <code>factory_boy</code>:</h1>

<pre class="sh_python"><code>class ProfileFactory(factory.Factory):
    FACTORY_FOR = Profile

    likes_cheese = True
    age = 32
    address = "3815 Brookside Dr"
    user = factory.SubFactory(UserFactory)

profile = ProfileFactory.create(
    age=18, user__username="carljm")</code></pre>

<p class="notes">Also there's milkman, model_mommy. I don't like random data generation.</p></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="models/20_fixtures/6">
<h1>Why factories?</h1>

<ul>
<li>Test data local to test code (explicit).</li>
<li>Easy to maintain.</li>
<li>Don't create any data you don't need for that test.</li>
<li>Works great even for large/complex test data sets (helper functions).</li>
</ul>
</div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="models/30_no_database/1">
<h1>Imposing no-DB discipline.</h1>

<ul>
<li>For certain test cases.</li>
</ul>
</div>
</div><div class="slide" data-transition="none"><div class="content" ref="models/30_no_database/2">
<pre class="sh_python"><code>from django.utils.unittest import TestCase

import mock

cursor_wrapper = mock.Mock()
cursor_wrapper.side_effect = \
    RuntimeError("No touching the database!")


@mock.patch(
    "django.db.backends.util.CursorWrapper",
    cursor_wrapper)
class NoDBTestCase(TestCase):
    """Will blow up if you database."""</code></pre>

<p class="notes">django.utils.unittest.TestCase vs django.test.TestCase. Latter's assertions mostly useful with the DB, but either way works.</p></div>
</div><div class="slide" data-transition="none"><div class="content section" ref="views/10_intro">
<h1>Testing views</h1></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="views/20_unit/1">
<h1><em>Unit</em> testing views is hard.</h1>

<ul>
<li><p>Views have many collaborators / dependencies.</p></li>
<li><p>Templates, database, middleware, url routing...</p></li>
<li><p>Write less view code!</p></li>
</ul>


<p class="notes">Views have access to everything, so code in views is easy to write but
hard to maintain and debug.</p></div>
</div><div class="slide" data-transition="none"><div class="content" ref="views/20_unit/2">
<h1>If you unit test views</h1>

<ul>
<li><p>Use <code>RequestFactory</code>.</p></li>
<li><p>Call the view callable directly.</p></li>
<li><p>Set up dependencies explicitly (e.g. <code>request.user</code>, <code>request.session</code>).</p></li>
</ul>
</div>
</div><div class="slide" data-transition="none"><div class="content" ref="views/20_unit/3">
<pre class="sh_python"><code>def test_change_locale(self):
    """POST sets 'locale' key in session."""
    request = RequestFactory().post(
        "/locale/", {"locale": "es-mx"})
    request.session = {}

    change_locale(request)

    self.assertEqual(
        request.session["locale"], "es-mx")</code></pre></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="views/20_unit/4">
<h1>Or don't.</h1>

<ul>
<li><p>I rarely unit test views.</p></li>
<li><p>I write less view code, and cover it via functional tests.</p></li>
</ul>
</div>
</div><div class="slide" data-transition="none"><div class="content" ref="views/30_system/1">
<h1>Integration testing views</h1></div>
</div><div class="slide" data-transition="none"><div class="content antipattern" ref="views/30_system/2">
<pre class="sh_python"><code>url = "/case/edit/{0}".format(case.pk)
step = case.steps.get()
response = self.client.post(url, {
    "product": case.product.id,
    "name": case.name,
    "description": case.description,
    "steps-TOTAL_FORMS": 2,
    "steps-INITIAL_FORMS": 1,
    "steps-MAX_NUM_FORMS": 3,
    "steps-0-step": step.step,
    "steps-0-expected": step.expected,
    "steps-1-step": "Click link.",
    "steps-1-expected": "Account active.",
    "status": case.status,
})</code></pre></div>
</div><div class="slide" data-transition="none"><div class="content" ref="views/30_system/3">
<h1>or...</h1></div>
</div><div class="slide" data-transition="none"><div class="content" ref="views/30_system/4">
<h1>WebTest!</h1>

<pre class="sh_python"><code>url = "/case/edit/{0}".format(case.pk)
form = self.app.get(url).forms["case-form"]
form["steps-1-step"] = "Click link."
form["steps-1-expected"] = "Account active."

response = form.submit()</code></pre>

<p class="notes">WebTest parses the form HTML and can submit it like a browser would.</p></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="views/30_system/5">
<h1>The markup matters.</h1>

<ul>
<li>If it can break, it should be tested.</li>
<li>It can especially break forms.</li>
<li>The output of your view is an HTTP response; the template + context is an
implementation detail.</li>
</ul>


<p class="notes">Have to know which markup matters, of course.</p></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="views/30_system/6">
<h2>WebTest &gt; django.test.Client</h2>

<ul>
<li>System tests are easier and faster to write.</li>
<li>Tests give you more confidence that the view works.</li>
<li>(django-webtest provides integration.)</li>
</ul>


<p class="notes">Django test client is in the "sour spot" - not a unit test, not a full system test. Could gain these features.</p></div>
</div><div class="slide" data-transition="none"><div class="content" ref="views/30_system/7">
<pre class="sh_python"><code>self.assertEqual(
   response.json, ["one", "two", "three"])

self.assertEqual(
    resp.html.find("a", title="Login").href,
    "/login/"
    )</code></pre>

<p class="notes">Automatically parses JSON or HTML responses (BeautifulSoup or lxml).</p></div>
</div><div class="slide" data-transition="none"><div class="content" ref="views/40_selenium/1">
<h1>In-browser testing</h1>

<p class="notes">More and more functionality depends on both JS and server. Needs to be tested too.</p></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="views/40_selenium/2">
<h2>Is easier than you think.</h2>

<ul>
<li><p>Especially in Django 1.4.</p></li>
<li><p><code>pip install selenium</code></p></li>
<li><p><code>LiveServerTestCase</code></p></li>
</ul>
</div>
</div><div class="slide" data-transition="none"><div class="content smaller" ref="views/40_selenium/3">
<pre class="sh_python"><code>from django.test import LiveServerTestCase
from selenium.webdriver.firefox.webdriver import WebDriver

class MySeleniumTests(LiveServerTestCase):

    @classmethod
    def setUpClass(cls):
        cls.selenium = WebDriver()
        super(MySeleniumTests, cls).setUpClass()

    @classmethod
    def tearDownClass(cls):
        super(MySeleniumTests, cls).tearDownClass()
        cls.selenium.quit()

    def test_login(self):
        self.selenium.get(
            "%s%s" % (self.live_server_url, "/login/"))
        username_input = self.selenium.find_element_by_name(
            "username")
        username_input.send_keys("myuser")
        password_input = self.selenium.find_element_by_name(
            "password")
        password_input.send_keys("secret")
        self.selenium.find_element_by_xpath(
            '//input[@value="Log in"]').click()</code></pre>

<p class="notes">LiveServerTestCase runs the development server in a separate thread.</p></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="views/50_guideline/1">
<h1>What type of test to write?</h1>

<ul>
<li><p>Write system tests for your views.</p></li>
<li><p>Write Selenium tests for Ajax, other JS/server interactions.</p></li>
<li><p>Write unit tests for everything else (not strict).</p></li>
<li><p>Test each case (code branch) where it occurs.</p></li>
<li><p>One assert/action per test case method.</p></li>
</ul>


<p class="notes">Very rough guidelines; what works for me. Not strict; e.g. tests for a ModelForm don't mock the model.</p></div>
</div><div class="slide" data-transition="none"><div class="content" ref="views/50_guideline/2">
<pre class="sh_python"><code>def add_quote(request):
    if request.method == "POST":
        form = QuoteForm(request.POST)
        if form.is_valid():
            return redirect("quote_list")
    else:
        form = QuoteForm()

    return TemplateResponse(
        request,
        "add_quote.html",
        {"form": form},
        )</code></pre>

<p class="notes">This view should have 3 tests. Model/form special cases should be unit tested. And views shouldn't get much more complex.</p></div>
</div><div class="slide" data-transition="none"><div class="content" ref="doctests/10_intro/1">
<h1>Testing documentation</h1></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="doctests/10_intro/2">
<h2>"We have always been at war with doctests"</h2>

<ul>
<li><p>Not entirely fair.</p></li>
<li><p>Doctests are great.</p></li>
<li><p>For testing documentation examples.</p></li>
</ul>
</div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="doctests/10_intro/3">
<h1>You have Sphinx docs.</h1>

<ul>
<li>Right?</li>
</ul>
</div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="doctests/10_intro/4">
<h2>You have API code examples.</h2>

<h2>In your Sphinx docs.</h2></div>
</div><div class="slide" data-transition="none"><div class="content" ref="doctests/10_intro/5">
<h1>in any test file</h1>

<pre class="sh_python"><code>def load_tests(loader, tests, ignore):
    path = os.path.join(
        settings.BASE_PATH,
        "docs",
        "examples.rst",
        )

    tests.addTests(
        doctest.DocFileSuite(path))

    return tests</code></pre></div>
</div><div class="slide" data-transition="none"><div class="content incremental" ref="doctests/10_intro/6">
<h1>Your examples are tested!</h1>

<ul>
<li><p>Please don't abuse this.</p></li>
<li><p>Keep them documentation first.</p></li>
</ul>
</div>
</div><div class="slide" data-transition="none"><div class="content" ref="settings/10_intro">
<h1>You have a setting.</h1>

<ul>
<li><code>ALLOW_COMMENTS</code>, for example.</li>
</ul>
</div>
</div><div class="slide" data-transition="none"><div class="content antipattern" ref="settings/20_bad">
<pre class="sh_python"><code>def test_comments_allowed(self):
  old_allow = settings.ALLOW_COMMENTS
  settings.ALLOW_COMMENTS = True
  try:
    # ...
  finally:
    settings.ALLOW_COMMENTS = old_allow</code></pre></div>
</div><div class="slide" data-transition="none"><div class="content" ref="settings/30_good">
<pre class="sh_python"><code>@override_settings(ALLOW_COMMENTS=True)
def test_comments_allowed(self):
  # ...</code></pre></div>
</div><div class="slide" data-transition="none"><div class="content smaller intro" ref="outro/01_out">
<h1>Questions?</h1>

<p><img src="./file/outro/logo.png" alt="OddBird"/></p>

<h2>Carl J Meyer</h2>

<h2>@carljm</h2>

<h2>carl@oddbird.net</h2>

<h2>github.com/carljm/django-testing-slides</h2>

<h2>github.com/carljm/django-testing-slides/code</h2></div>
</div></div>

</body>
</html>