Getting Started Testing: pytest edition

Brendon Smith (br3ndonland)

Description

This repository contains material from the Boston Python User Group's meetup on February 26, 2020, "Getting Started Testing: pytest edition," led by Ned Batchelder. The slides and code from Ned Batchelder are available here.

Meetup description:

Presentation night sponsored by Kyruus, hosted by PTC.

Ned Batchelder, Getting Started Testing

Do you want to learn how to write automated tests in Python with pytest? We'll start from the very beginning! See how pytest works, and how to write tests. Once the basics are covered, we'll get into fixtures, parameterization, and coverage measurement. Then we'll do a few more advanced topics: including test doubles (mocks and fakes).

It's a lot to cover, but we'll take our time and work through it. You'll get everything you need to start writing your own tests.

The talk is available now if you want a preview: https://bit.ly/pytest3

Doors open at 5:30 for mingling, networking, and exploring the PTC tech space. The presentation starts at 6:30.

This repository was generated from my template-python repository. For more info on template repositories, see GitHub's announcement.

Quickstart

❯ cd path/to/repo
# Install virtual environment with poetry: https://python-poetry.org/docs/
❯ poetry install
❯ poetry shell
# Install pre-commit hooks
.venv ❯ pre-commit install
# Try running the tests
.venv ❯ pytest
.venv ❯ coverage run -m pytest tests

Presentation notes

Part 1

First principles

• Writing tests is an upfront time commitment that actually saves time in the long-run.
• Testing is the "flossing of software." It feels like a chore, and everyone feels like they should be doing more. "Writing tests is serious effort that takes real time."
• Test frameworks all seem weird at first. The way you write for test frameworks is much different than the ways you write your actual code. There are new conventions to learn.

Test frameworks

• unittest is wordy and not Pythonic, because it was lifted from the Java jUnit framework.

• Nose is not maintained and should no longer be used. It was called nose because it would "sniff" out your tests automatically, rather than having you specify where they are.

• Pytest uses functions instead of classes. Ned admits that pytest is very powerful, and does many things that even he doesn't understand.

• Project structure (slide 21)

  • Put your tests in the tests/ directory.

  • When I moved the tests to the tests/ directory, pytest started throwing a ModuleNotFoundError. Pytest could find the tests in the tests/ directory, but the tests couldn't find the modules they were importing from the root directory. The solution, as explained on Stack Overflow, is to simply create an empty conftest.py file in the root directory. The conftest.py file is typically used for storing pytest fixtures, as explained below.

  • When referencing Python modules in different directories, use the syntax directory.module. For example:

    # tests/test_port6_pytest.py
    
    import pytest
    
    from portfolio.portfolio2 import Portfolio

• Running tests (slide 22)

  • Set up, act, assert.
  • See test_port1_pytest.py for a good example.
  • Try it: pytest -q tests/test_port1_pytest.py
  • To skip tests that are intentionally broken for the sake of example (in the modules ending in broken.py), either tell pytest to skip them at run time with pytest -k "not broken", or mark the tests as expected to fail and skip them with import pytest and then by adding @pytest.mark.xfail() as a decorator above the applicable test function.
  • unittest provides a similar @unittest.expectedFailure decorator.

• Test isolation (slide 27):

  • Tests shouldn't affect each other.
  • If one test fails, it shouldn't stop the subsequent tests.

Fixtures

• Slide 32
• pytest fixtures use decorators to create reusable test methods.
• In test_port6_pytest.py, we use a fixture to initialize class Portfolio() with some test data.
• The conftest.py file is normally used for sharing fixture functions.

Part 2

Coverage

• Coverage is like a test of your tests. It evaluates how much of your production code is actually being run by your tests.
• It's not always necessary to get to 100% coverage.
• Even if you are at 100%, it doesn't mean your application works perfectly.

Test doubles

• Test doubles stand in for real application data. Useful for simulating application dependencies.
• In this example, we use some pre-set stock prices as test doubles, instead of calling the stock pricing API. We also replace the Requests API call with a FakeRequests method.
• Mocks are more powerful test doubles. The pytest-mock mocker.patch fixture can replace (patch) application data with a mock object.

Testability

• You may sometimes need to refactor code in order to make it more testable.
• In particular, it may be helpful to separate code into smaller units.

Summing up

• Testing is complicated, important, worthy, and rewarding.
• The drawings were by Ned's son Ben, including "sleepy snake," the mascot for coverage.py.

Further information

See CONTRIBUTING.md.