django_e2e_benchmarks

This repo contains a simple Django application against which to benchmark end-to-end (e2e) tests developed using Playwright and pydoll. It also contains utils to measure execution time of each e2e test suite.

The Django project was generated using the djereo project template.

Prerequisites

To use django_e2e_benchmarks the following must be available locally:

• Python 3.12 or above
• Postgres
• uv
• just
• nox

Quickstart

Run the Django app locally

A justfile defines common development tasks. Run just to show all available recipes.

# 1/5 create a postgres user and database for your project - change 'password' below
psql -U $(whoami) -d postgres -v APP_USER_PASSWORD='password' -f _db/set_up.sql

# 2/5 update the {password} in the DATABASE_URL connection strings in both .env & .env.test
# The .env file is created for you as a post-creation task.

# 3/5 apply migrations to the database
just manage migrate

# 4/5 create three users: admin, staff and regular (non-privileged)
just manage seed_database

# 5/5 install dependencies in a virtual environment and run the Django development server
just runserver

IPython is available as the default shell. Start an interactive session with:

just shell

Run the benchmarking script

usage: benchmark.py [-h] [--target {pydoll_e2e,playwright_e2e}] [-n N]
                    [--max-attempts MAX_ATTEMPTS]

Run end-to-end test suites multiple times and report timing statistics.

By default runs pytest through uv, collects wall-clock durations,
and summarizes the minimum, maximum, and average times.

options:
  -h, --help            show this help message and exit
  --target {pydoll_e2e,playwright_e2e}
                        Which test directory to run (default: playwright_e2e).
  -n N                  Number of successful runs to collect (default: 10).
  --max-attempts MAX_ATTEMPTS
                        Maximum attempts before giving up (default: 2 × n).

Examples:
  python benchmark.py --target pydoll_e2e -n 5
  python benchmark.py --target playwright_e2e -n 3 --max-attempts 20

Develop

Development prerequisites

In addition to the Prerequisites above, you will need the following to develop django_e2e_benchmarks:

• pre-commit

Style

Code style is enforced by pre-commit hooks.

# before you start developing, install pre-commit hooks
pre-commit install

See Git principles & Style in the developer README for more.

Dependency management

Dependencies are defined in the pyproject.toml file, with exact versions captured by a uv.lock lockfile. uv is used to manage dependencies:

# add a dependency to the project
uv add some-package

Dependabot is configured to run weekly and update Python packages (minor & patch) and GitHub Actions. See .github/dependabot.yaml.

Logging

django_e2e_benchmarks uses structlog when DEBUG is False. This configuration is intended for deployed environments such as production. When DEBUG is True, as it usually is when developing locally, rich is used to format logs. The logging configuration is generated by logging.LoggingConfigFactory.

You may issue logs as follows:

import structlog

logger = structlog.get_logger()
logger.info("django_e2e_benchmarks is running", key="value")

Models

The core app (django_e2e_benchmarks) defines an UuidModel abstract model. Models that subclass it will have primary keys that are UUIDs instead of the default BigAutoField. Models in the 'users' app extend UuidModel and it can be used by any new models.

Utility abstract models CreatedAtModel, UpdatedAtModel and DeletedAtModel (for soft-deletion) are available to store timestamps.

htmx frontend

A vendorised copy of htmx is included in the _base.html template via django-htmx's {% htmx_script %} tag. The django-htmx package is used to smoothly integrate htmx into projects, abstracting away header checks and allowing views to change their behaviour and responses based on the request.htmx attribute.

See the django-htmx documentation for more.

Test

Django's test runner (using the standard library's unittest) is used for the suite of unit tests. Nox is used to automate testing across different Python versions. Run all tests and report on code coverage:

nox [-- <expression>]

# eg. to only run tests inside the `test_migrations` module
nox -- tests.test_migrations

By default nox will only run the session for the latest supported Python release.

To run all nox sessions (ie. for all supported Python releases):

nox -k tests

Profiling tests

Invoke just profile_tests to output a speedscope-compatible profile file to understand bottlenecks in your tests. N.B. this requires py-spy to be available globally on your system.

Release

Release Please is used to automate:

• Updating the changelog.
• Calculating the new SemVer tag based on conventional commit types.
• Creating a new GitHub release.

Release Please is configured as a GitHub action (release-please.yaml). It keeps a release pull request open that is refreshed as changes are merged into main. To cut a release, simply merge the release pull request.

GitHub Personal Access Token

In order for Release Please to automate the above process, a GitHub Actions secret called RELEASE_PLEASE_TOKEN must exist in GitHub (under github.com/<user>/<repo>/settings/secrets/actions). The contents of this secret must be a Personal Access Token (PAT) with the following permissions:

contents: write
pull-requests: write

For more information, consult the release-please-action project.

Deploy django_e2e_benchmarks

The _deploy/ directory includes a Dockerfile that uses the local development toolchain (the uv project manager) to containerise the application.

Deployment prerequisites

To deploy django_e2e_benchmarks the following must be available locally:

• Docker

Containerise the app

The Dockerfile runs pre-production steps, such as enabling bytecode compilation to trade a longer installation time for faster start-up times, or running Django's collectstatic command. The container's entrypoint runs gunicorn as the WSGI web server to make the Django application available.

Build a deployment-ready image with:

docker build \
  --file _deploy/deploy.Dockerfile \
  --platform linux/amd64 \
  --build-arg CACHEBUST=$(date +%s) \
  --tag django_e2e_benchmarks:amd64 \
  --load .

N.B. The CACHEBUST argument ensures the layer that runs collectstatic is not cached between builds.

The recommended use of the resulting container is to place it behind nginx as the reverse proxy, with gunicorn as the WSGI web server between the two.

---

Developer docs

The developer README (docs/README-dev.md) covers how to work on django_e2e_benchmarks in more detail. It covers:

• Development prerequisites
• The djereo project template
• Git principles
• Justfile recipes
• Running tests with Nox
• Use IPython as your shell
• Style & pre-commit hooks
• Managing settings
• Use project metadata in the running application
• Security
• Developer tools
• Upgrading
  • Upgrade checklist
• GitHub Actions
  • Custom GitHub Actions
  • GitHub Actions workflows
    • PR (pull request)
    • CI (continuous integration)
    • Release Please
    • On tag
    • Dependabot

---

© Alberto Morón Hernández