Skip to content

Development

Jump to bottom
Jakub Adámek edited this page Oct 6, 2026 · 5 revisions

🛠️ Development

Ruff, mypy, pytest, one alembic head — the gate that keeps main green, and CI runs the exact same one.

Setup

uv sync                # installs into .venv, including dev dependencies

No bare-metal run path (see Installation) — but tests run against in-memory SQLite, not real Postgres, so uv sync alone is enough for lint/type-check/tests.

Commands

uv run pytest                        # full suite
uv run pytest tests/test_x.py -k y   # one test
uv run ruff check .                   # lint
uv run ruff format .                  # format
uv run mypy app alembic tests        # type check (strict for app/, alembic/)

CI runs this same gate on every push/PR (.github/workflows/ci.yml) — ruff, mypy, the full test suite, and the single-alembic-head check, plus a separate pip-audit pass over exactly what uv.lock would install. Dependabot watches for newer fixed versions of Python, Docker, and GitHub Actions dependencies on top of that. Nothing here needs a real Postgres/Redis — see tests/conftest.py.

CodeQL (codeql.yml) scans Python, the app's JavaScript and the workflows; .github/codeql/codeql-config.yml leaves out vendored bundles, generated migrations and the Jinja templates (misread as JS), plus py/unused-import, which ruff already enforces. Models never import each other at the top of a module: type-checking-only imports sit in an if TYPE_CHECKING: block at the bottom, and an enum a model needs at class-definition time lives in its own model-free module (app/db/models/access_level.py, notification_channel.py), so CodeQL's "Module-level cyclic import" stays clean.

Fuzzing

fuzz/targets.py holds one target per parser of input the app doesn't control — honeypot command output (app.ssh.*), the OpenCanary event log (canary_log, canary_event), opencanary.conf, pasted SSH keys, the redirect guard, the event-list time filter and tags. Each checks a contract: parsers never raise, an OpenCanary line always becomes an event the database accepts (bounded strings, a real port or none), guards reject only with their documented error, and what the redirect guard lets through is a same-site path. fuzz.yml runs every target with Atheris (coverage-guided, Linux only) for a minute on each PR and 20 minutes weekly; tests/test_fuzz_targets.py runs the same targets over a seed corpus in the normal suite on every OS. Locally on Linux: uv sync --group fuzz, then uv run python -m fuzz.run <target> -max_total_time=60. A new parser of untrusted input gets a target (and a matrix entry — the test checks the two match); an input a crash was fixed for goes into the test's _SEEDS.

Browser smoke test

e2e/ opens the app in Chromium (Playwright) and fails on anything the browser console calls an error — a CSP violation, an uncaught script error, a static file that does not load — which the Python suite cannot see. It also tries a few interactions that need the page's scripts (the / search shortcut, a chart tooltip, drag-to-zoom).

  • Runs in CI as the browser job; it is not part of a plain pytest.
  • Locally: uv sync --group e2e, uv run playwright install chromium, uv run pytest e2e.
  • e2e/serve.py is the app on in-memory SQLite with a small fixed data set (no Postgres/Redis/Celery; tasks a page waits for return a canned result). python -m e2e.serve also works on its own for looking at a change by hand; it prints a SESSION_TOKEN for the session cookie.
  • A new page: add its path to PAGES in e2e/test_pages.py.

Files shared with debcontrol

The two apps are meant to look and work the same. The files listed in shared-ui.json (chart code, script and template, the range picker, the acknowledgement panel, tab navigation, search route and API, the time-window helper, a few small scripts) are word-for-word identical to debcontrol's, each with a SHA-256.

  • tests/test_shared_ui.py fails when a listed file changes here only; the weekly Shared files workflow compares the two repositories.
  • Change a shared file in debcontrol first, then there: python scripts/sync_shared_ui.py --update and python scripts/sync_shared_ui.py --to <this checkout>; open a PR in both repositories.
  • Nothing app-specific goes into a shared file. Translation keys under monitoring.*, ack.* and search.* exist in both apps for that.

Route modules

/honeypots is split by area, like debcontrol's /machines: honeypots.py only includes honeypots_list, honeypots_detail, honeypots_updates, honeypots_monitoring, honeypots_activity, honeypots_logs and honeypots_config (each with its own router from honeypots_common.honeypots_router()), in an order that puts fixed paths (/new, /bulk/...) before /{id}/....

Adding a migration

uv run alembic revision --autogenerate -m "Add honeypot.notes"

Always read the generated file — autogenerate misses things (a renamed column looks like a drop + an add; a server-side default on an existing table needs a data migration if the column isn't nullable). Then:

uv run alembic upgrade head
uv run alembic heads     # must show exactly one head before committing

Every new/changed model must also be imported in app/db/models/__init__.py and alembic/env.py — a model not imported there is invisible to --autogenerate.

Adding a company-scoped route

  1. Depend on app.auth.dependencies.get_current_user (read) or require_write (mutating).
  2. Resolve the company_id in question and call app.auth.scope.ensure_company_access(user, company_id, write=...) — or, for a list view, filter by app.auth.scope.visible_company_ids(user) (a set of ids; None for a superadmin means "no filter").
  3. If it's a mutation, wrap it with CSRF (Depends(verify_csrf) on the route, or the router-level dependencies=[...] if every route on it mutates) and call app.audit.log_event(...) after the commit.
  4. Add the equivalent i18n keys to every app/i18n/locales/*.json file if the page has user-facing strings going through t().

Adding a new i18n string

Wrap it in {{ t(request, "area.key") }} in the template, then add "area.key" to app/i18n/locales/en.json and cs.json (a missing key in one locale falls back to English, but a key added to only one file is an oversight, not a feature — see app/i18n/__init__.py's docstring).

A string with a count takes count= — t(request, "dashboard.companies_count", count=n) — and may have key.one and (in Czech) key.few variants next to the plain key, which is the form for 0 and for five or more: "1 firma", "3 firmy", "5 firem". Never write "honeypot(s)" or "událost(í)".

Czech texts use formal address (vykání), Czech names for every page and tab (Přehled, Nastavení, Inicializace, …), and "událost" for an OpenCanary event — not "event" or "alert"; "upozornění" is a notification. tests/test_czech_texts.py trips on the common slips.

Running it locally against real infrastructure

docker compose up -d --build

is the only supported way — see Installation. No uvicorn app.main:app --reload path yet (the app needs real Postgres/ Redis, and there's no hot-reload compose override); add one (docker-compose.override.yml mounting ./app, running uvicorn --reload) if that workflow turns out to matter.

Clone this wiki locally