TESTING

Testing

Test framework, structure, and patterns for tests/. Follow these exactly when writing new tests.

---

Toolchain

• Framework: pytest (>=8) via pyproject.toml [project.optional-dependencies] dev
• Run command: mise exec -- uv run pytest (never bare pytest — uv is not on PATH directly)
• Coverage: pytest-cov is installed; run with mise exec -- uv run pytest --cov=harnessed
• Test root: testpaths = ["tests"] in pyproject.toml
• No conftest.py — all fixtures and helpers are defined per-file

---

File layout

tests/
  test_emit.py              # emit.py artifact writers
  test_schema.py            # schema validators and loaders
  test_paths.py             # paths.py resolver
  test_persist_mounts.py    # persist mount emission (fast) + podman-gated round-trips
  test_persist_allowlist.py # global allowlist + ownership guard
  test_persist_gc.py        # persist GC lifecycle
  test_launcher_install.py  # install shim generation
  test_launcher_init.py     # init: marker check and one-shot exec
  test_recipes_integration.py  # assembly oracle + live container checks
  test_catalog_json_schemas.py # JSON Schema validation of catalog manifests
  ...

Each file opens with a module docstring naming what it covers and which test layer it operates in:

# tests/test_persist_mounts.py:1-9
"""T6 — persist mount emission (fast) + round-trip / isolation (podman-gated).

Two layers:

1. FAST (no podman): ...
2. PODMAN-gated (HARNESSED_PODMAN=1): ...
"""

---

Test class and method naming

Tests are grouped by the function or behaviour under test using classes. The class name is Test<Subject> and each method name is test_<scenario>:

# tests/test_emit.py:44-67
class TestWriteMcpJson:
    def test_creates_mcp_json_at_profile_root(self, tmp_path): ...
    def test_content_has_single_hatago_entry(self, tmp_path): ...
    def test_entry_has_http_type(self, tmp_path): ...
    def test_output_is_at_root_not_claude_subdir(self, tmp_path): ...

Method names are full sentences that describe the expected outcome: test_no_servers_writes_empty_settings, test_baked_none_returns_required_floor, test_different_projects_different_names. Avoid generic names like test_1 or test_basic.

---

Built-in fixtures

Use tmp_path (isolated Path per test) and monkeypatch (reversible patching) from pytest. Both are passed as parameters:

def test_uses_xdg_data_home(self, monkeypatch, tmp_path):
    monkeypatch.setenv("XDG_DATA_HOME", str(tmp_path))
    assert paths.profile_dir("my-stack") == tmp_path / "harnessed" / "profiles" / "my-stack"

Always redirect XDG_DATA_HOME and XDG_CONFIG_HOME via monkeypatch.setenv in tests that touch the filesystem. Never write to the real home or XDG dirs:

# tests/test_persist_allowlist.py:22-27
@pytest.fixture
def home(monkeypatch, tmp_path):
    h = tmp_path / "home"
    h.mkdir()
    monkeypatch.setenv("HOME", str(h))
    monkeypatch.setenv("XDG_CONFIG_HOME", str(h / ".config"))
    return h

---

Module-level helpers (not pytest fixtures)

Private helper functions (prefixed with _) build test objects inline. They are free functions, not fixtures, because they take arguments:

# tests/test_schema.py:26-31
def _make_recipe(name: str = "test", servers: list | None = None) -> Recipe:
    return Recipe(
        name=name,
        servers=servers or [],
        root=Path("/tmp/fake-recipe"),
    )

# tests/test_emit.py:23-24
def _hook_recipe(name: str, hooks: dict) -> Recipe:
    return Recipe(name=name, hooks=hooks)

# tests/test_persist_mounts.py:27-33
def _entry(**kw) -> PersistEntry:
    return PersistEntry(
        scope=kw.get("scope", "workspace"),
        location=kw.get("location", "host"),
        name=kw.get("name", None),
        path=kw.get("path", None),
        vcs=kw.get("vcs", None),
    )

Use these helpers to keep test bodies short and focused on the assertion, not on object construction.

---

Monkeypatching functions and attributes

Use monkeypatch.setattr to stub out functions without unittest.mock.patch. The target is always the object as imported into the module under test:

# tests/test_launcher_install.py:24-25
monkeypatch.setattr(paths, "find_in_catalog", lambda kind, name: stack_dir)
monkeypatch.setattr(launcher.shutil, "which", lambda _: "/opt/bin/harnessed")

# tests/test_launcher_init.py:35-43
monkeypatch.setattr(launcher, "load_stack_with_recipes", lambda root, stack: (stk, recipes))
monkeypatch.setattr(launcher, "load_stack", lambda d: stk)
monkeypatch.setattr(launcher, "_derived_image", lambda s: "harnessed-test_stack:latest")
monkeypatch.setattr(launcher, "_persist_mounts", lambda stack, project_path: [])
monkeypatch.setattr(paths, "git_common_dir", lambda p: None)

Path.home is often stubbed to redirect filesystem operations:

# tests/test_launcher_install.py:27-30
def _home_in(monkeypatch, tmp_path):
    home = tmp_path / "home"
    home.mkdir()
    monkeypatch.setattr(Path, "home", lambda: home)
    return home

---

Exception assertions

Use pytest.raises with a match= pattern to assert both the exception type and that the message contains the key term:

# tests/test_schema.py:39-42
def test_npm_command_raises(self):
    r = _make_recipe(servers=[McpServer(name="s", command="npm", args=["install"])])
    with pytest.raises(RecipeLintError, match="pnpm"):
        validate_no_raw_npm(r)

# tests/test_schema.py:93-95
def test_missing_manifest_raises(self, tmp_path):
    with pytest.raises(SchemaError, match="stack manifest not found"):
        load_stack(tmp_path / "nonexistent")

When testing that a function does NOT raise, call it and then make a positive assertion — do not rely on the absence of an exception as the only signal:

# tests/test_schema.py:35-36
def test_clean_recipe_passes(self):
    r = _make_recipe()
    validate_no_raw_npm(r)  # must not raise

---

Warning callback testing

Functions that accept a warn callback are tested by passing warns.append:

# tests/test_emit.py:162-166
def test_none_text_returns_none_silently(self):
    warns: list[str] = []
    assert read_baked_settings(None, warn=warns.append) is None
    assert warns == []  # absent file / cp failure is not a warning

def test_malformed_json_returns_none_and_warns(self):
    warns: list[str] = []
    assert read_baked_settings("{not json", warn=warns.append) is None
    assert len(warns) == 1  # a recipe wrote broken JSON — warn, do not crash

---

Filesystem fixture setup in tests

When a test needs an on-disk fixture (recipe dir, stack dir, etc.), create it inline using tmp_path:

# tests/test_schema.py:97-103
def test_minimal_stack_loads(self, tmp_path):
    d = tmp_path / "my-stack"
    d.mkdir()
    (d / "stack.yaml").write_text("name: my-stack\nharness: claude\nrecipes: []\n")
    stk = load_stack(d)
    assert stk.name == "my-stack"
    assert stk.harness == "claude"

Avoid writing to real catalog dirs. Pass tmp_path or a subdir as the root parameter to loaders that accept it.

---

Parametrize for value coverage

Use pytest.mark.parametrize to cover multiple valid/invalid values without repetition:

# tests/test_persist_allowlist.py:44-48
@pytest.mark.parametrize("sub", [".ssh", ".aws", ".gnupg"])
def test_sensitive_dirs_denied_even_if_allowlisted(self, home, sub):
    target = home / sub
    target.mkdir()
    _write_allowlist(str(target))
    with pytest.raises(PersistDeniedError):
        persist.resolve_global_persist(str(target))

---

Regression tests

Regression tests carry an explicit comment identifying what was broken:

# tests/test_emit.py:193-197
def test_baked_hooks_preserved_and_grant_added(self):
    # REGRESSION proof: the bug was that baked hooks were silently dropped at runtime.
    baked = {"hooks": {"PreToolUse": [{"matcher": "Bash"}]}}
    merged = merge_settings(baked, _REQUIRED)
    assert merged["hooks"] == {"PreToolUse": [{"matcher": "Bash"}]}

---

Mutation safety tests

When a function must not mutate its inputs (e.g. functions using deepcopy), assert the original is unchanged:

# tests/test_emit.py:231-234
def test_does_not_mutate_input(self):
    baked = {"permissions": {"allow": ["mcp__other"]}}
    merge_settings(baked, _REQUIRED)
    assert baked == {"permissions": {"allow": ["mcp__other"]}}  # deepcopy — caller's dict safe

---

Podman-gated tests

Tests that require a running podman instance are skipped by default and opt-in via the HARNESSED_PODMAN=1 environment variable:

# tests/test_persist_mounts.py — pattern
import pytest
import os

@pytest.mark.skipif(not os.environ.get("HARNESSED_PODMAN"), reason="HARNESSED_PODMAN not set")
def test_workspace_survives_fresh_relaunch(...): ...

# tests/test_recipes_integration.py:34
ROOT = Path(__file__).resolve().parents[1]  # repo root (HARNESSED_DIR for catalog resolution)

Fast tests (no podman) run in default CI. Podman tests run in a separate environment with HARNESSED_PODMAN=1.

---

What to test

Function type	What to assert
Parser / loader	Required fields accepted, missing fields raise SchemaError with the field name in the message, invalid values raise with the bad value in the message, valid defaults produce the right output
Emitter	Output file path is correct, JSON content matches expected structure, idempotent (reset then re-emit gives same result)
Path resolver	XDG fallbacks, env overrides, trailing-slash normalization, different inputs produce different outputs
Subprocess probe	returncode == 0 is the only signal, exceptions are caught and return a safe fallback
Security validator	Traversal attempts (../), absolute paths, and sensitive paths are rejected; valid inputs pass

Do not write tests that only verify no exception is raised. Every test body must have at least one assert.