Skip to content

CONCERNS

Mike Crowe edited this page Jul 6, 2026 · 5 revisions

Technical Concerns & Debt

Generated 2026-07-05. Prioritized by severity. Cites file:line where concrete.


HIGH

H-1 — launcher.py is a 1,991-line god module with 77 functions

src/harnessed/launcher.py contains the full launch lifecycle, image build orchestration, mount construction for every credential type (SSH, GPG, 1Password, YubiKey), firewall, persistence, secrets resolution, service management, CLI command definitions, and the Typer app. It has no unit tests of its own (see M-1 below — only integration tests via tests/test_launcher_install.py and tests/test_launcher_init.py). Adding or changing any credential-forwarding path requires reasoning across all 77 functions at once.

Risk: any refactor, credential path change, or new harness type touches a monolith with no isolation seam. Bugs in _ssh_agent_args, _gnupg_mounts, _credential_forward_args are hard to unit-test because they depend on host filesystem state.

Direction: Split into focused sub-modules (e.g. mounts.py, images.py, services.py) with thin public interfaces. The existing scan.py, persist.py, paths.py pattern already shows the right shape — launcher.py hasn't been split yet.


H-2 — macOS 1Password socket relay is PENDING VERIFICATION (unverified on real hardware)

src/harnessed/launcher.py _macos_op_socket_mount_source is explicitly marked PENDING VERIFICATION (macOS-gated — I could not test this from Linux). The function attempts to reverse-forward the 1Password agent socket into the podman machine VM. If the reverse-forward fails, the caller falls back gracefully with a user-facing warning, but the fallback path (raw host socket) is likely also broken on macOS (host unix sockets don't traverse the VM boundary). The user-facing warning emitted when the reverse-forward fails confirms users will see a yellow warning and the flow is untested.

Additionally, the not wired yet Docker-Desktop-relay comment in _macos_op_socket_mount_source explicitly notes Docker Desktop is not wired:

return None  # Docker Desktop uses a different relay; not wired yet (see the todo).

Risk: Any macOS user with 1Password SSH agent forwarding silently gets a non-functional SSH agent mount. Commit signing and git push over SSH will fail without feedback beyond the warning.

Direction: Verify on macOS hardware (tracked in bd (macOS SSH-agent forwarding = bd main-5ki)), or add a hard [yellow]warning[/yellow] that macOS SSH agent forwarding is not yet supported instead of a partial attempt that fails silently.


H-3 — In-container GPG signing is incomplete (scoped follow-up not tracked)

The _gnupg_mounts docstring in src/harnessed/launcher.py states that full openpgp GPG signing in-container (for non-YubiKey keys, even via agent) is a "scoped follow-up." The current _gnupg_mounts correctly avoids mounting private-keys-v1.d/ (prevents secret exfil), but the consequence is that software GPG keys cannot sign commits in-container. The follow-up is tracked in bd (macOS SSH-agent forwarding = bd main-5ki).

Risk: Users expecting GPG commit signing via software keys (not YubiKey) will get a silent failure (git commit succeeds, gpg signing fails or falls back unsigned depending on git config).


MEDIUM

M-1 — Missing unit tests for four source modules

The following src/harnessed/ modules have no corresponding tests/test_<module>.py:

Module Size Notes
assemble.py 134 lines Tested indirectly via integration; no direct unit tests
capability.py 593 lines Tested by tests/test_recipes_integration.py only (integration)
cli.py (argparse CLI) Scan CLI surface entirely untested
report.py 71 lines No test file

capability.py is the most significant gap — it drives live capability introspection and contains non-trivial parsing logic (_sse_to_objects, _names_from_llm_json, _collect_server_names). These are pure functions that could be unit-tested without podman.


M-2 — Scan subsystem is an open decision (D2) with real dead-surface risk

src/harnessed/scan.py (417 lines) exposes run_image_scan_online, run_image_scan, run_snyk_container_scan, and run_source_scan. These are called from src/harnessed/cli.py (the harnessed-tools binary) but NOT from src/harnessed/launcher.py (the main harnessed CLI entry point). The ROADMAP.md documents open decision D2: "keep run_image_scan_online only, or remove the gating scanner + harnessed-tools + test_scan.py entirely?"

As of 2026-07-05, scan.py is live code that ships with the package, but the primary user-facing workflow (harnessed launch/build) calls scan only indirectly via the derived image's baked scan layer. The harnessed-tools scan* subcommands are the only direct consumers.

Risk: the scan subsystem is maintained but the decision about its long-term role is deferred. Any refactor that touches _TIMEOUT, severity thresholds, or the osv-scanner JSON schema must also update tests/test_scan.py.

Action: resolve D2, track it as a bd issue (main-fut reference in roadmap).


M-3 — No linter or static type checker configured

pyproject.toml has no [tool.ruff], [tool.mypy], or any formatter configuration. The project uses type hints throughout (src/harnessed/schema.py, launcher.py, etc.), but nothing enforces them. There is one # noqa: E731 suppression on the norm = lambda in src/harnessed/launcher.py (lambda assignment) and one in test imports, implying ruff/flake8 was run manually at some point but not wired into CI.

Risk: type regressions and style inconsistencies accumulate silently between contributors.


M-4 — pip-audit==2.10.1 pinned as an exact runtime dependency

pyproject.toml pins pip-audit==2.10.1 as a hard runtime dependency (not a dev-only tool). Exact version pins on security scanners can mask available CVE database updates — the scanner ships its own advisory DB snapshot. If a critical vulnerability is added post-2.10.1, installed users won't get it until the pin is bumped explicitly.

Risk: harnessed-tools scan may report clean against a stale advisory database.


M-6 — _macos_op_socket_mount_source backgrounded SSH process leaks on repeated calls

src/harnessed/launcher.py:858-863 runs podman machine ssh -f -N -T -R ... in the background (-f flag). Each harnessed launch invocation on macOS will attempt this, spawning a new backgrounded ssh process. While StreamLocalBindUnlink=yes clears the stale socket on retry, the old ssh process is orphaned (not killed). On repeated launches in the same session, processes accumulate.

Risk: Low on a typical dev workstation but worth addressing before the macOS path is verified.


LOW

L-1 — Silent pass in exception handlers — four sites

These are intentional cleanup paths, but each silently swallows errors that could indicate a real problem:

  • src/harnessed/launcher.py:450 — cleanup of a temp env file on write failure: swallows OSError from os.unlink(tmp). The outer raise still fires, so the write failure propagates correctly — only the cleanup failure is silenced. Acceptable, but the outer except Exception: is broader than needed (could be OSError | PermissionError).
  • src/harnessed/launcher.py:830gpgconf --list-dirs failure falls back to a platform-default path. Reasonable defensive coding.
  • src/harnessed/launcher.py:1497secrets_env_file.unlink() failure in finally block. The outer finally is the right place; the silence here means a leaked temp file on cleanup failure would not be reported.
  • src/harnessed/capability.py:265 — teardown of a container on _exec failure: swallows SubprocessError | OSError. This runs podman rm; if it fails, the container leaks but the capability test result still propagates.

None of these are bugs today, but they will be invisible in observability tooling.


L-2 — floating-recipe test fixture Dockerfile contains a real --branch main clause

catalog/recipes/floating-recipe/Dockerfile:6 intentionally contains:

RUN git clone --branch main https://example.com/fake-repo.git /opt/fake

This is documented as a test fixture for validate_pin(). It does not trigger a real build (the URL is unreachable and the assembler rejects it before any build). However, a reader unfamiliar with the fixture purpose may be confused. The file has a header comment explaining it, but the comment is not in the Dockerfile's standard location (it's after a blank line, not at line 1).


L-3 — typer>=0.12,<1.0 version range spans a pre-1.0 API surface

pyproject.toml allows any typer 0.x from 0.12 onward. Typer's pre-1.0 API has had breaking changes between minor versions (e.g. typer.Typer constructor parameters, app.command decorator behavior). No upper minor bound is set, so a 0.13 or 0.14 release could break CLI entry points.


L-4 — Secrets temp file on disk between resolve and container start

src/harnessed/launcher.py:407-452_resolve_launch_secrets() writes a mode-0600 temp file (tempfile.mkstemp) containing resolved secret values. The file is unlinked in a finally block at launcher.py:1494-1498. On Linux, any root or same-uid process can read /proc/<pid>/fd/<n> between write and unlink. The window is small (seconds), but for secrets like API keys passed to containers, this is a potential local information leak on multi-tenant machines. The code is otherwise correct (mode-0600, finally-unlinked, prefix-named to be identifiable).

This matches the documented threat model (T-05-06) and is noted in the code. Flag here for completeness since the risk is non-zero on shared hosts.


L-5 — ruamel.yaml>=0.18,<0.19 major version locked to a pre-1.0 release

pyproject.toml locks ruamel.yaml to >=0.18,<0.19. This is a stable, widely-used library but is still pre-1.0. If 0.19 introduces breaking changes to the YAML(typ="safe") API (used in schema.py:27), upgrading requires re-pinning and retesting the entire schema parser. Low urgency but worth noting as the library nears 1.0.


Reference: Open Decisions (from ROADMAP.md)

These are documented open decisions, not newly discovered concerns:

  • D2 — scan subsystem future: keep run_image_scan_online or remove harnessed-tools scan* entirely. Tracked as bd main-fut.
  • D-compose — multi-container compose: service support (catalog/stacks/ pattern) is explicitly bottom priority (bd main-9cf).
  • macOS agent forwarding — tracked in bd (macOS SSH-agent forwarding = bd main-5ki) for H-2 and H-3 above.

Clone this wiki locally