-
Notifications
You must be signed in to change notification settings - Fork 0
CONCERNS
Generated 2026-07-05. Prioritized by severity. Cites file:line where concrete.
src/harnessed/launcher.py contains the full launch lifecycle, image build orchestration, mount
construction for every credential type (SSH, GPG, 1Password, YubiKey), firewall, persistence,
secrets resolution, service management, CLI command definitions, and the Typer app. It has no unit
tests of its own (see M-1 below — only integration tests via tests/test_launcher_install.py and
tests/test_launcher_init.py). Adding or changing any credential-forwarding path requires
reasoning across all 77 functions at once.
Risk: any refactor, credential path change, or new harness type touches a monolith with no
isolation seam. Bugs in _ssh_agent_args, _gnupg_mounts, _credential_forward_args are hard to
unit-test because they depend on host filesystem state.
Direction: Split into focused sub-modules (e.g. mounts.py, images.py, services.py) with
thin public interfaces. The existing scan.py, persist.py, paths.py pattern already shows the
right shape — launcher.py hasn't been split yet.
src/harnessed/launcher.py _macos_op_socket_mount_source is explicitly marked
PENDING VERIFICATION (macOS-gated — I could not test this from Linux). The function attempts to
reverse-forward the 1Password agent socket into the podman machine VM. If the reverse-forward fails,
the caller falls back gracefully with a user-facing warning, but the fallback path (raw host socket)
is likely also broken on macOS (host unix sockets don't traverse the VM boundary). The
user-facing warning emitted when the reverse-forward fails confirms users will see a yellow
warning and the flow is untested.
Additionally, the not wired yet Docker-Desktop-relay comment in _macos_op_socket_mount_source
explicitly notes Docker Desktop is not wired:
return None # Docker Desktop uses a different relay; not wired yet (see the todo).Risk: Any macOS user with 1Password SSH agent forwarding silently gets a non-functional SSH
agent mount. Commit signing and git push over SSH will fail without feedback beyond the warning.
Direction: Verify on macOS hardware (tracked in bd (macOS SSH-agent forwarding = bd main-5ki)),
or add a hard [yellow]warning[/yellow] that macOS SSH agent forwarding is not yet supported
instead of a partial attempt that fails silently.
The _gnupg_mounts docstring in src/harnessed/launcher.py states that full openpgp GPG signing in-container (for
non-YubiKey keys, even via agent) is a "scoped follow-up." The current _gnupg_mounts correctly
avoids mounting private-keys-v1.d/ (prevents secret exfil), but the consequence is that software
GPG keys cannot sign commits in-container. The follow-up is tracked in bd (macOS SSH-agent
forwarding = bd main-5ki).
Risk: Users expecting GPG commit signing via software keys (not YubiKey) will get a silent failure (git commit succeeds, gpg signing fails or falls back unsigned depending on git config).
The following src/harnessed/ modules have no corresponding tests/test_<module>.py:
| Module | Size | Notes |
|---|---|---|
assemble.py |
134 lines | Tested indirectly via integration; no direct unit tests |
capability.py |
593 lines | Tested by tests/test_recipes_integration.py only (integration) |
cli.py |
(argparse CLI) | Scan CLI surface entirely untested |
report.py |
71 lines | No test file |
capability.py is the most significant gap — it drives live capability introspection and contains
non-trivial parsing logic (_sse_to_objects, _names_from_llm_json, _collect_server_names).
These are pure functions that could be unit-tested without podman.
src/harnessed/scan.py (417 lines) exposes run_image_scan_online, run_image_scan,
run_snyk_container_scan, and run_source_scan. These are called from src/harnessed/cli.py
(the harnessed-tools binary) but NOT from src/harnessed/launcher.py (the main harnessed CLI
entry point). The ROADMAP.md documents open decision D2: "keep run_image_scan_online only, or
remove the gating scanner + harnessed-tools + test_scan.py entirely?"
As of 2026-07-05, scan.py is live code that ships with the package, but the primary user-facing
workflow (harnessed launch/build) calls scan only indirectly via the derived image's baked scan
layer. The harnessed-tools scan* subcommands are the only direct consumers.
Risk: the scan subsystem is maintained but the decision about its long-term role is deferred.
Any refactor that touches _TIMEOUT, severity thresholds, or the osv-scanner JSON schema must also
update tests/test_scan.py.
Action: resolve D2, track it as a bd issue (main-fut reference in roadmap).
pyproject.toml has no [tool.ruff], [tool.mypy], or any formatter configuration. The project
uses type hints throughout (src/harnessed/schema.py, launcher.py, etc.), but nothing enforces
them. There is one # noqa: E731 suppression on the norm = lambda in src/harnessed/launcher.py
(lambda assignment) and one in test imports, implying ruff/flake8 was run manually at some point but not
wired into CI.
Risk: type regressions and style inconsistencies accumulate silently between contributors.
pyproject.toml pins pip-audit==2.10.1 as a hard runtime dependency (not a dev-only tool). Exact
version pins on security scanners can mask available CVE database updates — the scanner ships its
own advisory DB snapshot. If a critical vulnerability is added post-2.10.1, installed users won't
get it until the pin is bumped explicitly.
Risk: harnessed-tools scan may report clean against a stale advisory database.
src/harnessed/launcher.py:858-863 runs podman machine ssh -f -N -T -R ... in the background
(-f flag). Each harnessed launch invocation on macOS will attempt this, spawning a new
backgrounded ssh process. While StreamLocalBindUnlink=yes clears the stale socket on retry, the
old ssh process is orphaned (not killed). On repeated launches in the same session, processes
accumulate.
Risk: Low on a typical dev workstation but worth addressing before the macOS path is verified.
These are intentional cleanup paths, but each silently swallows errors that could indicate a real problem:
-
src/harnessed/launcher.py:450— cleanup of a temp env file on write failure: swallowsOSErrorfromos.unlink(tmp). The outerraisestill fires, so the write failure propagates correctly — only the cleanup failure is silenced. Acceptable, but the outerexcept Exception:is broader than needed (could beOSError | PermissionError). -
src/harnessed/launcher.py:830—gpgconf --list-dirsfailure falls back to a platform-default path. Reasonable defensive coding. -
src/harnessed/launcher.py:1497—secrets_env_file.unlink()failure in finally block. The outer finally is the right place; the silence here means a leaked temp file on cleanup failure would not be reported. -
src/harnessed/capability.py:265— teardown of a container on_execfailure: swallowsSubprocessError | OSError. This runs podman rm; if it fails, the container leaks but the capability test result still propagates.
None of these are bugs today, but they will be invisible in observability tooling.
catalog/recipes/floating-recipe/Dockerfile:6 intentionally contains:
RUN git clone --branch main https://example.com/fake-repo.git /opt/fake
This is documented as a test fixture for validate_pin(). It does not trigger a real build (the
URL is unreachable and the assembler rejects it before any build). However, a reader unfamiliar
with the fixture purpose may be confused. The file has a header comment explaining it, but the
comment is not in the Dockerfile's standard location (it's after a blank line, not at line 1).
pyproject.toml allows any typer 0.x from 0.12 onward. Typer's pre-1.0 API has had breaking
changes between minor versions (e.g. typer.Typer constructor parameters, app.command decorator
behavior). No upper minor bound is set, so a 0.13 or 0.14 release could break CLI entry points.
src/harnessed/launcher.py:407-452 — _resolve_launch_secrets() writes a mode-0600 temp file
(tempfile.mkstemp) containing resolved secret values. The file is unlinked in a finally block
at launcher.py:1494-1498. On Linux, any root or same-uid process can read /proc/<pid>/fd/<n>
between write and unlink. The window is small (seconds), but for secrets like API keys passed to
containers, this is a potential local information leak on multi-tenant machines. The code is
otherwise correct (mode-0600, finally-unlinked, prefix-named to be identifiable).
This matches the documented threat model (T-05-06) and is noted in the code. Flag here for completeness since the risk is non-zero on shared hosts.
pyproject.toml locks ruamel.yaml to >=0.18,<0.19. This is a stable, widely-used library but
is still pre-1.0. If 0.19 introduces breaking changes to the YAML(typ="safe") API (used in
schema.py:27), upgrading requires re-pinning and retesting the entire schema parser. Low urgency
but worth noting as the library nears 1.0.
These are documented open decisions, not newly discovered concerns:
-
D2 — scan subsystem future: keep
run_image_scan_onlineor removeharnessed-tools scan*entirely. Tracked asbd main-fut. -
D-compose — multi-container
compose:service support (catalog/stacks/pattern) is explicitly bottom priority (bd main-9cf). - macOS agent forwarding — tracked in bd (macOS SSH-agent forwarding = bd main-5ki) for H-2 and H-3 above.
Start Here
Guides
- Recipe authoring
- Service authoring
- Stacks
- Extending stacks (proposed)
- Recipe catalog
- System prompt & rules (proposed)
- Secrets
- AWS SSO
- Pulumi (host login forwarding)
- Egress & exposing services
- Container filesystem
- Git hooks
- Troubleshooting
- Pin management (harnessed update)
Codebase Map
Planning & Roadmap
- open work: GitHub Issues
Research & Prompts
- research/ (home-folder requirements per harness, browse in-repo)
- prompts/ (reusable prompt templates, browse in-repo)