Skip to content

Add compose-to-docker bin: run a catalog app locally under the same hardening the cluster applies #268

Description

@v0l

Problem

Four catalog apps have been found unable to start in the last day — #248 (HAVEN missing 11 required env vars), #256 (three apps with no user: under runAsNonRoot), #263 (user: root + cap_drop: ALL breaks the mariadb entrypoint chown), #264 (read-only root filesystem with no writable scratch). Every one was found by reading Rust and comparing it against upstream image docs. None was found by starting the container, because there is no way to start one without a cluster.

We have compose-validate (lnvps_compose/src/bin/compose-validate.rs, master), and it is doing its job — it parses, validates and reports the footprint. But validation is a static check on the document. It cannot tell you that the image's entrypoint wants to write to /run, or that the app exits because RELAY_URL is empty. Those only surface when a container actually starts.

Ask

A second bin in lnvps_compose that transforms an lnvps-compose document into a standard docker-compose.yaml, so an agent can run docker compose up locally and watch the app come up — or fail — before opening a PR.

Design

Bin: lnvps_compose/src/bin/compose-to-docker.rs, following compose-validate's conventions (path args or - for stdin, non-zero exit on failure).

cargo run -p lnvps_compose --bin compose-to-docker -- app.yaml \
  --out-dir .local/haven --config RELAY_URL=wss://localhost:7777 --hostname localhost
docker compose -f .local/haven/docker-compose.yaml up

Resolution must go through the same code the operator uses, not a re-implementation:

  • resolve_config (lnvps_compose/src/lib.rs:1013) for submitted values ∪ declared defaults, with the same required-field error.
  • Compose::config_defaults (lib.rs:481), then generated secrets, then supplied config, then HOSTNAME — the ordering in build_vars (lnvps_operator/src/app_deployments.rs:827).
  • Compose::resolve_env (lib.rs:781), resolve_files (lib.rs:799), resolve_init (lib.rs:740).
  • Secrets generated like ensure_secrets (app_deployments.rs:867), but written to the out-dir so a re-run of the same app is stable.

Mapping:

lnvps-compose docker-compose why
service key service key k8s Service name == compose service name (app_deployments.rs:522), so mariadb:3306 resolves the same on docker's default network
image image
env (substituted) environment
volumes[] named volume mounted at path PVC equivalent
files[] rendered into --out-dir, bind-mounted :ro at path k8s mounts these read-only via subPath
ports[].container expose; publish to the host only for expose: ingress Expose (lib.rs:292)
resources.cpu / .memory cpus / mem_limit
user user lib.rs:81
depends_on depends_on
init[] (lib.rs:85) one-shot services + depends_on: {condition: service_completed_successfully}, tmpfs /tmp mirrors the init emptyDir at app_deployments.rs:631

The hardening is the point — emit it, do not skip it. A transform that produces a permissive docker-compose would have started all four broken apps cleanly and told us nothing. container_security_context_for (app_deployments.rs:332-348) sets:

  • read_only_root_filesystem: trueread_only: true
  • capabilities.drop: ["ALL"]cap_drop: [ALL]
  • allow_privilege_escalation: falsesecurity_opt: ["no-new-privileges:true"]
  • run_as_user / runs_as_root (lib.rs:160,165) → user:

#263 and #264 both reproduce locally under those four lines and under nothing weaker.

Known non-equivalences — document them, don't paper over them

  • Volume ownership. k8s sets fsGroup from run_as_user so a fresh PVC is writable by a non-root container. Docker has no equivalent; a fresh named volume is root-owned 0755 and a non-root service cannot write to it. Expect a class of failure that appears locally but not in the cluster. State the workaround in the docs rather than making the transform quietly chown.
  • No ingress, TLS or cert-manager: ${HOSTNAME} is whatever --hostname says.
  • No Recreate / RWO PVC semantics, no scheduler, no capacity accounting.

A green docker compose up means the image starts and the app's own startup checks pass under our security context. It is not a deployment test.

Docs

  • docs/managed-app-examples.md: new section after "Validating a compose document" (master line 250) — validate, transform, docker compose up, and the non-equivalences above.
  • docs/agents/build-and-test.md: add the flow as the expected step before opening a PR that touches a catalog compose.

Acceptance


Filed by Alejandra (PM) on Kieran's account.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions