Self-packaging #10: Documentation (CLI, architecture, design, security)

[jrosskopf]

Part of epic #40. Should land alongside or just after #5/#7/#8.

Goal

User-facing and architect-facing documentation for the self-packaging
feature. Covers the new CLI surface, the design choices (so future
maintainers know why ZIP-append and not 7z, why embed:// and not
extract-to-tmpdir), and the no-secrets-in-bundle invariant.

Scope

• docs/CLI_REFERENCE.md — new sections:

  • flapi pack / flapi info / flapi unpack (synopsis, flags,
    exit codes, examples).
  • FLAPI_CONFIG and FLAPI_LOG_LEVEL env vars and the CLI > env >
    config > default precedence rule.
  • macOS --macos-append flag, with the notarisation warning.

• docs/CONFIG_REFERENCE.md — new "Secrets and the bundle"
  subsection:

  • The default exclude list (*.env, secrets/*, *.pem,
    *.key).
  • The env-var contract for credentials (AWS_/GOOGLE_/AZURE_*/
    FLAPI_CONFIG_SERVICE_TOKEN).
  • The rationale: "if secrets were in the bundle, the bundle would
    be a secret."
  • The --allow-secrets override is documented as testing-only.

• docs/spec/ARCHITECTURE.md — add a "Self-packaging" component
  to the diagram and a short prose section covering bundle locator +
  embedded provider + embed FS + their data flow.

• docs/spec/DESIGN_DECISIONS.md — capture the four
  spike-validated decisions with their why-not counterparts:

  1. ZIP-append (vs. tar/7z/custom container)
  2. IFileProvider reuse (vs. extract-to-tmpdir)
  3. embed:// scheme (vs. teaching every DuckDB-touching code path
     about bundles)
  4. libarchive (vs. libzip / minizip-ng / bit7z)

• docs/spec/components/security.md — cross-reference the
  no-secrets-in-bundle invariant; how pack enforces it; threat
  model implications.

• CLAUDE.md — short "Self-packaging" entry under Core Concepts
  so future Claude sessions know it exists.

Green criteria

• Every new file or symbol shipped by Issues noticed while dockerizing the bq connection to flapi #1–Feature Request: Support Apache Arrow IPC Streaming over HTTP #9 has at least one mention
  in user docs.
• A reader who skims CLI_REFERENCE.md can produce a bundled binary
  without needing to read the source.
• The "why ZIP, why libarchive" rationale is captured permanently —
  this is the kind of context that gets lost six months later.

Files

• Modified: docs/CLI_REFERENCE.md, docs/CONFIG_REFERENCE.md,
  docs/spec/ARCHITECTURE.md, docs/spec/DESIGN_DECISIONS.md,
  docs/spec/components/security.md, CLAUDE.md.

Notes

• The spike README at
  https://github.com/jrosskopf/research/tree/main/2026-05-21-flapi-self-packaging
  already contains most of the prose we need — lifting and
  contextualising it for flapi-proper is the bulk of the work.