feat(self-packaging #45): flapi pack / info / unpack subcommands#55
Merged
Conversation
This was referenced May 22, 2026
Part of #40, depends on #41-#44. Closes the loop on self-packaging: the same binary that serves bundled content now also produces it. Library (src/{include/,}pack.cpp): - Pack(in_dir, out_path, options) -> PackResult 1. host bytes from `options.host_binary_override` (default `GetSelfPath()`) -- if the host already has a trailing ZIP, only the pre-bundle prefix is copied, so re-pack is idempotent 2. recursive walk of `in_dir`; refuses files matching the default secret exclude list unless `options.allow_secrets` is set 3. WriteArchive (#41) with SOURCE_DATE_EPOCH stamping and sorted entries, appended to the output 4. best-effort exec-bit copy from host -> output (Unix only) - PrintBundleInfo / PrintBundleInfoForPath -- EOCD offset, size, entry list of either GetSelfPath() or an explicit binary - UnpackBundle / UnpackBundleFromPath -- restore entries to a dst dir - IsSecretExcluded -- default deny list: *.env at any depth, secrets/ segment at any depth, *.pem, *.key CLI (src/main.cpp): - argparse subparsers for `pack`, `info`, `unpack` - zero-subcommand invocation (`./flapi -c flapi.yaml`) still runs the server, preserving the contract for existing operators - subcommand handlers return early -- never reach server startup - pack reports `Packed N entries (M bytes) into <path>` - unpack reports `Unpacked N entries to <path>` - error paths emit a single line to stderr and exit 1 The host_binary_override knob (PackOptions) is the testability seam: the sanitiser-instrumented test binary is ~1.7 GB and copying it for every test case would push runtime past several minutes per test. Tests inject a 4-KiB fake host with the EOCD signature scrubbed so the locator can't mistake it for an existing bundle. Tests (test/cpp/pack_test.cpp, 10 cases / 38 assertions): - IsSecretExcluded: default deny list + non-matches - Pack: entry count, output exists, EOCD locatable, bundle round-trip - exclude enforcement: PackError on .env, --allow-secrets bypass - idempotence: pack -> pack -> pack with stacked outputs, size stable - host prefix preservation: byte-for-byte match against the fake host - PrintBundleInfo: returns non-zero + "none" for unbundled - PrintBundleInfoForPath: lists entries from a bundled binary - UnpackBundleFromPath: 4 entries restored, byte-for-byte equal Smoke test (real flapi binary, not test binary): - `flapi --help` shows three subcommands - `flapi info` on unbundled binary reports "none" with exit 1 - `flapi pack --in <dir>` with a .env in the input refuses with exit 1 - `flapi pack ... --allow-secrets` bundles, exit 0 - `<bundled> info` shows offset / size / entry list - `<bundled> unpack --to <dir>` restores all files Closes #45.
9d7a4f2 to
e49ce46
Compare
10 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of epic #40. Stacked on #54 which is stacked on #53 -> #52 -> #51.
Summary
Closes the loop on self-packaging. After #41-#44 the runtime can
serve a bundled config tree; this PR adds the producer so the
same binary that serves bundles also creates them. The same artifact
is server + packager.
src/{include/,}pack.cpppack/info/unpackinsrc/main.cpp. Zero-subcommand invocation(
./flapi -c flapi.yaml) still runs the server, so existingoperators see no change.
Library API
Pack()flow:options.host_binary_override(defaults toGetSelfPath()). If the host already has a trailing ZIP, only thepre-bundle prefix is copied -- re-pack is idempotent.
in_dir. Files matching the default secretdeny list (
*.envat any depth,secrets/segment at anydepth,
*.pem,*.key) causePackErrorunlessoptions.allow_secretsis set.archive_io::WriteArchive(feat(self-packaging #41): libarchive + archive_io RAII wrapper #51) withSOURCE_DATE_EPOCHstampingand sorted entries; appended to the output.
CLI
Subcommand handlers
returnearly; the server-mode code path isonly reached when no subcommand was given.
The
host_binary_overridetestability seamThe sanitiser-instrumented test binary is ~1.7 GB. Copying it for
every test case would push runtime past several minutes per case.
PackOptions::host_binary_overridelets tests substitute a 4-KiBfake host (with the EOCD signature scrubbed so the locator can't
mistake it for an existing bundle). Production code never touches
this knob.
Tests (10 cases / 38 assertions)
IsSecretExcludedenvoy.conf,foo.pemmed)Packproduces a bundled binaryPackround-tripReadArchive-> entries match input.envin input ->PackError; output not written--allow-secrets0..host_sizebyte-for-byte match the host filePrintBundleInfono bundlePrintBundleInfoForPathflapi.yaml,sqls/customers.sql,data/cities.csvUnpackBundleFromPathSmoke test (real flapi binary, not test binary)
flapi --helpshows three subcommands.flapi infoon unbundled binary ->"Bundle: none (filesystem mode)", exit 1.flapi pack --in <dir>with.envin input -> single-line stderr error, exit 1.flapi pack ... --allow-secrets->Packed N entries (M bytes), exit 0.<bundled> info-> offset (~1.6 GB into the host), size, entry list.<bundled> unpack --to <dir>-> all files restored.Test plan
subcommands + happy + error paths.
./flapi -c flapi.yamlstill launches server mode.What's not in this PR (deferred)
embed://-- the issue body mentionsrewriting paths in YAML so the same configs work both unpacked
and bundled. In practice the
IFileProviderfactory dispatch(feat(self-packaging #43): EmbeddedArchiveFileProvider + factory dispatch + startup wire-up #53) already handles non-remote paths transparently from a
bundle, so this isn't needed for the IFileProvider side. It's
still needed for SQL templates that contain
read_csv('data/x.csv')(DuckDB-side reads bypass IFileProvider). Filing as follow-up
rather than expanding this PR.
the explicit scope of Self-packaging #6: Integration tests — round-trip parity (spike behaviours 1-9) #46.
Closes #45. Part of #40. Stacked on #54 -> #53 -> #52 -> #51.