Skip to content

fix!: harden the vault against injection, silent data loss, and weak keys - #18

Merged
jahvon merged 7 commits into
mainfrom
feat/harden-v0.3.0
Jul 26, 2026
Merged

fix!: harden the vault against injection, silent data loss, and weak keys#18
jahvon merged 7 commits into
mainfrom
feat/harden-v0.3.0

Conversation

@jahvon

@jahvon jahvon commented Jul 26, 2026

Copy link
Copy Markdown
Member

Summary

A security and correctness pass over the whole package, prompted by an audit done while
scoping curated external-provider presets. Building presets on top of the library as it
stood would have meant shipping a secrets feature on a foundation that could corrupt and
leak the secrets it held.

Every fix here replaces behaviour that failed silently — no error, no log, no crash.
That is the common thread, and it is why none of it had been noticed.

Three findings were verified empirically rather than argued from reading the code:

  1. Command injection on every SetSecret. SetSecret rendered the plaintext into a
    command string that is then parsed and executed by a shell, and the template engine does
    no quoting. This needed no attacker — an ordinary password was enough. p@$$w0rd had
    $$ expanded to the process ID and a different secret was stored, silently;
    correct horse battery word-split and stored correct; anything containing ; ran
    arbitrary commands. A set/get round trip through the real shell now survives
    $(id), backticks, quotes, globs, spaces and newlines byte-exact, and an injection
    payload provably does not execute.

  2. Concurrent writes lost secrets. load() ran once at construction and every save
    rewrote the whole file from that snapshot, with nothing synchronizing writers across
    processes. Measured directly: with the new lock disabled, 8 concurrent writes collapse
    to 1 surviving secret.

  3. scrypt ran at N=2²⁰ (~1 GiB and seconds per derivation). Concurrent derivations were
    a trivial local denial of service. The crypto test suite drops from 35s to 0.7s as a
    direct result of correcting it.

Notable Changes

Secret handling

  • Secrets travel to external backends over stdin only. Configs referencing {{value}}
    or {{password}} in any command template are rejected at load rather than silently
    corrupting values.
  • The input-template plumbing never worked: SetSecret, DeleteSecret and ListSecrets
    each guarded on their own InputTemplate but rendered Get's, Metadata gated on
    List's, and the value was never exposed to input templates at all — so the shipped
    pass example's stdin-based set has never functioned.
  • execute() no longer merges stderr into stdout on success, which was concatenating
    backend warnings (gpg: WARNING: ...) onto the secret value itself.

Data loss

  • A zero-length vault file was read as "no vault here", so the constructor initialized and
    immediately overwrote it — every secret gone, success reported. Now ErrVaultCorrupt,
    file left untouched.
  • Mutations take a cross-process advisory lock and re-load inside it. The per-instance
    RWMutex never spanned two providers, let alone two processes.
  • The temp file had a fixed <path>.tmp name written with os.WriteFile, so concurrent
    savers truncated each other and a planted symlink was followed with 0600 silently not
    applied. Now os.CreateTemp + O_EXCL, explicit chmod, fsync before rename, dir fsync after.

Crypto

  • DeriveKey tested salt == nil, but []byte("") from a variable string is non-nil, so
    the natural "generate me a salt" call ran scrypt unsalted — a deterministic,
    precomputable KEK identical for every user with the same passphrase.
  • aes.NewCipher accepts 16/24/32 bytes, so a short key silently downgraded an "AES256"
    vault to AES-128/192. Now exactly 32 bytes.
  • Salts carry the parameters that produced them, so future cost changes cannot silently
    change an existing key.

Crashes, validation, lifecycle

  • Every accessor guarded against nil state instead of panicking after Close().
  • Vault IDs are charset-validated and the resolved path is proved to stay inside the storage
    directory. filepath.Clean resolves traversal rather than sanitizing it:
    Clean("vault-../../../tmp/evil.enc") is "../tmp/evil.enc".
  • Keyring entry names are length-prefixed; "%s-secret-%s" let vault a/key b-secret-c
    collide with vault a-secret-b/key c.
  • New() returns a genuinely nil Provider on error rather than a nil pointer in a
    non-nil interface.
  • AgeVault refuses to create a vault, or remove a recipient, that would leave no recipient
    matching your own identity — one mistyped public key previously produced a permanently
    unreadable vault.
  • expandEnv no longer mutates the shared config map under a read lock (an unrecoverable
    concurrent map writes fault, guaranteed to recur with the pass example).
  • HasSecret no longer re-enters RLock, which deadlocks if a writer arrives in between.

Examples

  • Three of the four shipped configs were rejected by the new validation; the fourth had
    never worked. All rewritten, plus correctness fixes independent of the injection issue —
    bw get item matches by substring (so keys api and api_key broke each other), AWS
    used a listSeparator key that is not the field name (separator) so it was never
    applied, and pass's exists decrypted via GPG just to answer a boolean.
  • New TestShippedExampleProvidersAreUsable loads every config and renders all six
    operations. Nothing exercised these files before, which is how they drifted; it caught two
    broken output templates during this work.

Tests

  • The previous mock ignored both cmd and input and returned the first value in a map, so
    no test could observe a wrongly-rendered template — precisely why these bugs shipped.
    Replaced with a capturing mock. 126 tests pass under -race.

Compatibility

Breaking changes are documented in a new "Upgrading to v0.3.0" table in the README. Local
vault files written by earlier versions are read without migration.

Downstream cost was measured, not estimated: pointing flowexec/flow at this branch breaks
exactly 3 call sites, all from the Metadata() signature — internal/vault/demo.go:64
and internal/io/vault/view.go:226-227.

One audit finding was deliberately not fixed, and is documented rather than papered
over: SecureBytes/Zero() cannot protect the provider state maps, which hold immutable Go
strings whose bytes are neither addressable nor zeroable, and PlainTextString() hands out
more un-zeroable copies by design. Converting the maps to []byte would change the on-disk
format of every local vault to buy protection the next API call gives away again.

Change Type

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

🤖 Generated with Claude Code

https://claude.ai/code/session_01HtuDuGkSqTfAXSepf8b51p

jahvon and others added 7 commits July 26, 2026 14:42
SetSecret rendered the plaintext into a command string that is then parsed
and executed by a shell, and the template engine does no quoting. An
ordinary password was enough to break it: "p@$$w0rd" had $$ expanded to the
PID and a *different* secret was stored, silently; "correct horse battery"
word-split and stored "correct"; a value containing ";" ran arbitrary
commands.

Secrets now travel over stdin only. ExternalConfig.Validate rejects configs
that still reference {{ value }} or {{ password }} in any command template,
so the unsafe shape fails loudly at load instead of silently corrupting.

Fixing that surfaced the reason it was never noticed: the input-template
plumbing did not work. SetSecret, DeleteSecret and ListSecrets each guarded
on their own InputTemplate but rendered Get's, Metadata gated on List's, and
the value was never exposed to input templates at all -- so the shipped pass
example's stdin-based set has never worked.

Also in this pass:
- execute() no longer merges stderr into stdout on success, which was
  concatenating backend warnings onto the secret value itself.
- expandEnv returns a new map instead of mutating the shared config under a
  read lock (an unrecoverable "concurrent map writes" fault; guaranteed to
  recur with the pass example, whose "$(tty)" never stops looking expandable).
- HasSecret delegates to an unlocked helper rather than re-entering RLock,
  which deadlocks if a writer arrives in between.
- os.ExpandEnv no longer runs on command templates. It ran before the shell
  parsed them, destroying $VAR/${VAR}/$1/$?/$@, making a literal $
  unwritable, and substituting before quoting. The interpreter already
  resolves $VAR from the same environment, correctly.
- Metadata() returns an error instead of an empty struct, so a broken
  command, a timeout and "not configured" stay distinguishable.
- Every method reports ErrVaultClosed after Close() instead of proceeding.
- New optional not_found_pattern separates "absent" from "the backend is
  broken", which HasSecret could not previously tell apart.
- SetExecutionFunc takes the mutex; the execution context is settable.
- Timeout is parsed and validated at load, not at first use.

BREAKING CHANGE: Provider.Metadata() now returns (Metadata, error), and
external configs that interpolate the secret value into a command template
are rejected.

Tests: the previous mock ignored both cmd and input and returned the first
value in a map, so nothing could observe a wrongly rendered template -- which
is precisely why these shipped. Replaced with a capturing mock, plus a
round-trip through the real shell asserting that values containing $(id),
backticks, quotes, globs and newlines survive byte-exact and that an
injection payload does not execute.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtuDuGkSqTfAXSepf8b51p
Three ways the local providers (aes, age, unencrypted) could destroy a
vault, all silent:

1. A zero-length vault file was read as "no vault here". load() returned
   nil leaving state nil, the constructor took that for a new vault, and
   init() immediately save()d over it. Every secret gone, success reported.
   An existing but empty file is now ErrVaultCorrupt, and the file is left
   untouched for the user to restore.

2. The temp file had a fixed "<path>.tmp" name written with os.WriteFile,
   so two concurrent savers truncated each other's partial write, and a
   symlink planted at that predictable path was followed -- with the 0600
   mode silently not applied to the pre-existing target. Now os.CreateTemp
   (random name, O_EXCL), explicit chmod, fsync before the rename, and an
   fsync of the directory after it.

3. Nothing synchronized writers across processes. load() ran once at
   construction and every save rewrote the whole file from that snapshot,
   so two `flow secret set` runs silently lost one of the two updates. The
   per-instance RWMutex never helped: it does not span two providers in one
   process, let alone two processes. Mutations now take an advisory lock on
   a .lock sidecar and re-load inside it before applying and saving.

Verified rather than assumed: with the lock disabled, the new concurrency
test collapses 8 concurrent writes to 1 surviving secret.

Alongside those, in the same files:
- Every accessor guards a nil state instead of dereferencing it. GetSecret,
  SetSecret, DeleteSecret, ListSecrets, HasSecret and the age recipient
  methods all panicked after Close(); only Metadata() checked.
- save() reports ErrVaultClosed instead of silently doing nothing.
- ValidateSecretKey is applied on every entry point, not just SetSecret.
- Vault directories are created 0700 rather than group-readable 0750.
- Read failures that are not "not exist" (EACCES, I/O errors) no longer
  masquerade as ErrVaultNotFound, which callers read as "make a new one".
- AgeVault refuses to create a vault, or remove a recipient, that would
  leave no recipient matching your own identity -- previously a single
  mistyped public key produced a permanently unreadable vault, and
  RemoveRecipient guarded only the *last* recipient, not yours.
- RemoveRecipient commits to state only after the new set parses, so a
  failure cannot leave memory inconsistent with disk.

BREAKING CHANGE: an existing empty vault file is now an error instead of
being silently reinitialized, and closed vaults return ErrVaultClosed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtuDuGkSqTfAXSepf8b51p
DeriveKey's "generate a random salt" branch tested salt == nil, but
[]byte("") from a variable string is non-nil. The natural call --
DeriveEncryptionKey(passphrase, "") meaning "I have no salt, please make
one" -- therefore ran scrypt with a zero-length salt and returned "" as the
salt. The result was an unsalted, fully deterministic KEK: identical for
every user with the same passphrase, and precomputable. The tests only ever
covered nil, never []byte(""), so it went unnoticed. Now len(salt) == 0,
and salts shorter than 16 bytes are rejected outright.

aes.NewCipher accepts 16, 24 and 32 byte keys, so a short key silently
downgraded an "AES256" vault to AES-128 or AES-192 with no warning -- and
any 32-character passphrase made of base64 alphabet characters decodes
cleanly to 24 bytes and was accepted as a key. EncryptValue and DecryptValue
now require exactly 32 bytes.

scrypt ran at N=1<<20, r=8: roughly 1 GiB of memory and several seconds per
derivation, which thrashes or OOMs on a modest machine and makes concurrent
derivations a trivial local denial of service. Lowered to N=1<<16 (~64 MiB).
The crypto test suite drops from 35s to 0.7s as a direct result.

Because changing the cost silently changes every derived key, the salt now
carries the parameters that produced it ("scrypt$N=...,r=...,p=...$<b64>").
Salts without a parameter block predate this and still derive with the
original cost, so nothing already issued changes meaning. Verified that
nothing outside this package derives keys: flow uses GenerateEncryptionKey,
which is unaffected.

The AES-256-GCM construction itself was already correct -- fresh random
nonce per seal, standard Seal(nonce, nonce, ...) idiom, length-checked open,
real authentication -- and is unchanged.

BREAKING CHANGE: keys that are not exactly 32 bytes are now rejected, and
DeriveKey returns a parameter-tagged salt that must be passed back verbatim
rather than base64-decoded first.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtuDuGkSqTfAXSepf8b51p
…sion

A vault ID is interpolated straight into a filename, and filepath.Clean does
not sanitize that -- it *resolves* it. Clean("vault-../../../tmp/evil.enc")
is "../tmp/evil.enc", because the first ".." pops the literal "vault-.."
element and the rest survive; joining that onto the storage directory
escapes it, so a crafted ID could make save() overwrite an arbitrary file.
Config.Validate only checked that the ID was non-empty. IDs are now
charset-validated, and the resolved path is separately proved to stay inside
the storage directory.

Keyring entry names were built as "%s-secret-%s" over an unvalidated ID and
a key charset that permits dashes, so vault "a" with key "b-secret-c" and
vault "a-secret-b" with key "c" both mapped to "a-secret-b-secret-c" -- one
vault silently reading, overwriting or deleting another's secret.
Length-prefixed now.

Also:
- KeyringVault only initializes metadata on keyring.ErrNotFound. Treating
  every error as "vault doesn't exist" meant a locked keychain, a DBus
  failure or a dismissed prompt overwrote the real Created timestamp.
- New() returns an explicit nil Provider on error. Returning the typed
  pointer wrapped a nil *AgeVault in a non-nil interface, so the usual
  `if provider != nil` guard passed and the next call panicked.
- Every exported constructor calls Config.Validate(); previously only
  vault.New() did, so the direct constructors bypassed validation entirely.
- StoragePath now goes through expandPath/validateSecurePath like key and
  identity paths always did. The README's own WithAESPath("~/secrets.vault")
  created a literal "~" directory in the working tree.
- expandPath: ".config/x" no longer loses its leading dot, "../x" no longer
  silently drops the parent reference, "~user/x" no longer becomes
  "<home>user/x", and "$HOME/vault" resolves instead of looking up a
  variable named "HOME/vault". An unset variable is an error rather than
  expanding to "/vault".
- validateSecurePath compares path elements instead of substrings, so
  "my..backup" is allowed and "/etcetera" is no longer caught by "/etc".
- ValidateSecretKey rejects a leading dash (flag injection into backend
  CLIs) and "."/".." (path elements), is applied at every provider entry
  point rather than only SetSecret, and compiles its regex once instead of
  on every call.
- KeyResolver propagates key-file failures. `err == nil &&` discarded typos
  and permission errors, surfacing them as a generic "no encryption keys
  found" or silently falling through to another source.
- SecretValue.String() takes a value receiver so a dereferenced copy is
  masked too; Zero() drops its forced per-call runtime.GC() and the
  cargo-cult random prefill, and now documents what it cannot guarantee.
- DefaultVaultKeyEnv is a const; as a var any dependency could repoint every
  default key lookup in the process.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtuDuGkSqTfAXSepf8b51p
…aring

- WithLocalPath switched on the provider type at the moment it ran, making
  it a silent no-op unless WithProvider was passed first; the failure then
  surfaced as a confusing "storage path is required". The path is now
  resolved after every option has been applied.
- The vault format version was written on every save but never read back, so
  a file from a future format would have been parsed as though it were the
  current one. Loads now reject a version newer than the build understands.
- AESState.Version carried a `json:` tag on a struct marshaled with yaml.v3,
  which ignores json tags entirely; it keyed correctly only by lowercasing
  coincidence. Corrected to `yaml:`.
- Close() now clears the secrets map rather than only dropping the pointer to
  it, and says plainly in a comment why that is the limit of what it can do.
- README: the storage path is a directory, but every example named it like a
  file ("~/secrets.vault"), which -- now that ~ expands correctly -- would
  create a directory by that name. Also updated for the Metadata() signature.

Note on secret lifetime, deliberately not "fixed": SecureBytes/Zero() cannot
protect the provider state maps, which hold immutable Go strings whose bytes
are neither addressable nor zeroable, and PlainTextString() hands out more
un-zeroable copies by design. Converting the maps to []byte would change the
on-disk format of every local vault for a benefit that PlainTextString gives
away again immediately. The limitation is now documented on the types rather
than papered over with machinery that does not deliver.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtuDuGkSqTfAXSepf8b51p
…ested

Three of the four example configurations were rejected outright by the new
validation because they interpolated the secret into a shell command, and
the fourth (pass) had never worked: its stdin-based set depended on the
input-template bug. Nothing in the repo loaded these files, so all of it
went unnoticed.

Rewritten so the secret reaches each backend over stdin, and to fix
correctness problems independent of the injection issue:

- bitwarden: `bw get item <name>` matches by substring and errors on
  multiple hits, so with keys "api" and "api_key" the old get, exists and
  delete were all broken. Replaced with an exact-name jq select over
  `list items`. The old set built JSON by string-interpolating the value,
  so any quote or backslash in a secret produced malformed JSON; jq -Rs now
  builds it from stdin. Deletes are --permanent rather than leaving the
  secret in the trash, and every call is --nointeraction as there is no TTY.
- 1password: set only ever ran `item create`, so it failed whenever the item
  already existed; it now probes and edits or creates. `--` before the key
  closes flag injection.
- aws-ssm: `describe-parameters` listed the entire account and needed
  account-wide IAM; replaced with `get-parameters-by-path`, whose JSON
  output also lets the CLI's paginator return every page. The config also
  specified "listSeparator", which is not the field name -- the struct tag
  is "separator" -- so the tab separator was silently never applied while
  the output was tab-delimited. `--value file:///dev/stdin` keeps the secret
  out of argv.
- pass: exists ran `pass show`, decrypting (and possibly triggering a
  pinentry prompt) just to answer a boolean; it is now `test -f`, a builtin
  in the interpreter. list no longer needs `tree`, which `pass ls` shells
  out to and which is frequently not installed. The `GPG_TTY: "$(tty)"`
  entry never did anything: expandEnv uses os.ExpandEnv, not a shell, so the
  value stayed the literal string.

All four now declare not_found_pattern, so an expired session or a
permissions error is no longer indistinguishable from "does not exist".

Added TestShippedExampleProvidersAreUsable, which loads every config,
renders all six operations against provider-shaped sample output, checks the
list templates actually parse it, and asserts the secret reaches stdin and
never the command string. It caught two broken output templates in this
commit before they shipped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtuDuGkSqTfAXSepf8b51p
Each breaking change replaces behaviour that failed silently, so the upgrade
table says what was wrong rather than only what changed. Also corrects the
external provider example, which showed the secret being interpolated into a
command -- the exact shape that is now rejected.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtuDuGkSqTfAXSepf8b51p
@jahvon
jahvon merged commit 8305c84 into main Jul 26, 2026
2 checks passed
@jahvon
jahvon deleted the feat/harden-v0.3.0 branch July 26, 2026 20:24
jahvon added a commit to flowexec/flow that referenced this pull request Jul 26, 2026
…418)

# Summary

Picks up [vault v0.3.0](flowexec/vault#18), a
security and
correctness release. In the library, secrets no longer travel through a
shell command,
empty vault files are no longer silently reinitialized and overwritten,
writes are
serialized across processes, and closed vaults report an error instead
of panicking.

The surface it breaks in flow is small — three call sites — but one
compatibility problem
was **not** a compile error and is the more important half of this PR.

**Notable Changes**

- `demoVaultProvider` implements the new `Metadata() (Metadata, error)`.
- `vaultFromName` handles the new error, and reports a metadata failure
as a
`metadataError` field rather than failing the whole view. For an
external vault that
failure means the backend CLI is missing or the session has expired —
precisely when a
user wants to inspect the vault's configuration. It also called
`Metadata()` **twice**,
which for an external vault meant running the backend command twice per
view.
- `ValidateIdentifier` now rejects a leading dash or underscore. flow's
rule was a
*superset* of the library's new `ValidateVaultID`, so flow would accept
a name like
`-myvault` or `_myvault` that the library then refuses — failing at
creation with a less
specific message, and leaving any vault already created under such a
name **unreachable**
after upgrade. A new test asserts the subset property directly, because
the two rules
  live in separate repositories and will otherwise drift apart silently.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant