fix!: harden the vault against injection, silent data loss, and weak keys - #18
Merged
Conversation
SetSecret rendered the plaintext into a command string that is then parsed
and executed by a shell, and the template engine does no quoting. An
ordinary password was enough to break it: "p@$$w0rd" had $$ expanded to the
PID and a *different* secret was stored, silently; "correct horse battery"
word-split and stored "correct"; a value containing ";" ran arbitrary
commands.
Secrets now travel over stdin only. ExternalConfig.Validate rejects configs
that still reference {{ value }} or {{ password }} in any command template,
so the unsafe shape fails loudly at load instead of silently corrupting.
Fixing that surfaced the reason it was never noticed: the input-template
plumbing did not work. SetSecret, DeleteSecret and ListSecrets each guarded
on their own InputTemplate but rendered Get's, Metadata gated on List's, and
the value was never exposed to input templates at all -- so the shipped pass
example's stdin-based set has never worked.
Also in this pass:
- execute() no longer merges stderr into stdout on success, which was
concatenating backend warnings onto the secret value itself.
- expandEnv returns a new map instead of mutating the shared config under a
read lock (an unrecoverable "concurrent map writes" fault; guaranteed to
recur with the pass example, whose "$(tty)" never stops looking expandable).
- HasSecret delegates to an unlocked helper rather than re-entering RLock,
which deadlocks if a writer arrives in between.
- os.ExpandEnv no longer runs on command templates. It ran before the shell
parsed them, destroying $VAR/${VAR}/$1/$?/$@, making a literal $
unwritable, and substituting before quoting. The interpreter already
resolves $VAR from the same environment, correctly.
- Metadata() returns an error instead of an empty struct, so a broken
command, a timeout and "not configured" stay distinguishable.
- Every method reports ErrVaultClosed after Close() instead of proceeding.
- New optional not_found_pattern separates "absent" from "the backend is
broken", which HasSecret could not previously tell apart.
- SetExecutionFunc takes the mutex; the execution context is settable.
- Timeout is parsed and validated at load, not at first use.
BREAKING CHANGE: Provider.Metadata() now returns (Metadata, error), and
external configs that interpolate the secret value into a command template
are rejected.
Tests: the previous mock ignored both cmd and input and returned the first
value in a map, so nothing could observe a wrongly rendered template -- which
is precisely why these shipped. Replaced with a capturing mock, plus a
round-trip through the real shell asserting that values containing $(id),
backticks, quotes, globs and newlines survive byte-exact and that an
injection payload does not execute.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtuDuGkSqTfAXSepf8b51p
Three ways the local providers (aes, age, unencrypted) could destroy a vault, all silent: 1. A zero-length vault file was read as "no vault here". load() returned nil leaving state nil, the constructor took that for a new vault, and init() immediately save()d over it. Every secret gone, success reported. An existing but empty file is now ErrVaultCorrupt, and the file is left untouched for the user to restore. 2. The temp file had a fixed "<path>.tmp" name written with os.WriteFile, so two concurrent savers truncated each other's partial write, and a symlink planted at that predictable path was followed -- with the 0600 mode silently not applied to the pre-existing target. Now os.CreateTemp (random name, O_EXCL), explicit chmod, fsync before the rename, and an fsync of the directory after it. 3. Nothing synchronized writers across processes. load() ran once at construction and every save rewrote the whole file from that snapshot, so two `flow secret set` runs silently lost one of the two updates. The per-instance RWMutex never helped: it does not span two providers in one process, let alone two processes. Mutations now take an advisory lock on a .lock sidecar and re-load inside it before applying and saving. Verified rather than assumed: with the lock disabled, the new concurrency test collapses 8 concurrent writes to 1 surviving secret. Alongside those, in the same files: - Every accessor guards a nil state instead of dereferencing it. GetSecret, SetSecret, DeleteSecret, ListSecrets, HasSecret and the age recipient methods all panicked after Close(); only Metadata() checked. - save() reports ErrVaultClosed instead of silently doing nothing. - ValidateSecretKey is applied on every entry point, not just SetSecret. - Vault directories are created 0700 rather than group-readable 0750. - Read failures that are not "not exist" (EACCES, I/O errors) no longer masquerade as ErrVaultNotFound, which callers read as "make a new one". - AgeVault refuses to create a vault, or remove a recipient, that would leave no recipient matching your own identity -- previously a single mistyped public key produced a permanently unreadable vault, and RemoveRecipient guarded only the *last* recipient, not yours. - RemoveRecipient commits to state only after the new set parses, so a failure cannot leave memory inconsistent with disk. BREAKING CHANGE: an existing empty vault file is now an error instead of being silently reinitialized, and closed vaults return ErrVaultClosed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HtuDuGkSqTfAXSepf8b51p
DeriveKey's "generate a random salt" branch tested salt == nil, but
[]byte("") from a variable string is non-nil. The natural call --
DeriveEncryptionKey(passphrase, "") meaning "I have no salt, please make
one" -- therefore ran scrypt with a zero-length salt and returned "" as the
salt. The result was an unsalted, fully deterministic KEK: identical for
every user with the same passphrase, and precomputable. The tests only ever
covered nil, never []byte(""), so it went unnoticed. Now len(salt) == 0,
and salts shorter than 16 bytes are rejected outright.
aes.NewCipher accepts 16, 24 and 32 byte keys, so a short key silently
downgraded an "AES256" vault to AES-128 or AES-192 with no warning -- and
any 32-character passphrase made of base64 alphabet characters decodes
cleanly to 24 bytes and was accepted as a key. EncryptValue and DecryptValue
now require exactly 32 bytes.
scrypt ran at N=1<<20, r=8: roughly 1 GiB of memory and several seconds per
derivation, which thrashes or OOMs on a modest machine and makes concurrent
derivations a trivial local denial of service. Lowered to N=1<<16 (~64 MiB).
The crypto test suite drops from 35s to 0.7s as a direct result.
Because changing the cost silently changes every derived key, the salt now
carries the parameters that produced it ("scrypt$N=...,r=...,p=...$<b64>").
Salts without a parameter block predate this and still derive with the
original cost, so nothing already issued changes meaning. Verified that
nothing outside this package derives keys: flow uses GenerateEncryptionKey,
which is unaffected.
The AES-256-GCM construction itself was already correct -- fresh random
nonce per seal, standard Seal(nonce, nonce, ...) idiom, length-checked open,
real authentication -- and is unchanged.
BREAKING CHANGE: keys that are not exactly 32 bytes are now rejected, and
DeriveKey returns a parameter-tagged salt that must be passed back verbatim
rather than base64-decoded first.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtuDuGkSqTfAXSepf8b51p
…sion
A vault ID is interpolated straight into a filename, and filepath.Clean does
not sanitize that -- it *resolves* it. Clean("vault-../../../tmp/evil.enc")
is "../tmp/evil.enc", because the first ".." pops the literal "vault-.."
element and the rest survive; joining that onto the storage directory
escapes it, so a crafted ID could make save() overwrite an arbitrary file.
Config.Validate only checked that the ID was non-empty. IDs are now
charset-validated, and the resolved path is separately proved to stay inside
the storage directory.
Keyring entry names were built as "%s-secret-%s" over an unvalidated ID and
a key charset that permits dashes, so vault "a" with key "b-secret-c" and
vault "a-secret-b" with key "c" both mapped to "a-secret-b-secret-c" -- one
vault silently reading, overwriting or deleting another's secret.
Length-prefixed now.
Also:
- KeyringVault only initializes metadata on keyring.ErrNotFound. Treating
every error as "vault doesn't exist" meant a locked keychain, a DBus
failure or a dismissed prompt overwrote the real Created timestamp.
- New() returns an explicit nil Provider on error. Returning the typed
pointer wrapped a nil *AgeVault in a non-nil interface, so the usual
`if provider != nil` guard passed and the next call panicked.
- Every exported constructor calls Config.Validate(); previously only
vault.New() did, so the direct constructors bypassed validation entirely.
- StoragePath now goes through expandPath/validateSecurePath like key and
identity paths always did. The README's own WithAESPath("~/secrets.vault")
created a literal "~" directory in the working tree.
- expandPath: ".config/x" no longer loses its leading dot, "../x" no longer
silently drops the parent reference, "~user/x" no longer becomes
"<home>user/x", and "$HOME/vault" resolves instead of looking up a
variable named "HOME/vault". An unset variable is an error rather than
expanding to "/vault".
- validateSecurePath compares path elements instead of substrings, so
"my..backup" is allowed and "/etcetera" is no longer caught by "/etc".
- ValidateSecretKey rejects a leading dash (flag injection into backend
CLIs) and "."/".." (path elements), is applied at every provider entry
point rather than only SetSecret, and compiles its regex once instead of
on every call.
- KeyResolver propagates key-file failures. `err == nil &&` discarded typos
and permission errors, surfacing them as a generic "no encryption keys
found" or silently falling through to another source.
- SecretValue.String() takes a value receiver so a dereferenced copy is
masked too; Zero() drops its forced per-call runtime.GC() and the
cargo-cult random prefill, and now documents what it cannot guarantee.
- DefaultVaultKeyEnv is a const; as a var any dependency could repoint every
default key lookup in the process.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtuDuGkSqTfAXSepf8b51p
…aring
- WithLocalPath switched on the provider type at the moment it ran, making
it a silent no-op unless WithProvider was passed first; the failure then
surfaced as a confusing "storage path is required". The path is now
resolved after every option has been applied.
- The vault format version was written on every save but never read back, so
a file from a future format would have been parsed as though it were the
current one. Loads now reject a version newer than the build understands.
- AESState.Version carried a `json:` tag on a struct marshaled with yaml.v3,
which ignores json tags entirely; it keyed correctly only by lowercasing
coincidence. Corrected to `yaml:`.
- Close() now clears the secrets map rather than only dropping the pointer to
it, and says plainly in a comment why that is the limit of what it can do.
- README: the storage path is a directory, but every example named it like a
file ("~/secrets.vault"), which -- now that ~ expands correctly -- would
create a directory by that name. Also updated for the Metadata() signature.
Note on secret lifetime, deliberately not "fixed": SecureBytes/Zero() cannot
protect the provider state maps, which hold immutable Go strings whose bytes
are neither addressable nor zeroable, and PlainTextString() hands out more
un-zeroable copies by design. Converting the maps to []byte would change the
on-disk format of every local vault for a benefit that PlainTextString gives
away again immediately. The limitation is now documented on the types rather
than papered over with machinery that does not deliver.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01HtuDuGkSqTfAXSepf8b51p
…ested Three of the four example configurations were rejected outright by the new validation because they interpolated the secret into a shell command, and the fourth (pass) had never worked: its stdin-based set depended on the input-template bug. Nothing in the repo loaded these files, so all of it went unnoticed. Rewritten so the secret reaches each backend over stdin, and to fix correctness problems independent of the injection issue: - bitwarden: `bw get item <name>` matches by substring and errors on multiple hits, so with keys "api" and "api_key" the old get, exists and delete were all broken. Replaced with an exact-name jq select over `list items`. The old set built JSON by string-interpolating the value, so any quote or backslash in a secret produced malformed JSON; jq -Rs now builds it from stdin. Deletes are --permanent rather than leaving the secret in the trash, and every call is --nointeraction as there is no TTY. - 1password: set only ever ran `item create`, so it failed whenever the item already existed; it now probes and edits or creates. `--` before the key closes flag injection. - aws-ssm: `describe-parameters` listed the entire account and needed account-wide IAM; replaced with `get-parameters-by-path`, whose JSON output also lets the CLI's paginator return every page. The config also specified "listSeparator", which is not the field name -- the struct tag is "separator" -- so the tab separator was silently never applied while the output was tab-delimited. `--value file:///dev/stdin` keeps the secret out of argv. - pass: exists ran `pass show`, decrypting (and possibly triggering a pinentry prompt) just to answer a boolean; it is now `test -f`, a builtin in the interpreter. list no longer needs `tree`, which `pass ls` shells out to and which is frequently not installed. The `GPG_TTY: "$(tty)"` entry never did anything: expandEnv uses os.ExpandEnv, not a shell, so the value stayed the literal string. All four now declare not_found_pattern, so an expired session or a permissions error is no longer indistinguishable from "does not exist". Added TestShippedExampleProvidersAreUsable, which loads every config, renders all six operations against provider-shaped sample output, checks the list templates actually parse it, and asserts the secret reaches stdin and never the command string. It caught two broken output templates in this commit before they shipped. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HtuDuGkSqTfAXSepf8b51p
Each breaking change replaces behaviour that failed silently, so the upgrade table says what was wrong rather than only what changed. Also corrects the external provider example, which showed the secret being interpolated into a command -- the exact shape that is now rejected. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HtuDuGkSqTfAXSepf8b51p
jahvon
added a commit
to flowexec/flow
that referenced
this pull request
Jul 26, 2026
…418) # Summary Picks up [vault v0.3.0](flowexec/vault#18), a security and correctness release. In the library, secrets no longer travel through a shell command, empty vault files are no longer silently reinitialized and overwritten, writes are serialized across processes, and closed vaults report an error instead of panicking. The surface it breaks in flow is small — three call sites — but one compatibility problem was **not** a compile error and is the more important half of this PR. **Notable Changes** - `demoVaultProvider` implements the new `Metadata() (Metadata, error)`. - `vaultFromName` handles the new error, and reports a metadata failure as a `metadataError` field rather than failing the whole view. For an external vault that failure means the backend CLI is missing or the session has expired — precisely when a user wants to inspect the vault's configuration. It also called `Metadata()` **twice**, which for an external vault meant running the backend command twice per view. - `ValidateIdentifier` now rejects a leading dash or underscore. flow's rule was a *superset* of the library's new `ValidateVaultID`, so flow would accept a name like `-myvault` or `_myvault` that the library then refuses — failing at creation with a less specific message, and leaving any vault already created under such a name **unreachable** after upgrade. A new test asserts the subset property directly, because the two rules live in separate repositories and will otherwise drift apart silently.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A security and correctness pass over the whole package, prompted by an audit done while
scoping curated external-provider presets. Building presets on top of the library as it
stood would have meant shipping a secrets feature on a foundation that could corrupt and
leak the secrets it held.
Every fix here replaces behaviour that failed silently — no error, no log, no crash.
That is the common thread, and it is why none of it had been noticed.
Three findings were verified empirically rather than argued from reading the code:
Command injection on every
SetSecret.SetSecretrendered the plaintext into acommand string that is then parsed and executed by a shell, and the template engine does
no quoting. This needed no attacker — an ordinary password was enough.
p@$$w0rdhad$$expanded to the process ID and a different secret was stored, silently;correct horse batteryword-split and storedcorrect; anything containing;ranarbitrary commands. A set/get round trip through the real shell now survives
$(id), backticks, quotes, globs, spaces and newlines byte-exact, and an injectionpayload provably does not execute.
Concurrent writes lost secrets.
load()ran once at construction and every saverewrote the whole file from that snapshot, with nothing synchronizing writers across
processes. Measured directly: with the new lock disabled, 8 concurrent writes collapse
to 1 surviving secret.
scrypt ran at N=2²⁰ (~1 GiB and seconds per derivation). Concurrent derivations were
a trivial local denial of service. The crypto test suite drops from 35s to 0.7s as a
direct result of correcting it.
Notable Changes
Secret handling
{{value}}or
{{password}}in any command template are rejected at load rather than silentlycorrupting values.
SetSecret,DeleteSecretandListSecretseach guarded on their own
InputTemplatebut renderedGet's,Metadatagated onList's, and the value was never exposed to input templates at all — so the shippedpassexample's stdin-based set has never functioned.execute()no longer merges stderr into stdout on success, which was concatenatingbackend warnings (
gpg: WARNING: ...) onto the secret value itself.Data loss
immediately overwrote it — every secret gone, success reported. Now
ErrVaultCorrupt,file left untouched.
RWMutexnever spanned two providers, let alone two processes.<path>.tmpname written withos.WriteFile, so concurrentsavers truncated each other and a planted symlink was followed with
0600silently notapplied. Now
os.CreateTemp+O_EXCL, explicit chmod, fsync before rename, dir fsync after.Crypto
DeriveKeytestedsalt == nil, but[]byte("")from a variable string is non-nil, sothe natural "generate me a salt" call ran scrypt unsalted — a deterministic,
precomputable KEK identical for every user with the same passphrase.
aes.NewCipheraccepts 16/24/32 bytes, so a short key silently downgraded an "AES256"vault to AES-128/192. Now exactly 32 bytes.
change an existing key.
Crashes, validation, lifecycle
Close().directory.
filepath.Cleanresolves traversal rather than sanitizing it:Clean("vault-../../../tmp/evil.enc")is"../tmp/evil.enc"."%s-secret-%s"let vaulta/keyb-secret-ccollide with vault
a-secret-b/keyc.New()returns a genuinely nilProvideron error rather than a nil pointer in anon-nil interface.
AgeVaultrefuses to create a vault, or remove a recipient, that would leave no recipientmatching your own identity — one mistyped public key previously produced a permanently
unreadable vault.
expandEnvno longer mutates the shared config map under a read lock (an unrecoverableconcurrent map writesfault, guaranteed to recur with thepassexample).HasSecretno longer re-entersRLock, which deadlocks if a writer arrives in between.Examples
never worked. All rewritten, plus correctness fixes independent of the injection issue —
bw get itemmatches by substring (so keysapiandapi_keybroke each other), AWSused a
listSeparatorkey that is not the field name (separator) so it was neverapplied, and
pass'sexistsdecrypted via GPG just to answer a boolean.TestShippedExampleProvidersAreUsableloads every config and renders all sixoperations. Nothing exercised these files before, which is how they drifted; it caught two
broken output templates during this work.
Tests
cmdandinputand returned the first value in a map, sono test could observe a wrongly-rendered template — precisely why these bugs shipped.
Replaced with a capturing mock. 126 tests pass under
-race.Compatibility
Breaking changes are documented in a new "Upgrading to v0.3.0" table in the README. Local
vault files written by earlier versions are read without migration.
Downstream cost was measured, not estimated: pointing
flowexec/flowat this branch breaksexactly 3 call sites, all from the
Metadata()signature —internal/vault/demo.go:64and
internal/io/vault/view.go:226-227.One audit finding was deliberately not fixed, and is documented rather than papered
over:
SecureBytes/Zero()cannot protect the provider state maps, which hold immutable Gostrings whose bytes are neither addressable nor zeroable, and
PlainTextString()hands outmore un-zeroable copies by design. Converting the maps to
[]bytewould change the on-diskformat of every local vault to buy protection the next API call gives away again.
Change Type
🤖 Generated with Claude Code
https://claude.ai/code/session_01HtuDuGkSqTfAXSepf8b51p