Repository navigation
ADR Centralize secret storage and harden admin credentials
Date: 2026-10-12
Last revised: 2026-11-13 (see Revision log)
Related issue: https://github.com/device-management-toolkit/console/issues/1067
Related architecture: ADR-Centralize-secret-storage-and-harden-admin-credentials-Architecture
Standalone (non-OAuth2) Console deployments stored the administrator
username, password, and JWT signing key in plaintext in config.yml or the
corresponding environment variables, and compared the submitted login
password with a plain ==.
This ADR covers, for deployments that do not use OAuth2/OIDC:
- local first-run bootstrap of the administrator credentials
- secure storage of those credentials in the OS keyring
- password hashing
- selection of the secret backend (
configorkeyring) and the plan for changing its default - a credential reset and recovery workflow
The JWT signing key is explicitly out of scope. This ADR does not change its
existing configuration/environment handling or store it in the dmt-console
keyring service; its storage and lifecycle remain as they are today.
OAuth2/OIDC deployments are unaffected: when AUTH_CLIENT_ID is configured,
Console never loads or stores standalone admin credentials.
Existing deployments rely on credentials in config.yml or the environment,
and some (for example Docker Compose) have no usable OS keyring. The design
therefore adds the keyring as an opt-in backend in v1.x and does not change
existing behavior until a coordinated major release.
Decision: OAuth2/OIDC takes absolute precedence. When AUTH_CLIENT_ID is
non-empty, Console uses the OAuth2/OIDC path and ignores all standalone
credential sources and settings (keyring, config, AUTH_SECRET_BACKEND).
Rationale: Prevents silent override of configured identity providers and keeps OAuth2 deployments deterministic.
Decision: The OS keyring is the secure credential store for standalone
Console. When the keyring backend is selected, the administrator credentials
are stored only in the OS keyring (never in config.yml or the environment),
via github.com/zalando/go-keyring, as follows:
-
Service name:
dmt-console. This is separate from the existing encryption-key service (device-management-toolkit), which is unchanged and not reused. -
Entries: two entries,
admin-username(the username) andadmin-password-hash(the PBKDF2 hash from Decision 3, which embeds its own salt and iteration count). No plaintext password and no other secret, such as a signing key, is stored. -
Startup states:
- Both entries present: Console loads them and proceeds to login verification.
- Both entries absent: first-run setup runs (Decisions 7 and 8).
- Only one entry present (partial or corrupted): Console fails at startup with recovery guidance and never overwrites the remaining entry. The operator resolves it by deleting the entries (Decision 6).
- Keyring failure: if the keyring is missing, locked, access is denied, or no supported backend exists, Console fails with an actionable error. It never falls back to reading or writing plaintext credentials (Decision 5).
- Login: the submitted password is verified against the stored hash. The stored hash and the plaintext password are never returned, logged, or displayed.
Rationale: The OS keyring is scoped to the OS user, is OS-native, and
keeps credentials out of config.yml, separating secret storage from
configuration files. Console already uses a keyring for its encryption key, so
this adds no new class of dependency (confirm that go-keyring is already a
dependency before implementation). Failing closed on partial or unavailable
state avoids silently creating or overwriting administrator credentials. The
username is stored in the keyring too because it is part of the credential
set and then shares the same access controls.
Limitation: The keyring depends on the user session. On Linux the Secret
Service API needs a D-Bus session and an unlocked keyring, which headless
servers, services, and containers often lack. This is why config remains the
v1.x default (Decision 4). Entries are local to the host and OS user, so they
are not shared across replicas.
Supported keyrings:
- Windows: Credential Manager
- macOS: Keychain
- Linux: Secret Service API (GNOME Keyring, KWallet)
Entry names, partial-state handling, and failure behavior are specified in the ADR-Centralize-secret-storage-and-harden-admin-credentials-Architecture.
Decision: Passwords are hashed with PBKDF2-HMAC-SHA256 (600,000 iterations,
16-byte salt, 32-byte hash) before storage in the keyring.
The stored representation uses the $pbkdf2-sha256$ prefix to identify the
algorithm, with the salt and hash encoded using standard padded Base64.
Every new password hash, including initial setup, password changes, and setup
after recovery, must use a freshly generated 16-byte salt from crypto/rand.
Previously stored salts must never be reused for a new hash, even when the
password is unchanged. If random generation fails, the operation fails without
replacing existing credentials.
Rationale: The Go standard library provides crypto/pbkdf2 (Go 1.24 or
later; confirm against go.mod) so no additional cryptographic dependency is
needed. PBKDF2-HMAC-SHA256 is NIST-approved and 600,000 iterations is the
current OWASP minimum for it. It provides one-way, deliberately slow hashing.
The stored hash is never the plaintext password.
Trade-off: PBKDF2 is not memory-hard, so it resists GPU cracking less well than bcrypt or Argon2. Accepted to avoid a new dependency; the hash format is versioned so the algorithm can change later.
Decision: Console supports AUTH_SECRET_BACKEND=config|keyring:
-
config(default throughout v1.x): credentials continue to come fromconfig.yml/environment, as today. -
keyring: opt-in. Credentials come from the OS keyring only (Decision 5).
The default changes to keyring only in a future major release (for example
Console v2), after the keyring backend and the UI first-run setup are fully
implemented. That change is coordinated with the installer release and
accompanied by advance notice and migration guidance. Timing is decided in
Engineering Sync.
Rationale:
- Existing customers keep the behavior they rely on, with no forced change or credential migration during v1.x.
- Docker Compose and other managed deployments with no usable OS keyring continue to work.
- Customers can opt into the keyring while the backend and UI setup are completed.
The config backend stores credentials in plaintext. This is the existing
behavior and is not made safer by this ADR. Whether config mode logs a
reduced-security warning is an implementation detail; it must not be exposed
through unauthenticated endpoints.
Decision: When AUTH_SECRET_BACKEND=keyring, Console checks the keyring
only. It does not read, import, or use admin credentials from config.yml,
.env, or environment variables, does not prompt to migrate them, and does not
modify or scrub config.yml. There is no fallback between backends in either
direction.
If the keyring is missing, locked, denied, or unsupported, Console fails with an actionable error. If the required keyring entries are absent, first-run setup runs (Decisions 7 and 8).
Rationale: Removing fallback and migration logic reduces code paths and
makes behavior predictable. Existing credentials are left untouched, and no
operation modifies customer configuration without the customer's action. This
supersedes the earlier design that migrated legacy plaintext values after
operator confirmation (-confirm-legacy-migration), which is not implemented.
Consequence: An operator who opts into the keyring must set up credentials
through first-run setup and should remove the old plaintext values from
config.yml themselves. Console does not do this.
Decision: To reset credentials and re-run first-run bootstrap, operators must
manually delete both keyring entries (admin-username and admin-password-hash)
from the OS keyring using OS-native tools.
For Phase 2 authenticated standalone password changes, Console must revoke server-side all previously issued JWTs authenticated with the old credentials, including the current session. After a successful change, those tokens must be rejected even if unexpired, and the operator must sign in with the new password. Console must not report success until the password update and revocation are effective. If either fails, report the failure and fail closed; do not silently leave old tokens usable with the changed password. JWT signing key storage and OAuth2/OIDC token lifecycle remain unchanged.
Rationale: Avoids Console-managed recovery tooling (such as console -clean),
which would add security responsibilities; it can be revisited if customers ask
for it. Phase 2 UI provides authenticated password change for operators who
know the current password. Exact per-OS commands are in the Architecture
document.
Decision: Phase 1 (before Phase 2 UI) uses a simple CLI-based setup for
customers who opt into keyring: on first startup with no keyring entries,
Console prompts the operator to enter a username and password via interactive
CLI input. Console validates the input, hashes the password with PBKDF2, and
stores both in the keyring. This temporary path is documented for customers
who opt in before the UI is available. It is removed in Phase 2 (Decision 8).
Rationale: Gets first-run initialization working without waiting for the Phase 2 UI. Minimal implementation surface. The operator chooses their own credentials.
Decision: Phase 2 first-run setup uses a single bootstrap service shared by Console in headless mode and Console with the UI, not two independent code paths. Entering the first-time username and password at the command line (Decision 7) is removed in Phase 2; first-run setup is no longer an interactive prompt on startup.
-
Shared service and API: Both modes use the same loopback setup REST API,
a thin wrapper over the bootstrap service. Console with the UI checks setup
status and submits the chosen username and password over HTTPS from the setup
form, then returns to sign-in after successful setup. In headless mode there
is no UI, so the operator, on the Console host, calls the same setup API
directly (for example with
curl). Only the Console backend process reaches the keyring; neither the UI nor the API client does. -
Temporary UI bootstrap token: When no credentials are present, Console
prints the complete local setup URI to its CLI, with a short-lived, single-use
token in the URL fragment, for example
https://localhost:<port>/setup#token=<setup-token>. When Console opens a browser locally, it opens this same URI; otherwise the operator opens the printed URI. The setup page reads the token into memory and immediately removes the fragment usinghistory.replaceState. The operator enters only the new username, password, and password confirmation. The UI submits the credentials and token together in the HTTPS request body. The backend validates the token and credentials before storing the username and password hash. There is no separate token-to-cookie step. On success, the token is invalidated, the UI shows confirmation, clears setup state, and returns to sign-in with the user logged out. On failure, the UI shows an error without reporting setup success. The token is not an authenticated user session and is never written to application logs or included in URL query parameters. Headless setup continues to use the loopback API directly. -
Credential handling in headless mode: The password must not be passed on
the
curlcommand line, where it is visible in the process list and shell history. Document supplying the request body from a file or standard input (for examplecurl --data @-) and never logging the request body. -
Loopback-only exposure: The setup endpoint is served only on a loopback
listener (
127.0.0.1,::1), never on the public listener and never through a reverse proxy. The restriction comes from which listener serves the route, not from client IP or forwarded headers, because a reverse proxy on the same host also connects from loopback. - Serialization: The service takes a lock that works across processes, rechecks the keyring state under that lock before writing, and reports success only after the write is durable. Keyring errors are reported as setup failures; there is no plaintext fallback.
-
Error contract: If a non-loopback listener is enabled, setup requests
reaching it are rejected with
403 SETUP_LOCAL_ONLY(the route is not served there). A local setup request after initialization receives409 SETUP_ALREADY_COMPLETED. - After initialization: The sign-in page replaces setup. Setup is reachable again only after an authorized credential reset or manual keyring deletion (Decision 6).
- Partial keyring state: Same as Decision 2. Console does not overwrite the remaining entry and gives recovery guidance.
- API specification: The setup API is specified in OpenAPI.
Rationale: A single implementation avoids code duplication and keeps behavior consistent between headless mode and the UI, and giving headless deployments the same setup path as the UI without an interactive prompt. Binding to loopback prevents remote unauthenticated users from claiming the first admin account. The lock and recheck prevent concurrent setup from racing, and a durable-write check avoids reporting success for credentials that were not stored.
Security limitation: The CLI-displayed token authorizes initial local browser setup. It does not authenticate a local user or process that can access the CLI output and loopback listener. The complete setup URI is a secret: terminal capture, browser launch arguments, browser history, or extensions may expose it. The fragment avoids sending the token in the initial HTTP request; it does not eliminate these local exposure risks.
Open point: Whether the username and hash become a single versioned keyring record in Phase 2 remains to be validated.
Sequence diagrams (before and after initialization) and the remote-browser cases are in the Architecture document.
Phases
- Existing customers see no change in v1.x unless they opt into
AUTH_SECRET_BACKEND=keyring. - No plaintext-to-keyring migration exists. Plaintext values in
config.ymlare never read, copied, or scrubbed by keyring mode. - The stored hash format is
$pbkdf2-sha256$<iterations>$<base64(salt)>$<base64(hash)>. - Phase 1 first-run setup prompts the operator for a username and password on the command line on first startup (no auto-generation).
- Phase 2 removes the command-line prompt. First-run setup is reachable
through a loopback-only REST API, used by the UI or, in headless mode, called
directly by the operator (for example with
curl). - Username and password hash are the only credentials stored in the keyring.
- Local keyring entries are not shared across replicas.
- The default backend change in v2 is a breaking change that needs advance notice and migration guidance.
- In
configmode (the v1.x default), credentials remain in plaintext.
- Keyring as the default in v1.x with automatic or confirmed migration. Rejected: forces a change on existing customers and breaks Docker Compose.
- bcrypt or Argon2 for password hashing. Stronger against GPU cracking but adds a dependency; PBKDF2 was chosen (Decision 3).
-
Console-managed reset (
console -clean). Rejected for now (Decision 6). -
Dedicated secret-file provider for Docker/Kubernetes. Deferred;
configremains the backend for these deployments until it ships.
Phase 1 (planned): Backend credential storage, PBKDF2 generation and verification, command-line first-run setup (temporary), and standalone login verification. No browser-based credential screens.
Phase 2 (planned, design specified): First-run setup form, standalone sign-in form, authenticated password-change form. Uses the Phase 1 keyring and PBKDF2 backend, and replaces the Phase 1 command-line first-run prompt with the bootstrap service (Decision 8). Detailed design is in the Architecture document.
These are not decided by this ADR and need a decision before or during implementation:
- Phase 1 first-run setup when unattended. The Phase 1 command-line prompt does not work for Console running as a service with no terminal. Define the behavior (for example fail with guidance) until Phase 2 provides the setup API.
- Single versioned keyring record. Whether Phase 2 replaces the two entries with one record to avoid partial-write states.
- Support secure storage and retrieval of Console credentials for deployments without OAuth2. Opt-in OS keyring backend, designed. Planned (Phase 1).
-
Provide a way to disable plaintext storage of sensitive credentials in
config/config.yml. WithAUTH_SECRET_BACKEND=keyring, Console does not read admin credentials fromconfig.yml; operators remove the old values themselves. Plaintext remains the v1.x default. Partially addressed. - Provide a customer-usable workflow for setting, updating, and removing stored credentials. Phase 2 first-run setup and authenticated password change are designed. Phase 1 recovery requires manual OS keyring deletion. The UI workflow is a design target; implementation is future work. Partially addressed.
-
Support the deployment environments currently supported by Console.
Windows, macOS, and Linux via OS keyring are covered. Docker/Kubernetes
managed deployments keep using the
configbackend; a dedicated secret-file provider is deferred. Partially addressed.
- 2026-10-12: Initial ADR.
- 2026-10-21: Added
AUTH_SECRET_BACKEND, confirmed legacy migration, and the Phase 2 bootstrap service design. - 2026-10-26: Dropped
console -clean; manual keyring deletion is the recovery path. - 2026-10-27: Accepted Ganesh's 10/26 proposal:
configis the v1.x default, keyring mode is isolated, and the legacy migration (Decision 5) was removed. Decision numbering is unchanged; the old "Legacy plaintext migration" decision is replaced by "Keyring mode is isolated". - 2026-11-13: Added a complete local setup URI printed to the CLI, carrying a short-lived token in its fragment and used when opening the local browser. The UI submits the token with the new credentials. Success returns the user to sign-in, logged out; failure shows an error. Loopback-only exposure remains required.
- 2026-11-13: Adopted standard padded Base64 for the stored salt and hash fields in response to Divneil's review feedback.
- 2026-11-13: Required a fresh
crypto/randsalt for every new password hash, with no reuse and explicit failure on random-generation errors. - 2026-11-13: Required server-side revocation of existing standalone JWTs on authenticated password changes.
- Migration consent (Ganesh 10/14 #1). Superseded. Migration was first gated on explicit confirmation, then removed entirely by the 10/26 proposal (Decision 5). Nothing is migrated, so nothing is migrated without consent.
- UI setup delivered with the backend (Ganesh 10/14 #2). Not adopted. Phase 2 UI remains deferred; the command-line path in Decision 7 covers opt-in customers until then.
-
Docker Compose / managed deployments (Ganesh 10/14 #3). Adopted.
AUTH_SECRET_BACKENDlets managed deployments keep usingconfig; with the 10/26 change that is also the default. -
Drop
console -clean(Ganesh 10/14 #4). Adopted (Decision 6). - Headless vs. UI bootstrap detail (Sudhir 10/15 #1). Adopted (Decision 8). The single-versioned-record refinement remains open (see Open items); browser setup reads the token from the CLI-displayed setup URI and submits it with the new credentials as specified in Decision 8.
- Default backend and keyring isolation (Ganesh 10/26). Adopted (Decisions 4 and 5). Point 3, coordination of the default change with v2 planning and the installer, is an Engineering Sync action.
-
Give the keyring-mode operator guidance clear docs. Document that
keyring mode ignores
config.ymlcredentials and that the operator must remove old plaintext values. - Add an explicit regression test for the setup endpoint. A request that reaches the public listener, even if path-identical to the setup endpoint, must not be served. This is a security property, not just a routing detail.
-
Do not expose backend mode on unauthenticated endpoints. If surfacing
reduced-security (
config) mode, do it in an authenticated admin view.
-
How will existing binary deployments be migrated?
Will credentials be moved from
config.ymlor the environment into the OS keyring only with the user's consent? Prompting or migrating credentials without the user's prior consent does not feel like the right design. What happens if they decline?
Resolved 10/27/2026: Superseded by the 10/26 feedback. There is no migration. See Decision 5.
- Could the UI setup flow be delivered with the backend changes? The UI is the primary way customers expect to set up and use Console, so it could let them create their credentials during first-run setup. Credentials should be printed to the CLI only for headless deployments, where no UI is available. Could the UI flow be included with the backend changes rather than deferred to a later phase?
Not adopted (10/21/2026): Phase 2 UI remains deferred. The Phase 2 UI design is specified in the Architecture document.
- How will standalone Console be supported in Docker Compose? The ADR says Console will fail to start if the OS keyring is unavailable, while Docker/Kubernetes secret-file support is deferred. Does this mean Docker Compose users will be unable to run Console with standalone authentication under this design? If not, how will Console access and persist their admin credentials? We have a customer using this deployment, and I want to make sure this change does not disrupt them.
Resolved 10/27/2026:
AUTH_SECRET_BACKEND=configis the default throughout v1.x (Decision 4), so Docker Compose deployments are unaffected.
-
In my opinion, the
-cleanoption is not needed at this point. Users can delete credentials directly from the OS secret store. Adding a Console-managed recovery path gives us additional security responsibilities; we can revisit it if customers ask for it.
Adopted (10/26/2026):
console -cleanis not implemented. Operators manually delete both keyring entries using OS-native tools.
Keep AUTH_SECRET_BACKEND=config as the default throughout v1.x, and change the default only after the keyring backend and UI setup are fully implemented.
-
Keep
AUTH_SECRET_BACKEND=configas the default throughout v1.x. Customers can opt intokeyring. Make it the default only with a future major release, such as Console v2, coordinated with the installer release and accompanied by advance notice and migration guidance. -
Keep keyring mode isolated to the keyring. When
keyringis selected, Console checks the keyring only and does not read, import, or use credentials fromconfig.ymlor environment variables, nor prompt to migrate them. When required entries are absent, use first-run credential setup in the UI. Since UI setup is deferred to Phase 2 and keyring is not yet in production or preview, continue withconfigas the v1.x default and document the temporary Phase 1 setup path for customers who opt into keyring. - Coordinate the default change with major-version release planning. Use Engineering Sync to determine timing and coordinate Console with the installer, possibly together with discovery and the rpc-go download feature.
Adopted 10/27/2026. See Decisions 4 and 5.
Overarching comment: If Phase1 is temporary and code will be removed. Why do it? Phase2 should be revisited as mentioned below in Loopback-only exposure point.
Response 11/13/2026 - partially addressed: Decision 7 retains the temporary CLI prompt so opt-in customers can initialize credentials before the UI is available. The keyring and hashing backend are reused in Phase 2; only the prompt is removed. Whether to skip the temporary prompt remains unresolved.
- Pls define "headless mode" or share the reference to the definition of the same.
Addressed 11/13/2026: Decision 8 defines headless mode as Console running without the UI. The operator calls the loopback setup API from the Console host, supplying the request body from a file or standard input.
-
Loopback-only exposure: Why is this interface exposed for configuration. My understanding is that the username configuration is:
- at installation/configuration time before deployment
- using cli
- authenticated interface with older password
The loopback interface introduces a race on multi-user system. The user who accesses the URL first becomes the admin
Response 11/13/2026 - partially addressed: Decision 8 now requires the browser setup form to submit the new credentials together with a short-lived, single-use token carried in the fragment of the complete setup URI printed to the local CLI. Opening the setup page without a valid token cannot complete this browser flow. Setup is serialized and the credential state is rechecked under a cross-process lock. Success invalidates the token and returns the user to sign-in, logged out; failure shows an error. The headless path still needs an explicit authorization requirement before this concern can be considered fully resolved.
- JWTs Revocation: In case of password change, the in-flight JWTs have to be revoked because they were issued in response to authentication using username/password
Adopted 11/13/2026: Decision 6 requires server-side revocation of all previously issued standalone JWTs on a successful password change, including the current session. Old tokens are rejected even if unexpired, and the operator must sign in again.
- PBKDF2 Salt: Pls state the salt will be a new crypto random number on every password storage. It cannot be reused.
Adopted 11/13/2026: Every new password hash uses a fresh 16-byte salt generated with
crypto/rand, including when the same password is set again. Salt generation failure aborts the operation without replacing credentials.
-
$pbkdf2$<iterations>$<hex salt>$<hex hash>could be-
$pbkdf2->$pbkdf2-sha256for now -
$<hex salt>$<hex hash>->$<base64 salt>$<base 64 hash>. base64 is generally accepted format
-
Adopted 11/13/2026: Decision 3 and the architecture specification use
$pbkdf2-sha256$<iterations>$<base64(salt)>$<base64(hash)>, with standard padded Base64 for both fields.