Howto Recover broken OIDC

How to recover from a broken OIDC config

Use this guide when OIDC login is completely broken and you need to restore access — either to fix the OIDC config or to fall back to local password login while you diagnose the problem.

---

Step 1 — Regain admin access via API key

The API key bypasses OIDC entirely. If you have it, you can manage providers without logging in through the UI.

# Test that API key access still works
curl -s http://bindery:8787/api/v1/auth/status \
  -H "X-Api-Key: <your-api-key>" | jq .

# List current OIDC providers
curl -s http://bindery:8787/api/v1/settings/auth/oidc/providers \
  -H "X-Api-Key: <your-api-key>" | jq .

If the API key works, skip to step 3.

API key lost? Retrieve it from the database directly:

# Docker
docker exec bindery sqlite3 /config/bindery.db \
  "SELECT value FROM settings WHERE key='auth.api_key';"

# Kubernetes
kubectl exec deploy/bindery -- sqlite3 /config/bindery.db \
  "SELECT value FROM settings WHERE key='auth.api_key';"

---

Step 2 — Fall back to local password login (if locked out)

If you can't reach the API and need UI access, temporarily switch auth mode away from proxy or disable OIDC-only enforcement:

# Switch to standard enabled mode (local password + OIDC both work)
curl -X PUT http://bindery:8787/api/v1/auth/mode \
  -H "X-Api-Key: <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{"mode": "enabled"}'

If you've forgotten the local admin password, recover it via the database:

# Delete users table row — triggers setup wizard on next page load
# Docker
docker exec bindery sqlite3 /config/bindery.db "DELETE FROM users;"

# Kubernetes
kubectl exec deploy/bindery -- sqlite3 /config/bindery.db "DELETE FROM users;"

Navigate to Bindery in the browser — you'll see the setup wizard. Create a new admin account. OIDC user accounts are separate rows and are not deleted by this.

---

Step 3 — Diagnose the broken provider

Enable debug logging to see exactly what's failing:

curl -X PUT http://bindery:8787/api/v1/system/loglevel \
  -H "X-Api-Key: <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{"level": "debug"}'

Then attempt an OIDC login and watch the logs:

# Docker
docker logs -f bindery 2>&1 | grep -i "oidc\|token\|issuer\|callback\|state"

# Kubernetes
kubectl logs -f deployment/bindery | grep -i "oidc\|token\|issuer\|callback\|state"

Common log signatures and their fixes:

Log message	Cause	Fix
oidc: failed to fetch discovery document	Bindery can't reach IdP	Check network/firewall; test curl <issuer>/.well-known/openid-configuration from inside the container
oidc: invalid_client	Client secret wrong	Rotate and re-enter the secret (see Howto-Rotate-OIDC-secrets)
oidc: state mismatch	Cookie lost in transit	Check proxy isn't stripping Set-Cookie headers; confirm login and callback are on the same domain
oidc: issuer mismatch	issuer field in provider config wrong	Match exactly to the iss claim in the token — visible in debug logs
oidc: token expired	System clock skew between Bindery and IdP	Sync clocks (chronyc / NTP); acceptable skew is typically ≤5 minutes

Reset log level when done:

curl -X PUT http://bindery:8787/api/v1/system/loglevel \
  -H "X-Api-Key: <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{"level": "info"}'

---

Step 4 — Fix or remove the broken provider

Fix in place

# Update the broken provider (replace full object)
curl -X PUT http://bindery:8787/api/v1/settings/auth/oidc/providers/<provider-id> \
  -H "X-Api-Key: <your-api-key>" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "<provider-id>",
    "name": "<name>",
    "issuer": "<corrected-issuer-url>",
    "client_id": "<client-id>",
    "client_secret": "<secret>",
    "scopes": "openid email profile"
  }'

Remove the broken provider

curl -X DELETE http://bindery:8787/api/v1/settings/auth/oidc/providers/<provider-id> \
  -H "X-Api-Key: <your-api-key>"

Removing a provider does not delete the user accounts that were provisioned through it. Those accounts remain in the database — users can still log in via local password if they have one set, or via a different OIDC provider if one is configured.

---

Step 5 — Re-enable and verify

After fixing:

1. Attempt a fresh OIDC login in a private window.
2. Confirm the full callback flow completes.
3. Check Settings → Users — the user account appears or was re-authenticated.

---

When this goes wrong

Symptom	Cause	Fix
API key doesn't work	API key was regenerated and you have the old one	Retrieve from DB (step 1)
Can't exec into container	No shell in distroless image	Use kubectl debug with an ephemeral container: kubectl debug -it deploy/bindery --image=alpine -- sh then mount the DB volume
Deleting users table broke something else	Other tables have FK references to users	Run PRAGMA foreign_key_check; after deletion to confirm. FK violations shouldn't occur since user rows are the referenced side, not the referencing side.
OIDC users locked out after local admin password reset	OIDC accounts are separate rows — they still exist	OIDC users log in via the OIDC button, not the password form. They were never affected by the password reset.

See also: Troubleshooting — OIDC | docs/troubleshooting-auth.md | Howto-Rotate-OIDC-secrets