Skip to content

Troubleshooting

Tommi Grönlund edited this page Oct 23, 2026 · 4 revisions

Documentation for the image tag 1.0.0-rc1. See Versions for the documentation of a release.

Troubleshooting

Start with these three checks. They answer most questions.

docker ps                          # is every container "healthy"?
docker logs qertum --tail 100      # single container; multi: docker logs qertum-apiservice
qca doctor                         # read-only checks of API, identity, trust, revocation, enrollment

Log lines start with a short trace id: [HH:mm:ss LVL <trace id>] message. Every API response carries the same id in an X-Correlation-Id header, and error messages from the web UI and qca print it as correlation id: .... Search the logs for that id to find the matching entries.

Startup

The container is "unhealthy" or restarts forever

In the single container a service that exits is restarted every second, forever, so a crash loop shows as unhealthy, not as a stopped container. Read docker logs for the first error of the service that keeps failing. The usual causes are below.

Multi container: a container exits with code 64

Exit 64 is a configuration refusal, and the log says which:

  • QERTUM_HOST is localhost for apiservice or webfrontend. Use a real host name or IP (host.docker.internal on Docker Desktop).
  • estservice has no certificate. It refuses to start without both QERTUM_EST_TLS_CERT and QERTUM_EST_TLS_KEY, and never falls back to plain HTTP.
  • SERVICE is missing or unknown (only if you edited the compose file).

The web UI returns 500 and the log mentions IDX20803

The web UI could not fetch the identity provider's metadata from the address in QERTUM_HOST. Inside a container, localhost is the container itself. Set QERTUM_HOST to an address that the containers and your browser can both reach, and check that the identity port is published.

Services fail to write, "permission denied" on /data

The container makes /data writable for UID 1654 on first start. If the storage cannot be re-owned (some NFS exports squash root), fix the export, or run the container with --user 1654:1654 against a volume that user can write.

EST container will not read its key

The key must be readable by UID 1654: chown 1654 tls.key && chmod 400 tls.key.

Everything is gone after I re-created the container

/data was not mounted on a volume, so it lived inside the container. Restore your latest backup onto a container that mounts /data (Backup and Restore). From now on mount -v qertum-data:/data or a host path.

I remapped a port and sign-in or ACME broke

Qertum advertises its port numbers in the OIDC authority, redirect URIs and ACME directory. A published host port must equal the container port. If you must change one, set the matching WEB_PORT, IDENTITY_PORT, API_PORT, ACME_PORT or EST_PORT to the published value.

TPM: Settings does not offer "Local TPM", or the API reports the TPM unavailable

The service processes need the TPM device and its group. Pass both (devices: and group_add: with the numeric group of /dev/tpmrm0, which stat -c %g /dev/tpmrm0 prints), for every container that builds the key ring. Images built before 2026-10-20 dropped the group when the container dropped from root, so the TPM was never usable; pull the image again if yours is older. Check with curl <api>/v1/secret-storage/status (needs a token): tpmAvailable must be true.

TPM: services stop with a tpm2_* error

'tpm2_load ...' exited with code ... means the TPM that sealed the key is cleared, replaced, or not passed through. Check the device and group are passed to the container. If the TPM is gone, see Backup and Restore: the sealed data is not recoverable, and an older backup from before TPM storage is the way back.

Signing in

I cannot sign in as admin

  • The seed password is only read while the identity store is empty. Changing IdentityService__SeedAdminPassword later does nothing.
  • If the password is still the default password, the first login can only change the password or sign out, and no token is issued until it is changed. Do that first.
  • To start over on a lab instance, stop the container and remove users.json from /data/identity; the next start seeds admin again. Only do this when you accept losing the other users.

Sign-in fails with invalid_redirect_uri or a redirect mismatch

The web client's registered redirect URI does not match the address you browse to. Common causes: you put a TLS proxy in front but did not set Services__webfrontend__https__0, or QERTUM_HOST or WEB_PORT differs from the address in your browser. See TLS and Reverse Proxy.

The API answers 401 to valid tokens

Identity__Authority must equal the token's iss claim exactly, including scheme, host, port and any path (Keycloak: https://host/realms/<realm>). Also check aud contains qertum-api (or your Identity__Audience), and that clocks are in sync across hosts. Revoked tokens stay valid until they expire.

A sensitive action returns 401 with insufficient_user_authentication

Step-up is on (Identity__StepUp__Enabled=true) and the token lacks a recent second factor. The web UI sends you back to the identity provider; with qca, run the qca auth login --acr-values ... --max-age ... it prints. Machine tokens are refused unless the identity provider stamps acr and auth_time on them. A 403 instead means your role lacks the permission.

An operation is blocked, "needs approval"

Two-person approval is on. A second administrator must approve, and with only one administrator it cannot proceed. Switch it off (no approval needed) or add an administrator.

ACME and EST

ACME client reports "URL mismatch"

ACME is behind a TLS-terminating proxy. Remove it from the proxy and let clients reach port 8083 directly (or let the ACME container terminate TLS itself in multi container).

http-01 challenges never validate

The challenge-network allowlist starts empty, and empty means no http-01 challenge validates. Add the networks the CA may connect to under Settings. dns-01 does not use the list.

Orders and requests sit in Pending

That is the default. Approve them in the UI (qca requests approve <id>) or turn on ACME/EST auto-approval in Settings. qca acme orders list --failing shows stuck ACME orders and why.

ACME client cannot connect, TLS errors

The client does not trust your root certificate. Install the root on the client before using the CA (TLS and Reverse Proxy).

EST /simpleenroll answers 401 over HTTP

Est:RequireHttps is on by default. Use HTTPS to port 8084. Disable the setting only when TLS terminates at a proxy you trust.

EST /.well-known/est/cacerts and /crl answer 503

The CA has not been onboarded yet. Create or import a root in the web UI.

Issuing and revoking

A request stays in Approved or Processing

The queue consumer signs approved requests. Check it is running (docker logs qertum-queueconsumer, or the single container's log) and that the ca.queue.depth metric is not stuck. After a restart it resumes.

A revoked certificate still validates

Relying parties use a cached CRL until it expires. The CRL is at /crl on the API port. Check its age; the queue consumer refreshes it on a schedule.

429 Too Many Requests

The API rate limiter on /v1/certificate-requests rejected the call. Slow the client down or raise RateLimiting__PermitLimit.

Webhooks are not delivered to my internal receiver

Private, loopback and link-local addresses are blocked unless an administrator adds an exact rule: Webhooks page, or qca webhooks destinations add --url http://receiver.lan:8088 --ip 192.168.1.20.

Backup and restore

restore-test.sh exits 2

It could not test: Docker is unavailable, there is no manifest next to the archive (backups from before manifests existed cannot be verified), or the archive came from another Qertum version than the running image. Use the image that made the backup (--image <ref>).

restore-test.sh exits 1

The output names the differing path or the failed check (manifest, archive, key-ring, ...). Do not restore from that archive unless you have no other.

Local development

Symptom Fix
dotnet run for the AppHost fails asking for Aspire Install the Aspire CLI 13.6 or newer: dotnet tool install --global aspire.cli --version 13.6.0
Integration tests (IT-*) fail Start Docker; they need a daemon. These are not regressions when Docker was down
Stale state after changes scripts/reset-dev-environment.sh
Port already in use Stop the other process; the Aspire dashboard lists the URL each service got

Still stuck?

Collect the docker logs output around the failure, the qca doctor output, the correlation id from the error, and the Qertum release you run, then contact whoever supplies your Qertum build. Do not include private keys, tokens or passwords.

Clone this wiki locally