[Design Discussion] Audit logging

[ThumulaPerera]

Related Feature Issue

#5591

Problem Summary

Admins can't rebuild what happened in a security incident or tell who was responsible, and end users can't review activity on their own identity. ThunderID has no rule for which events are audit-worthy, no common record shape, and nowhere to store or view audit records. This design defines audit events, stores them durably using the eventing framework in #5590, and shows them to admins and end users.

High-Level Approach

Terminology:

• Target: the one object or relationship the action touched, or the subject of a decision. For example, a user, a role assignment, or an application.
• Subject: the identity whose activity log the event belongs in. When an admin adds u1 to group g1, the target is the membership and the subject is u1.

The approach:

• A rule for what is audit-worthy. Each audit event records one actor's attempt to produce an outcome (section 1).
• One record shape for every audit event (section 2).
• Delivery through [Design Discussion] Generic eventing framework #5590. The audit event type is delivered at least once.
• A dedicated audit store that ignores duplicate deliveries and only allows appends (section 3).
• Tamper evidence. We can detect changed or deleted records, but not prevent them (section 4).
• APIs and Console views for admins (all events) and end users (their own events) (section 5).
• Retention. Records are deleted after a configurable period.

1. What counts as an audit event

Record each attempt by one actor to produce one of these outcomes:

Kind of outcome	Examples
Management state change	User created, role assigned to group, IdP updated
Runtime state change	Session created, consent revoked, token revoked
Security decision	Sign-in succeeded or denied, authorization denied, token issued
Disclosure of secrets	User data viewed

Rules

• One actor, one attempt, one event. When an outcome needs several actors, each actor's attempt is its own event.
• Emit when the attempt is confirmed: when the change is committed, or when the decision is final. Not when the attempt starts.
• Denied and failed attempts count, once the actor is known or has tried to sign in.

Every audited action is listed in a catalog. The catalog is used to validate events, to generate the docs list, and for Console labels.

2. Record shape

Field	Meaning	Example
Actor	Who made the attempt: user, agent, application, system or anonymous. Includes the user an agent acts for, if any.	admin a1
Action	What was attempted, from the catalog	user.create
Target	What it touched	user u1
Subject	Whose activity log it appears in	u1
Outcome	succeeded, denied (a security decision refused it) or failed (allowed, but didn't complete)	denied
Reason	Error or decision code, required unless succeeded	AUTH-1002
Time	When the attempt was confirmed
Context	Trace ID, IP, user agent, session ID, application, organization unit

The eventing framework fills in the event ID, time, trace ID and other request context from #5590. Records keep a copy of display names, so they stay readable after the target is deleted.

3. Store

• An audit subscriber receives audit events from [Design Discussion] Generic eventing framework #5590 and writes each one to the store.
• Duplicates are ignored using the event ID, since at-least-once delivery can deliver the same event twice.
• A separate auditdb datasource, because audit data grows fastest, has its own retention, and shouldn't share a DB with the sessions every sign-in depends on. Records are scoped by deployment ID, so one DB can serve several deployments.

4. Immutability and Tamper evidence

In the initial cut, ThunderID will not provide built-in support for this. A deployment which requires this can configure a DB that provides tamper evidence and immutability as the audit DB.

5. APIs and Console

• Admins:
  • GET /audit-events, with filters for time, actor, subject, target, action, outcome and trace ID.
  • Requires a dedicated audit read permission.
  • Shown as an Audit log page in the Console.
• End users:
  • GET /users/me/activity, returning their own events with fewer fields (no admin IDs or internal reasons).
  • Shown as an Activity view.

Architecture Overview

flowchart LR
  S[ThunderID services] -->|publish audit event| E[Eventing #5590]
  E --> AS[Audit subscriber]
  AS --> A[(auditdb)]
  J[Checkpoint and retention jobs] --> A
  A --> SVC[Audit service]
  SVC --> API1[/audit-events/] --> C1[Console: Audit log]
  SVC --> API2[/users/me/activity/] --> C2[Console: Activity]

Loading

All of this lives in a new internal/audit package. Services that publish audit events depend only on the event type and the catalog, never on the store.

Security Considerations

• No secrets in records. Records never contain passwords, OTPs, tokens, secrets or attribute values. Updates list only the names of changed attributes.
• Who can read the trail.
  • Admins need a dedicated permission, and their queries are themselves audited.
  • End users see only their own events.
• Integrity. The docs will recommend a DB for auditdb that provides tamper evidence and immutability.
• Personal data. Records hold user IDs, names and IPs. Retention limits how long they're kept, and erasure is question 3.
• Availability. Sign-ins depend on the audit event being recorded. See [Design Discussion] Generic eventing framework #5590.

Impacted Areas

• Eventing ([Design Discussion] Generic eventing framework #5590)
• New internal/audit package
• Authentication, tokens, consent, sessions and authorization
• User, group, role, application, IdP, flow and organization unit management
• Persistence: new auditdb schema (PostgreSQL, SQLite)
• APIs and OpenAPI specs
• Console: Audit log and Activity views
• Configuration (deployment.yaml, default.json, Helm chart) and docs

Alternatives Considered

Alternative 1: Store audit records in runtime_persistent

• Pros: no new datasource.
• Cons: the fastest-growing data would share a DB with sessions, with no separate retention, sizing or permissions.
• Decision: rejected.

Questions for Community Input

1. Is tamper evidence needed in initial version?