Skip to content

redact cannot express a computed replacement, so anything needing logic must be scrubbed before emit() #463

Description

@aaronvanston

Problem

emitWideEvent() writes to the console sink before any drain runs:

// packages/evlog/src/logger.ts, main @ 2.22.4
finalizeAudit(formatted)

if (globalRedact) {
  formatted = redactEvent(formatted, globalRedact) as WideEvent   // declarative only
  markGloballyRedacted(formatted)
}

if (!globalSilent) {
  if (globalPretty) prettyPrintWideEvent(formatted)
  else if (globalStringify) console[getConsoleMethod(level)](JSON.stringify(formatted))
  else console[getConsoleMethod(level)](formatted)
}

if (!deferDrain) { /* globalDrain + plugin drains */ }

That ordering is correct and documented, and the redaction docs state it plainly:

"Redaction scrubs PII from events before console output and before any drain sees
the data."

The gap is that the redaction stage is the only place a consumer can act before the console write, and its policy language is entirely declarative. RedactConfig (packages/evlog/src/types.ts) exposes paths, patterns, builtins, and replacement?: string. No user-facing member is function-valued — the only function slots are the @internal _maskers built by resolveRedactConfig from the built-ins — and no plugin hook runs earlier: EvlogPlugin.enrich is documented as "Runs before drain", which is after the console write, so it can never scrub what already reached stdout. The middleware path is the same: its redact is applied inside runEnrichAndDrain (packages/evlog/src/shared/middleware.ts), which runs after the event has been emitted.

So any redaction that needs logic has nowhere to run. Concretely, ours replaces a bearer credential in a request path with a stable fingerprint derived from that credential, so operators can still correlate requests without the credential being readable:

/public/claim/eyJhbGciOi...   ->   /public/claim/[tok:9f3a1c]

replacement?: string can only produce a constant, so the correlation value is lost. The same applies to any policy that is conditional (redact this field only for this tenant, or only when a sibling field has a given value), schema-driven, or allowlist-shaped rather than denylist-shaped.

The only current escape is silent: true plus a custom drain, which throws away all console output to gain a transform. On platforms that ingest stdout (Vercel, most container runtimes) losing the console sink is not an acceptable trade.

Minimal repro

createEvlog({
  service: 'demo',
  redact: { patterns: [/\/public\/[a-z]+\/([A-Za-z0-9._-]{12,})/g], replacement: '[redacted]' },
})

Request /public/claim/<token>. stdout shows [redacted], with no way to emit a fingerprint of the matched value instead. Attempting the same from enrich or from a drain leaves the raw token already printed to stdout by the time either runs.

Proposed change

Make the existing emit-time redaction stage accept a function. Either shape works; the second is the smaller change:

// A: a transform that runs at the same point as redactEvent, before the console write
redact?: boolean | RedactConfig | { transform: (event: WideEvent) => void }

// B: allow a function replacement, matching String.prototype.replace semantics
replacement?: string | ((match: string, path: string) => string)

Either way the contract is: runs after finalizeAudit, before the console write, synchronously, for every sink including the console. Errors thrown by the hook should be caught and reported the way drain failures already are, so a bad policy degrades to unredacted-but-logged rather than to a dropped request.

This is not a request for a new subsystem. Redaction already runs in the right place; it only needs to be programmable.

Why this is worth doing beyond our case

Redaction config reaching the emit path has already been the subject of two bugs, #408 (createEvlog ignored redact for main request events) and #441 (createInstrumentation().register() dropped redact and locked the logger, silently disabling it). Both were the same class of defect: redact failing to arrive on one of several init paths. Consumers who cannot express their policy declaratively end up wrapping the handler themselves and scrubbing before emit(), which puts the policy outside evlog entirely and outside the reach of any future fix to those paths.

Our workaround, as evidence of the cost

We wrap withEvlog so the scrub happens inside the request scope before the event is emitted, and separately pre-scrub the request object handed to onRequestError, because instrumentation error events hit the console sink on the same path. We also keep a drain-side scrub as defence in depth, so the same policy is expressed twice in two places that must not drift. A function-valued redaction hook would collapse all three into one declared policy.


Verified against main at evlog@2.22.4; we pin 2.17.0, and the emit ordering, RedactConfig shape, and plugin hook ordering are the same on both.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions