fix(producer): normalize error messages to prevent [object Object] in telemetry#1099
Conversation
… telemetry When a render fails and the caught value is a plain object (not an Error instance), String(error) produces [object Object], masking the real error in PostHog telemetry (~24 errors/day). Add normalizeErrorMessage() that tries Error.message, string passthrough, .message on plain objects, JSON.stringify, and String() as a last resort. Apply it on the two telemetry-feeding paths: the main render failure handler (renderOrchestrator.ts:2099) and buildRenderErrorDetails (cleanup.ts), plus the error classifier isRecoverableParallelCaptureError so timeout detection works even when the thrown value is a plain object.
vanceingalls
left a comment
There was a problem hiding this comment.
Code Review
CI is all green. Implementation is clean. A few findings before this ships:
Blocker: CLI telemetry path is still broken
The stated goal is fixing [object Object] in PostHog telemetry (~24 errors/day). But the PostHog send for render_error events happens in packages/cli/src/commands/render.ts:1000-1018 — handleRenderError — which still uses the un-normalized pattern:
// render.ts:1007 — NOT touched by this PR
const message = error instanceof Error ? error.message : String(error);
trackRenderError({ ..., errorMessage: message, ... });executeRenderJob re-throws error at the end of its catch block (throw error). The CLI catches the original value and runs String(error) on it before PostHog. The normalized job.error and job.errorDetails.message this PR sets are used for the SSE/HTTP response path — not for the CLI's PostHog event.
The studio server has no independent PostHog path; trackRenderError tags source: props.source ?? "cli", so the ~24/day events are CLI-sourced. This PR normalizes the wrong side of the rethrow. handleRenderError needs the same fix.
Important: shouldFallbackToScreenshotAfterCalibrationError not converted
isRecoverableParallelCaptureError was converted — correct, it classifies errors via regex and plain-object errors would produce [object Object] → regex miss → wrong result. Its sibling shouldFallbackToScreenshotAfterCalibrationError in packages/producer/src/services/render/captureCost.ts:419-420 has identical code shape and identical failure mode but wasn't touched:
// captureCost.ts:419-420 — NOT touched by this PR
export function shouldFallbackToScreenshotAfterCalibrationError(error: unknown): boolean {
const message = error instanceof Error ? error.message : String(error);
return /HeadlessExperimental\.beginFrame timed out|.../i.test(message);
}If Chrome throws a plain object on BeginFrame timeout (plausible for protocol-layer errors), this regex never matches → no screenshot fallback → harder failure. Same fix: normalizeErrorMessage(error). Should be in this PR alongside isRecoverableParallelCaptureError.
Nit: circular-reference test asserts the bug output
// errorMessage.test.ts:40-41
// Falls through JSON.stringify failure to String()
expect(normalizeErrorMessage(obj)).toBe("[object Object]");The function falls back to String() for unstringifiable inputs, which produces the exact string the PR is trying to eliminate from telemetry. Defensible (don't throw on bad inputs), but consider a sentinel like "<circular reference>" or Object.keys(obj).join(",") — something distinguishable. As-is, a circular-reference error would still produce [object Object] in PostHog. Not a blocker, but the test comment "Falls through JSON.stringify failure to String()" reads like an apology for the behavior rather than an assertion of intent.
Not a concern: remaining String(err) sites in renderOrchestrator.ts
Lines 118, 561, 725, 784, 943, 1036, 1330, 2055 all go to log.warn/log.debug, not telemetry. They're inconsistency, not a P0 issue. Fine to address in a follow-up codemod.
Not a concern: stack in errorDetails server response
Pre-existing, not introduced here.
Not a concern: require() in engine ESM packages
Checked — no require() calls in packages/engine/src/. Clean.
The normalizeErrorMessage utility itself is correct and well-tested for the cases it covers (Error instances, strings, plain objects, null/undefined, primitives, non-string .message). The PR's approach is right. The two issues above are about coverage, not design.
— Vai
vanceingalls
left a comment
There was a problem hiding this comment.
Both prior findings verified fixed:
-
CLI telemetry path — in now imports and uses from instead of the inline pattern. The PostHog path is covered.
-
**** — converted to in .
The utility itself is solid: correct priority ordering (Error → string → .message → JSON.stringify → String fallback → "unknown error"), handles circular references, and has good test coverage across the relevant edge cases.
LGTM. Ship it.
— Vai
vanceingalls
left a comment
There was a problem hiding this comment.
Both prior findings verified fixed:
-
CLI telemetry path —
handleRenderErrorinpackages/cli/src/commands/render.tsnow imports and usesnormalizeErrorMessagefrom@hyperframes/producerinstead of the inlineinstanceof Errorpattern. The PostHogrender_errorpath is covered. -
shouldFallbackToScreenshotAfterCalibrationError— converted tonormalizeErrorMessageincaptureCost.ts.
The normalizeErrorMessage utility itself is solid: correct priority ordering (Error → string → .message → JSON.stringify → String fallback → "unknown error"), handles circular references, and has good test coverage across the relevant edge cases.
LGTM. Ship it.
— Vai
…solution The Vite test runner can't resolve runtime imports from @hyperframes/producer since its exports point to dist/. Copy the utility into the CLI package and import locally instead.
Summary
Errorinstance),String(error)produces[object Object], masking the real error in PostHog telemetry (~24 errors/day).normalizeErrorMessage()utility that extracts.messagefrom plain objects, falls back toJSON.stringify, and only usesString()as a last resort for primitives.renderOrchestrator.ts:2099),buildRenderErrorDetailsincleanup.ts, andisRecoverableParallelCaptureError(so timeout classification works even with plain-object errors).Test plan
normalizeErrorMessagecovering Error instances, strings, plain objects with.message, objects without.message, null, undefined, numbers, non-string.message, and circular referencesbun test --filter errorMessagepasses (9 tests)