Skip to content

Stream errors not forwarded through .pipe() in image processing, causing hangs on mid-transfer failures #187

Description

@bbornino

sync-author/processor.ts's processProfileImg had a bug where Readable.fromWeb(stream).pipe(pipeline) doesn't forward source stream error events to the sharp destination pipeline. If the GitHub content stream fails mid-transfer, the pipeline hangs waiting for data and s3.upload blocks indefinitely instead of failing fast. Fixed on PR #186 by attaching an explicit error listener that destroys the pipeline.

While fixing that, found the identical bug in a second spot, and a related but differently-shaped case in a third:

  • sync-collection/processor.ts:35 (processImg, used for collection cover/social image uploads) — identical shape to the sync-author bug: Readable.fromWeb(stream as never).pipe(pipeline) with no error forwarding. Should take the same fix.
  • url-metadata/utils/processImage.ts:102,127 (body.pipe(pipeline) and metadataStream.pipe(transformer)) — source here is already a Node stream from undici, not converted via Readable.fromWeb, but .pipe() still doesn't forward error events by default, so the same hang-on-failure risk likely applies. This one has more surrounding surface area (existing custom timeout/dump/compare logic), so it needs its own review rather than a copy-paste of the sync-author/sync-collection fix — worth checking whether the existing timeout logic already provides a safety net here before assuming it's broken the same way.

Context

Surfaced via a CodeRabbit finding on PR #186 (#177, author_roles table). See that PR for the confirmed fix pattern (attach an error listener that calls pipeline.destroy(err) on the source stream before piping).

Metadata

Metadata

Assignees

Labels

Type

Fields

No fields configured for Bug.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions