Trying not to loose stream data on "Input buffer contains unsupported image format"

[alexandrebodin]

Question about an existing feature

What are you trying to achieve?

Hi, I'm mainly looking for the recommended way to achieve the following:

// assume I have a stream coming in

try {
  const pipeline = sharp();
  stream.pipe(pipeline);
  
  await pipeline.stats(); 
  
  // do some processing 
  
  pipeline.pipe(fs.createWriteStream('./some-file'));
  
} catch {
  // just store the file and warn the user it wasn't processed
  stream.pipe(fs.createWriteStream'./some-file'));
}

In this scenario, I am receiving any kind of files, when the file is something I can process with sharp I want to run a pipeline for it, else I want to store the file as is.

Currently, pipeline.stats() starts reading the stream. When it fails on non supported files we endup in the catch where there is nothing left to read.

Is there a way to use pipeline.clone().pipe(fs.createWriteStream('./some-file')) and tell sharp to just be a "PassThrough" if it can't support a file ?

When you searched for similar issues, what did you find that might be related?

Found a lot around cloning but all the scenarios assume the file is acceptable by sharp

Please provide a minimal, standalone code sample, without other dependencies, that demonstrates this question

See above

Please provide sample image(s) that help explain this question

• use any zip you have locally for ex (not gonna provide sth unsafe here :D)

[lovell]

tell sharp to just be a "PassThrough" if it can't support a file

sharp will always perform a decode/encode roundtrip.

The scenario you describe, where you need alternative logic depending on the contents of a Stream, suggests you'll either need to create your own Transform Stream or perhaps remove Streams entirely with something like https://github.com/stream-utils/raw-body

If this is for Strapi, please see strapi/strapi#16978 (comment)

[alexandrebodin]

Yes it's for strapi and I'm actually reworking the internals to read the stream directly from the request instead of having a file on disk.
Are you recommending not using streams as a general comment ? is it because in anycase you endup loading enverything into memory or sth like that ?

It might be valuable to know (or document) what operations (metadata,stats,resize,...) actually are buffering everything into memory and which actually benefit from streams 🤔

[alexandrebodin]

Going to close this as your answer is enough to know I will need to manage on our side anyway ;)

[lovell]

If you're generating multiple output images (dimensions and formats) from a single input and a local filesystem is available then my advice to use it, at least for the input, remains. This will both lower fragmentation and increase throughput.