Does sharp begin processing when the first stream chunk arrives?

[Jaid]

Question about an existing feature

What are you trying to achieve?

I can currently decide between feeding sharp with a finished Buffer or with a Readable that loads 8 MB in 120 chunks over a timespan of 800 milliseconds. Is there an expected performance increasing for using the latter, such as processing before all chunks arrived? I use raw input, so the pixel information of earlier bytes does not depend on next bytes. The only operation I currently use is sharp.extract on an image area the bytes arrive very early for (everything needed is loaded in the first 30% of bytes).

When you searched for similar issues, what did you find that might be related?

• pipeline await and toBuffer #2975
• why use toBuffer() is to slow #2036
• Composite stream #3136

Example

Sharp

Buffer

console.time("run")

const sharpFactory = buffer => sharp(buffer, {
  raw: {
    width: 1080,
    height: 1920,
    channels: 4,
  },
})
  .extract({
    left: Math.floor(0.6 * 1080),
    top: Math.floor(0.08 * 1920),
    width: Math.floor(0.4 * 1080),
    height: Math.floor(0.02 * 1920),
  })

const buffer = execution.stdout.subarray(12)
const saver = fs.createWriteStream("./dist/image.bin")
sharpFactory(buffer).pipe(saver).on("finish", () => {
  console.timeEnd("run")
})

Pipeline

import {pipeline} from "node:stream/promises"

console.time("run")

const sharpFactory = sharp({
  raw: {
    width: 1080,
    height: 1920,
    channels: 4,
  },
})
  .extract({
    left: Math.floor(0.6 * 1080),
    top: Math.floor(0.08 * 1920),
    width: Math.floor(0.4 * 1080),
    height: Math.floor(0.02 * 1920),
  })

const saver = fs.createWriteStream("./dist/image.bin")
await pipeline(execution.stdout, new TransformerRemove12Bytes, sharpFactory, saver)
console.endTime("run")

The Pipeline usage (900 ms average) is much faster than the Buffer usage (3500 ms average), but I don't know whether this is because of my personal implementation or because of the pipe optimizations Sharp could do.

[lovell]

Hi, did you see the future possible enhancement at #179 ?

[germansokolov13]

I read #179 but it is unclear from either issues or docs what's the current status of this feature.

[lovell]

@germansokolov13 The phrase "future possible enhancement" means that it is a valid and potential enhancement that might exist in the future and requires someone to do the work in order to make that happen.

[lovell]

I hope this information helped. Please feel free to re-open with more details if further assistance is required.