Discussion: ability to execute the pipeline synchronously

[nicolaspanel]

Idea: the current implementation always executes the pipeline asynchronously using PipelineWorker. It would be great to be able to execute it synchronously.

Example:

var res = sharp(input)
  .resize(200, 200)
  .toFormat('jpeg')
  .toBufferSync() // instead of .toBuffer(function(){...})

// and

var res = sharp(input)
  .resize(200, 200)
  .toFormat('jpeg')
  .toFileSync(pathToFile);  // instead of .toFile(pathToFile, function(){...})

@lovell : What do you think? I can work on a pull request if you like the idea.

[lovell]

Bonsoir, is the need for sync behaviour based on executing the operation maintenant vs joining a shared queue that might be full of otherwise-blocking local file operations?

Have you looked at something like https://github.com/abbr/deasync ?

There are some potential changes coming in libuv that would make the behaviour of any native module (like this one) that uses uv_queue_work become synchronous, albeit still appearing to be async - see libuv/libuv#382

One thing to consider is that the async methods can pass more than one value to a callback, which has already been noted with the Promise-returning toBuffer lacking the info parameter - see #143.

Separating the pipeline processing from worker/queuing would clean up the internals a little :)

[nicolaspanel]

Hi, thank you for the answer and for the link to DeAsync.

My need is to avoid callbacks, so DeAsync seems to be a good workaround for me.

It could also be an easy way to extend the API with synchronous methods such as .toBufferSync and toFileSync with something like:

var deasync = require('deasync');
...
Sharp.prototype.toBufferSync = function() {
  var done = false;
  var data, info;

  return this.toBuffer(function(err, _data_, _info_){
    if (err){ throw err; }
    data = _data_;
    info = _info_;
    done = true;
  });
  deasync.loopWhile(function(){return !done;});
  return {data:data, info: info};
};

[lovell]

Voila, thanks for confirming.

If you only need data, the following (untested) might also work:

var pipeline = sharp(input).resize(200, 200).jpeg();
var data = deasync(pipeline.toBuffer)();

[niftylettuce]

@lovell Can we please re-open this and add functions such as these (tested)?

// <https://github.com/lovell/sharp/issues/360#issuecomment-185162998>
Sharp.prototype.toBufferSync = function() {
  let done = false;
  let data;
  this.toBuffer((err, _data_) => {
    if (err) {
      throw err;
    }
    data = _data_;
    done = true;
  });
  deasync.loopWhile(() => {
    return !done;
  });
  return data;
};

Sharp.prototype.metadataSync = function() {
  let done = false;
  let data;
  this.metadata((err, _data_) => {
    if (err) {
      throw err;
    }
    data = _data_;
    done = true;
  });
  deasync.loopWhile(() => {
    return !done;
  });
  return data;
};

[lovell]

@niftylettuce I'd prefer not to include a dependency on deasync internally. Should anyone be willing to create, publish and support a wrapper module that combines the two I'd be happy to link to it from the docs.

I'd guess those who require an imperative approach will increasingly use async/await syntax, which is already supported.

[delijah]

Isn't it possible for you to do a toBufferSync without using deasync? Couldn't you just use sync functions in the background like fs.writeFileSync? Sorry, don't know the internals of sharp, maybe a stupid question :)

Would be great to have sync functions though.

[niftylettuce]

I've released https://lipo.io which offers toBufferSync, toFileSync, and metadataSync methods! 🎉

[dkrnl]

working deasync example:

  const pipeline = sharp(content).toFormat('jpeg', { quality: 90 });
  const output = deasync(pipeline.toBuffer.bind(pipeline))();