Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding Core support for Promises #5020

Closed
wants to merge 33 commits into from

Conversation

@chrisdickinson
Copy link
Contributor

@chrisdickinson chrisdickinson commented Feb 1, 2016

Update 02/02/16 15:07:00 PST: Current state of the PR.
Update 02/01/16 14:10:00 PST: Current state of the PR.

Newcomers: As a heads up, GitHub has started clipping the thread down. I'm going to start cataloguing common questions tonight, and I'll include them under this notice so that you can quickly look to see if there's an answer to your issue without having to search through the entire thread. Thanks for taking a look & weighing in! Whether you're for this change or against it, your feedback is valuable and I'll respond to it as soon as I'm able.

FAQ / TL;DR for newcomers

What is the current proposal?

If landed, this will land behind a flag, moving out from behind the flag in the next major version as an unsupported API, finally becoming fully supported one major version after that. It does not replace or supersede the callback API, which will continue to be the canonical Node interface.

Promise-returning variants will be exposed for all "single-operation" Node methods — that is to say, Streams and EventEmitter-based APIs are not included in this discussion.

To use promises:

const fs = require('fs')

fs.readFile.promise(__filename).then(data => {
})

Using promises with destructuring assignment (where available):

const {readFile} = require('fs').promise

readFile(__filename).then(data => {
})

As a prerequisite to this PR landing, domains must work as expected with native Promises (they currently do not.) AsyncWrap is not blocked on this PR, neither is this PR blocked on AsyncWrap. Both PRs are blocked on getting adequate instrumentation of the microtask queue from V8 (we need a callback for "microtask enqueued" events so we can track their creation, and ideally a way to step the microtask queue forward ourselves, running code between invocations.)


Why use the promisify approach?

Response summed up here.


Why not fs.readFileAsync?

@phpnode notes that it could break existing users of bluebird.promisifyAll. Plus, the naming is contentious.


Why not return promises when callbacks are not given?

  • @jasnell expresses deep discomfort with switching return types.
    • We already have to provide alternate APIs in some places due to

Should programmer errors throw, or reject?

Summed up here.


What is the value in adding them to core?

@rvagg has been massively helpful in driving this conversation forward:


How do you address the need to keep core small while expanding the exposed API?

Where do we draw the line in supporting language-level constructs?

@yoshuawuyts, @rvagg, and @DonutEspresso have largely driven this conversation:


Why are modules (like require('fs/promise')) not in scope for this discussion?

@phpnode, @RReverser, @mikeal, @ktrott, (and others, I'm sure!) have brought this up, and I've been pushing to have this conversation in a different, later issue. The conversation has mainly revolved around consensus, and the difficulty in attaining it when adding modules to the mix.


This is the original text of the PR. It does not reflect the current state of the conversation. See above for common themes!
This PR introduces Core support for Promises. Where possible, callbacks have
been made optional; when omitted, a Promise is returned. For some APIs (like
http.get and crypto.randomBytes) this approach is not feasible. In those
cases promise-returning *Async versions have been added
(crypto.randomBytesAsync). The naming there isn't set in stone, but it's
based on how bluebird's promisifyAll
method generates names.

This PR allows users to swap the implementation of promises used by core.
This helps in a few dimensions:

  • Folks that prefer a particular flavor of Promises for sugar methods can set
    the promise implementation at app-level and forget about it.
  • This should alleviate concerns about stifling ecosystem creativity — putting
    promises in Core doesn't mean Core needs to pick a winner.
  • Folks who prefer a faster implementation of promises can swap in their
    preferred library; folks who want to use whatever native debugging is added
    at the V8 level for native promises can avoid swapping in promises.
  • A world of benchmarks opens up for Promise library authors — any code in the
    ecosystem that uses promises generated by core and has a benchmarking suite
    is a potential benchmark for promise libraries.

The API for swapping implementations is process.setPromiseImplementation().
It may be used many times, however after the first time deprecation warnings
are logged with the origin of the first call. This way if a package misbehaves
and sets the implementation for an application, it's easy to track down.

Example:

process.setPromiseImplementation(require('bluebird'));

const fs = require('fs');

fs.readFile('/usr/share/dict/words', 'utf8')
  .then(words => console.log(`there are ${words.split('\n').length} words`))
  .catch(err => console.error(`there are no words`));

Streams are notably missing from this PR – promisifying streams is more
involved since the ecosystem already relies on a common (and more importantly,
often pinned) definition of how that type works. Changing streams will take
effort and attention from the @nodejs/streams WG. All other callback-exposing
modules and methods have been promisified (or an alternative made available),
though.

Three new internal modules have been added:

  • lib/internal/promises — tracks the original builtin Promise object, as
    well as the currently selected Promise implementation. All promises
    created by Node are initially native, and then passed to the selected
    implementation's .resolve function to cast them.
  • lib/internal/promisify — turn a callback-accepting function into one that
    accepts callbacks or generates promises. Errors thrown on the same tick
    are still thrown, not returned as rejected promises — in other words,
    programmer errors, such as invalid encodings, are still eagerly thrown.
  • lib/internal/callbackify — turn a synchronous or promise-returning function
    into a callback-accepting function. This is only used in lib/readline for
    the completer functionality. This could probably fall off this PR, but it
    would be useful for subsequent changes to streams.

Breaking Changes

  • In general: any API that would have thrown with no callback most likely does
    not throw now.
  • fs APIs called without a callback will no longer crash the process with an
    exception.
  • ChildProcess#send with no callback no longer returns Boolean, instead
    returns Promise that resolves when .send has completed.
  • Wrapped APIs:
    • child_process.ChildProcess#send
    • cluster.disconnect
    • dgram.Socket#bind
    • dgram.Socket#close
    • dgram.Socket#send
    • dgram.Socket#sendto
    • dns.lookupService
    • dns.lookup
    • dns.resolve
    • fs.access
    • fs.appendFile
    • fs.chmod
    • fs.chown
    • fs.close
    • fs.exists
    • fs.fchmod
    • fs.fchown
    • fs.fdatasync
    • fs.fstat
    • fs.fsync
    • fs.ftruncate
    • fs.futimes
    • fs.lchmod
    • fs.lchown
    • fs.link
    • fs.lstat
    • fs.mkdir
    • fs.open
    • fs.readFile
    • fs.readdir
    • fs.readlink
    • fs.realpath
    • fs.rename
    • fs.rmdir
    • fs.stat
    • fs.symlink
    • fs.unlink
    • fs.utimes
    • fs.writeFile
    • net.Socket#setTimeout
    • readline.Interface#question
    • repl.REPLServer#complete
    • zlib.deflateRaw
    • zlib.deflate
    • zlib.gunzip
    • zlib.gzip
    • zlib.inflateRaw
    • zlib.inflate
    • zlib.unzip

New APIs

  • process.setPromiseImplementation
  • net.connectAsync
  • tls.connectAsync
  • crypto.randomBytesAsync
  • crypto.pbkdf2Async
  • crypto.pseudoRandomBytesAsync
  • crypto.rngAsync
  • crypto.prngAsync
  • http.getAsync
  • https.getAsync
  • http.requestAsync
  • https.requestAsync
  • child_process.execAsync
  • child_process.execFileAsync

Next steps

I haven't finished this PR yet: the primary missing piece is tests for the
promise-based APIs, to ensure that they resolve to the correct values. Docs
also need updated. I'll be working on this in my free time over the next week.

Here are the bits that I'd like folks reading this to keep in mind:

  • The code changes here are fairly cheap, time-wise — one evening's worth of work.
    • Technical possibility has never been the primary blocker.
  • We know async/await is coming down the pipe, and many devs are interested in
    it.
    • With ChakraCore potentially coming in, this may happen sooner than anyone
      previously imagined.
    • Even sans the ChakraCore PR, it's my understanding that V8 will be supporting
      async/await by EOY (Correct me if I'm wrong, here!)
  • Promises may not be your cup of tea. This is okay. This is not an attempt to
    replace callbacks or streams with promises. They can co-exist. While Promises
    are complicated, much of that complication falls out of the problem space that
    both callbacks and promises abstract over. Give this a fair shake.
case 1: return resolve();
case 2: return resolve(arguments[1]);
}
return resolve([...arguments].slice(1));

This comment has been minimized.

@chrisdickinson

chrisdickinson Feb 1, 2016
Author Contributor

eslint chokes on ... — it may need upgraded?

This comment has been minimized.

@targos

targos Feb 1, 2016
Member

try to add spread: true to ecmaFeatures in our eslint config

This comment has been minimized.

@sebmck

sebmck Feb 1, 2016

@chrisdickinson Looks like the ecmaFeatures section of .eslintrc is missing spread: true.

This comment has been minimized.

@sebmck

sebmck Feb 1, 2016

@chrisdickinson Also I'd be worried that the use of arguments here causing a deoptimisation. It should be better to use a rest param and refer to that.

This comment has been minimized.

@Fishrock123

Fishrock123 Feb 1, 2016
Member

Afaik rest params are currently still quite slow in our versions of V8, can we avoid them for now?

@gergelyke
Copy link
Contributor

@gergelyke gergelyke commented Feb 1, 2016

Hey,
just out of curiosity: why the need for setPromiseImplementation? Wouldn't it be better to go with the standard ES6 promises?

@chrisdickinson
Copy link
Contributor Author

@chrisdickinson chrisdickinson commented Feb 1, 2016

@gergelyke This provides an escape hatch at application for folks that prefer different promise implementations, while also making sure that core doesn't suck the air out of the promise library ecosystem. Libraries can compete on features and performance without one implementation or the other winning by default. This also means that, in a vm-neutral future, developers working on applications can shield themselves from differences between VMs to some degree by using a third-party promises library.

@sebmck
Copy link

@sebmck sebmck commented Feb 1, 2016

Is process.setPromiseImplementation necessary? What does it achieve that global.Promise = foo; doesn't?

@chrisdickinson
Copy link
Contributor Author

@chrisdickinson chrisdickinson commented Feb 1, 2016

@timoxley, @kittens: Mostly this is so we can use a "verified good for our purposes" promise implementation, and guard it with "hey, don't try and set the implementation twice" warnings. This also means that core always creates known-native promises at the outset and only casts them to user-chosen implementations. This may prove useful when combined with v8 dev tools (OTOH, it could be a case of YAGNI!)

@ChALkeR
Copy link
Member

@ChALkeR ChALkeR commented Feb 1, 2016

@gergelyke Note that native v8 promises performance / memory consumption is far from perfect atm. See https://github.com/petkaantonov/bluebird/tree/master/benchmark for a comparison (note the difference between promises-ecmascript6-native and promises-bluebird.

@madbence
Copy link

@madbence madbence commented Feb 1, 2016

@timoxley the Promise impl should be controlled by the application you write. If a library tries to mess with global.Promise (or process.setPromiseImplementation), that's just wrong.

@chrisdickinson
Copy link
Contributor Author

@chrisdickinson chrisdickinson commented Feb 1, 2016

@madbence That's why setPromiseImplementation warns (with a trace noting both the original setPromiseImplementation call as well as a trace to the second call) on subsequent attempts to set the promise impl — so you can track down the offending module and remove it from your application. This API is intended for top-level applications only.

@timoxley
Copy link
Contributor

@timoxley timoxley commented Feb 1, 2016

@chrisdickinson I guess this raises the question of why add this feature specifically for Promises? should node provide pluggable implementations for other APIs as well?

@trevnorris
Copy link
Contributor

@trevnorris trevnorris commented Feb 1, 2016

net.connectAsync() is fundamentally improperly named. The "executor" is still ran immediately, and if no hostname is given then TCP will bind to the port immediately. To get around this synchronous behavior node places the callback in a nextTick. Since I assume the callback will execute through the resolver it will be placed on the MicrotaskQueue and executed just after the nextTickQueue is completed. Causing another nextTick to be queued, and the nextTickQueue to be processed again.

In the end this API isn't any more async than it currently is.

Also I am curious how promisify is supposed to handle swallowing errors if the user's actual callback is fired on a different stack.

@danbucholtz
Copy link

@danbucholtz danbucholtz commented Oct 26, 2016

@chrisdickinson, there's a lot to take in here 😄 . Is there an update on the general topic of Node.js core API supporting promises instead of callbacks?

Thanks,
Dan

@jasnell
Copy link
Member

@jasnell jasnell commented Feb 28, 2017

Closing given the lack of forward progress on this. We'll definitely want to revisit this later tho.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

You can’t perform that action at this time.