Control flow

Overview

The defining characteristic of node.js, and a primary reason for its success, is its asynchronous "non-blocking" IO (input/output) APIs. In the case of JavaScript, asynchronous means the value will be processed at a later tick of the event loop:

// Note, all of the below ignore errors

console.log(1)

// async/await version
async ()=>{
  console.log(await fs.promise.readdir(__dirname))
}()

// Promise version
fs.promise.readdir(__dirname)
  .then(files => console.log(files))
  
// Callback version
fs.readdir(__dirname, (err, files) => console.log(files))

console.log(2)

/* Output:
1
2
['index.js']
['index.js']
['index.js']
*/

Note: The node.js core APIs, by default, use callbacks. These guides will assume you are using songbird to expose the promise = module.promise.method(...) helper API (e.g., promise = fs.promise.readdir(...)).

`async/await`

Introduction

Async functions were added to JavaScript in ES7, giving us the ability to use a synchronous-style syntax for asynchronous programming.

Functions marked with the async keyword, can use await within their body:

async ()=> {
  console.log(await fs.promise.readdir(filename)) // ['index.js']
}

await pauses the execution of the current stack frame until the asynchronous operation (aka Promise) resolves.

`async`

The async keyword is used to create an asynchronous function.

async ()=> {
  // ...
}

In almost every way, async functions behave exactly like regular functions. They can be passed arguments, return values, named, nested, invoked, invoked immediately, inherit from Function.prototype, etc. However, they differ in a few ways.

async functions:

Catch errors thrown within their body
Always return a Promise that
- Fulfills to the return value
- Rejects to a thrown error
Can use await within the shallow function body (not in nested functions)

async function add(one, two) {
  await process.promise.nextTick()
  return one + two
}

let promise = add(1, 2)
promise.then(result => console.log(result)) // 3

`await`

Use await to wait for a Promise and its resolution value:

async ()=>{
  let filenames = await fs.promise.readdir(__dirname)
  console.log(filenames) // ['index.js']
}()

If the promise fails, an error will be thrown instead:

async ()=>{
  let filenames = await fs.promise.readdir(' does not exist')
}().catch(e => console.log(e)) // Error: ENOENT: no such file or directory

Further, await conveniently allows us to use language control-flow features like for, if and try/catch when writing asynchronous code:

async function ls(dirPath) {
  let stat = await fs.promise.stat(dirPath)
  if (!stat.isDirectory()) return [dirPath]
  
  let filenames = []
  for (let name of await fs.promise.readdir(dirPath)) {
    let result = await ls(path.join(dirPath, name))
    filenames.push(...result)
  }
  return filenames
}

async ()=> {
  try {
    console.log(await ls(__dirname))
  } catch(e) {
    console.log(e.stack)
  }
}()

Promises

Introduction

Added to the language in ES6, Promise API will be the standard idiom (in combination with async/await) for writing asynchronous JavaScript going forward.

// Consuming a promise API
let promise = fs.promise.readFile(__filename)
promise.then(result => {}, err => {})

// Returning promises
readFile(name) {
  return fs.promise.readFile(path.join(__dirname, name))
}

readFile('README.md')
	.then(result => {}, err => {})

Note: The Promise spec makes guarantees that are encapsulated within the API. Unlink callbacks, you need not worry about things like remembering to pass errors, never calling callbacks more than once or whether a control-flow library is compatible with those expectations. There's even a test suite to verify Promise spec compliance.

`.then(onFulfilled, onRejected)`

newPromise = promise.then(result=>{}, err=>{}) is the primary promise API.

fs.promise.readFile(__filename)
  .then(function onFulfilled(result) {
    console.log(String(result)),
  },
  function onRejected(err) {
    console.log(err.stack)
  })

It's important to note, .then returns a new promise based on the following logic:

Both onFulfilled and onRejected are optional

If promise is fulfilled, call onFulfilled with the result
If promise is rejected, call onRejected with the error
promise will either be fulfilled or rejected, never both
A resolved promise will call onFulfilled or onRejected only once per .then()

promise.then() returns a new Promise, newPromise

newPromise resolves to an error if onFulfilled or onRejected throws
Otherwise, newPromise resolves to the return value of onFulfilled or onRejected

If promise is fulfilled, it will skip all onRejected to the next onFulfilled
Likewise, if promise is rejected, it will skip all onFulfilled to the next onRejected

Promise.resolve(1)
  .then(res => ++res)
  .then(res => ++res)
  .then(console.log) // 3
  
// Skipping onFulfilled/onRejected
Promise.reject(new Error('fail'))
  .then(() => console.log('hello')) // Never executes
  .then(null, err => err.message) // Convert rejection to fulfillment
  .then(null, console.log) // Never executes
  .then(console.log) // 'fail'

Recommended: To get more comfortable with Promises, checkout the Nodeschool.io Promise It Won't Hurt Workshopper.

`.catch(onRejected)`

.catch(onRejected) is a short-hand for .then(null, onRejected)

`Promise.all([promises])`

Converts an array of promises to a single promise that is fulfilled when all the promises are fulfilled, and is rejected otherwise.

let promises = [Promise.resolve(1), Promise.resolve(2)]
Promise.all(promises)
  .then(console.log) // 1, 2
  
  
let promises = [Promise.resolve(1), Promise.reject(new Error('fail')]
Promise.all(promises)
  .then(console.log) // Never executes
  .catch(console.log) // 'fail'

Noted: Use Promise.all to wait on promises in parallel. This is especially useful when combined with await:

async function ls(dirPath) {
  let stat = await fs.promise.stat(dirPath)
  if (!stat.isDirectory()) return [dirPath]
  
  let promises = []
  for (let name of await fs.promise.readdir(dirPath)) {
    // Run recursive ls in parallel
    let promise = ls(path.join(dirPath, name))
    promises.push(promise)
  }
  // Wait for them in parallel
  return _.flatten(await promises)
}

async ()=> {
  try {
    console.log(await ls(__dirname))
  } catch(e) {
    console.log(e.stack)
  }
}()

Callbacks (aka Errbacks)

Introduction

The node.js core APIs, by default, use callbacks.

Note: Node.js callbacks are sometimes referred to as "errbacks", "error-backs", "node-backs" or "error-first callbacks" to denote the specific flavor of callbacks in node.js.

Simple Description:

A function passed as the last argument to an async API
Takes 2 arguments: (err, result) => {}

fs.readFile(__filename, (err, result) => {} )

Simple right? But beware, "Here be dragons!"

Callback Contract

When writing callbacks, there are numerous unenforced implicit expectations.

Here is the complete Callback Contract:

Passed as the last argument
Takes 2 arguments:

The first argument is an error
The second argument is the result
Non-idiomatic callbacks may pass >1 result

Never pass both
error instanceof Error === true
Must never excecute on the same tick of the event loop
Return value is ignored
Must not throw / must pass resulting error to any available callback
Must be called only once

Whenever you write a callback, you must remember to follow all of the above requirements or else difficult-to-debug bugs may occur!

You will quickly find this to be both tedious and error-prone. As such, callbacks are by definition anti-patterns and are highly discouraged despite their mass adoption.

Control-flow Design Patterns

When dealing with asynchronous operations, sometimes you must wait on the previous operation's result. Other times, you do not. Sometimes you only care about the end result of a chain of operations. Below are examples of how to handle these scenarios.

Serial

Executing 3 tasks in series:

// async/await
async ()=>{
  let a = await one()
  let b = a + await two()
  let c = b + await three()
  console.log(c / 2)
}()

// promises
one()
  .then(a => two().then(res => res + a))
  .then(b => three().then(res => res + b))
  .then(c => console.log(c / 2))
  
// callbacks + async package
async.waterfall([
  (callback) => one(callback),
  (a, callback) => two((err, res) => callback(err, res + a)),
  (b, callback) => three((err, res) => callback(err, res + b)),
  (c, callback) => console.log(c / 2)
])

Parallel

Executing 3 tasks in parallel:

// async/await
async ()=>{
  let [a, b, c] = await Promise.all([
    one(),
    two(),
    three()
  ])
  console.log(a + b + c)
}()

// promises + bluebird (for .spread)
Promise.all([
  one(),
  two(),
  three()
]).spread((a, b, c) => console.log(a + b + c))
  
// callbacks + async package
async.parallel([
  (callback) => one(callback),
  (callback) => two(callback),
  (callback) => three(callback)
], (err, res) => console.log(res[0] + res[1] + res[2]))

Branch

Branch is a term used to describe an independent chain of control-flow that results in a single value / outcome. Branches tend to correlate well with moving logic to separate functions.

// async/await
async ()=>{
  // some stuff in series
  let promiseX = async()=>{
    let a = await one()
    let b = await two()
    return await three()
  }
  let promiseY = fs.promise.readdir(__dirname)
  
  // some stuff in parallel
  let [x, y] = await Promise.all([promiseX, promiseY])
  console.log(x + y)
}()

// promises
// some stuff in series
let promiseX = one()
  .then(a => two())
  .then(b => three())
let promiseY = fs.promise.readdir(__dirname)

// some stuff in parallel
let [x, y] = await Promise.all([promiseX, promiseY])
console.log(x + y)

// callbacks + async package
// some stuff in parallel
async.parallel([
  (callback) => {
    // some stuff in series
    async.waterfall([
      (callback) => one(callback),
      (a, callback) => two(callback),
      (b, callback) => three(callback)
    ], callback)
  },
  (callback) => fs.readdir(callback)
], (err, res) => console.log(res[0] + res[1]))

Interoperability

You may not find yourself in all of these scenarios, but they're here for reference.

Async -> Promise

// Scenario 1: Named function
async function foo() {
  // logic
}
promise = foo()

// Scenario 2: Lambda
promise = async ()=>{
  // logic
}()

Promise -> Async

async fooAsync() {
  return await fooPromiseReturning()
}

Async -> Callback

Using bluebird-nodeify:

async function fnAsync() {
  // some logic
}
nodeify(asyncFn(), callback)

Callback -> Async

Using songbird:

async function asyncFn() {
  // Callback -> Promise
  return await fs.promise.readFile(...)
}

Promise -> Callback

Using bluebird-nodeify:

nodeify(promise, callback)

Callback -> Promise

Using songbird:

promise = fs.promise.readFile(...)

Async Generators

You can use async/await semantics in node.js today without babel by using yield and "async" Generator Functions (available in v0.12) in combination with APIs like bluebird.coroutine or co and co.wrap(function*(){...}):

$ # Turn on generator support
$ node --harmony --use-strict file-that-uses-generators.js

// Scenario 1: Reusable function*
let fnAsync = co.wrap(function *() {
  // yield on promises like async/await
  let files = yield fs.promise.readdir(__dirname)
})
let promise = fnAsync()

// Scenario 2: Lambda
let promise = co(function *() {
  // yield on promises like async/await
  let files = yield fs.promise.readdir(__dirname)
})

References

Note: In order of recommendation

Guides

How Do Promises Work?

Videos

Workshoppers

promise-it-wont-hurt - Promise API workshopper
learn-generators - Async generators are an ES6 hack for writing ES7 async functions (will teach async/await but with function*/yield instead)
async-you - Guide to using the most popular callback control-flow library, async

Blogs

Guide to asyncawait - Callback heaven with async/await
HTML5Rocks Promises Tutorial
The Long Road to async/await in JavaScript
async/await: The Hero JavaScript Deserved
Taming the Async Beast with ES7
We have a Problem with Promises - Common Promise mistakes explained
You're Missing the Point of Promises
Control-flow Performance Comparison
Promises in Node.js

Control flow

Overview

async/await

Introduction

async

await

Promises

Introduction

.then(onFulfilled, onRejected)

.catch(onRejected)

Promise.all([promises])

Callbacks (aka Errbacks)

Introduction

Callback Contract

Control-flow Design Patterns

Serial

Parallel

Branch

Interoperability

Async -> Promise

Promise -> Async

Async -> Callback

Callback -> Async

Promise -> Callback

Callback -> Promise

Async Generators

References

Guides

Videos

Workshoppers

Blogs

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Clone this wiki locally

`async/await`

`async`

`await`

`.then(onFulfilled, onRejected)`

`.catch(onRejected)`

`Promise.all([promises])`