You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I started reading Node.js Design Patterns this week. I got the Third Edition, and have not spent any time looking into what's changed from prior editions. The first 6 chapters cover fundamental knowledge, before getting into the meaty named Design Patterns, so these notes are from that first "half" of the book.
1. libuv and the Reactor Pattern
libuv is something I've often heard about as a low level Node.js library, but now I have a glimpse of what it does for us. As the book says:
Libuv represents the low-level I/O engine of Node.js and is probably the most important component that Node.js is built on. Other than abstracting the underlying system calls, libuv also implements the reactor pattern, thus providing an API for creating event loops, managing the event queue, running asynchronous I/O operations, and queuing other types of task.
The Reactor pattern, together with demultiplexing, event queues and the event loop, is core to how this works - a tightly coordinated dance of feeding async events into a single queue, executing them as resources free up, and then popping them off the event queue to call callbacks given by user code.
2. Module Design Patterns
I am superficially familiar with the differences between CommonJS modules and ES Modules. But I liked the explicit elaboration of 5 module definition patterns in CommonJS:
Named exports: exports.foo = () => {}
Exporting a function: module.exports = () => {}
Exporting a class: module.exports = class Foo() {}
Exporting an instance: module.exports = new Foo() which is like a singleton, except when it is not because of multiple instances of the same module.
In ES Modules, I enjoyed the explanation of "read-only live bindings", which will look weird to anyone who has never seen it and has always treated modules as stateless chunks of code:
// counter.jsexportletcount=0exportfunctionincrement(){count++}// main.jsimport{count,increment}from'./counter.js'console.log(count)// prints 0increment()console.log(count)// prints 1count++// TypeError: Assignment to constant variable!
This mutable module internal state pattern is endemic in Svelte and Rich Harris' work and I enjoy how simple it makes code look. I don't know if there are scalability issues with this pattern but so far it seems to work fine for ES Modules people.
The last important topic I enjoyed was ESM and CJS interop issues. ESM doesn't offer require, __filename or __dirname, so you have to reconstruct them if needed:
This looks innocent enough, except when you use it as async and then sync:
constreader1=createFileReader('data.txt')// asyncreader1.onDataReady(data=>{console.log(`First call: ${data}`)constreader2=createFileReader('data.txt')// syncreader2.onDataReady(data=>{console.log(`Second call: ${data}`)})})// only outputs First call - never outputs Second call
make I/O purely async by only using async APIs, using CPS, and deferring synchronous memory reads by using process.nextTick()
The same line of thinking can also be done for EventEmitter Observers as it is for Callbacks.
Callbacks should be used when a result must be returned in an asynchronous way, while events should be used when there is a need to communicate that something has happened.
You can combine both the Observer and Callback patterns, for example with the glob package which takes both a callback for its simplier, critical functionality and a .on for advanced events.
A note on ticks and microtasks:
process.nextTick sets up a microtask, which executes just after the current operation and before any other I/O
whereas setImmediate runs after ALL I/O events have been processed.
setTimeout(callback, 0) is yet another phase behind setImmediate.
4. Managing Async and Limiting Concurrency with async
It's easy to spawn race conditions and accidentally launch unlimited parallel execution bringing down the server, with Node.js. The Async library gives battle tested utilities for defining and executing these issues, in particular, queues that offer limited concurrency.
The book steps you through 4 versions of a simple web spider program to develop the motivations for requiring managing async processes and describe the subtle issues that present themselves at scale. I honestly cant do it justice, I didn't want to just copy out all the versions and discussions of the web spider project as that is a significant chunk of the book, you're just gonna have to read thru these chapters yourself.
source: devto
devToUrl: "https://dev.to/swyx/5-tils-about-node-js-fundamentals-from-the-node-js-design-patterns-book-4dh2"
devToReactions: 164
devToReadingTime: 5
devToPublishedAt: "2020-09-27T08:48:33.316Z"
devToViewsCount: 8282
title: 5 TILs about Node.js Fundamentals from the Node.js Design Patterns Book
published: true
description: 5 Things I Learned about Node.js Fundamentals from the Node.js Design Patterns Book
category: snippet
tags: Tech, TIL, Nodejs, JavaScript, Design Patterns
slug: til-node-fundamentals-design-patterns-book
canonical_url: https://swyx.io/til-node-fundamentals-design-patterns-book
I started reading Node.js Design Patterns this week. I got the Third Edition, and have not spent any time looking into what's changed from prior editions. The first 6 chapters cover fundamental knowledge, before getting into the meaty named Design Patterns, so these notes are from that first "half" of the book.
1.
libuv
and the Reactor Patternlibuv
is something I've often heard about as a low level Node.js library, but now I have a glimpse of what it does for us. As the book says:The Reactor pattern, together with demultiplexing, event queues and the event loop, is core to how this works - a tightly coordinated dance of feeding async events into a single queue, executing them as resources free up, and then popping them off the event queue to call callbacks given by user code.
2. Module Design Patterns
I am superficially familiar with the differences between CommonJS modules and ES Modules. But I liked the explicit elaboration of 5 module definition patterns in CommonJS:
exports.foo = () => {}
module.exports = () => {}
module.exports = class Foo() {}
module.exports = new Foo()
which is like a singleton, except when it is not because of multiple instances of the same module.In ES Modules, I enjoyed the explanation of "read-only live bindings", which will look weird to anyone who has never seen it and has always treated modules as stateless chunks of code:
This mutable module internal state pattern is endemic in Svelte and Rich Harris' work and I enjoy how simple it makes code look. I don't know if there are scalability issues with this pattern but so far it seems to work fine for ES Modules people.
The last important topic I enjoyed was ESM and CJS interop issues.
ESM
doesn't offerrequire
,__filename
or__dirname
, so you have to reconstruct them if needed:ESM also cannot natively import JSON, as of the time of writing, whereas CJS does. You can work around this with the
require
function from above:Did you know that? I didn't!
3. Unleashing Zalgo
APIs are usually either sync or async in Node.js, but TIL you can design APIs that are both:
This looks innocent enough, except when you use it as async and then sync:
This is because module caching in Node makes the first call async and the second call sync. izs famously called this "releasing Zalgo" in a blogpost.
You can keep Zalgo caged up by:
process.nextTick()
The same line of thinking can also be done for EventEmitter Observers as it is for Callbacks.
You can combine both the Observer and Callback patterns, for example with the
glob
package which takes both a callback for its simplier, critical functionality and a.on
for advanced events.A note on ticks and microtasks:
process.nextTick
sets up a microtask, which executes just after the current operation and before any other I/OsetImmediate
runs after ALL I/O events have been processed.process.nextTick
executes earlier, but runs the risk of I/O [starvation] (https://en.wikipedia.org/wiki/Starvation_(computer_science)) if takes too long.setTimeout(callback, 0)
is yet another phase behindsetImmediate
.4. Managing Async and Limiting Concurrency with
async
It's easy to spawn race conditions and accidentally launch unlimited parallel execution bringing down the server, with Node.js. The Async library gives battle tested utilities for defining and executing these issues, in particular, queues that offer limited concurrency.
The book steps you through 4 versions of a simple web spider program to develop the motivations for requiring managing async processes and describe the subtle issues that present themselves at scale. I honestly cant do it justice, I didn't want to just copy out all the versions and discussions of the web spider project as that is a significant chunk of the book, you're just gonna have to read thru these chapters yourself.
5. Streams
I've often commented that Streams are the best worst kept secret of Node.js. Time to learn them. Streams are more memory and CPU efficient than full buffers, but they are also more composable.
Each stream is an instance of
EventEmitter
, streaming either binary chunks or discrete objects. Node offers 4 base abstract stream classes:Readable
(where you can read in flowing (push) or paused (pull) mode)Writable
- you're probably familiar withres.write()
from Node'shttp
moduleDuplex
: both readable and writableTransform
: a special duplex stream with two other methods:_transform
and_flush
, for data transformationPassThrough
: aTransform
stream that doesnt do any transformation - useful for observability or to implement late piping and lazy stream patterns.izs recommends minipass which implement a PassThrough stream with some better features. Other useful stream utils:
Although the authors do recommend that piping and error handling be best organized with the native stream.pipeline function.
The text was updated successfully, but these errors were encountered: