ECMAScript Explicit Resource Management proposal integration with NodeJS APIs

[rbuckton]

The ECMAScript Explicit Resource Management proposal is currently at Stage 2 and is nearing advancement Stage 3. I'd like to get a feel from NodeJS as to whether, and how, Node might integrate the capabilities of this proposal into existing APIs. I've created a rough list of potential integration points in the proposal explainer.

What follows is a brief summary of the proposal. You can read more about the proposal in the proposal repository and the proposal specification text. If you are unfamiliar with the TC39 staging process, please see the TC39 process document.

---

The Explicit Resource Management proposal provides the ability for JavaScript users to leverage RAII ("Resource Acquisition is Initialization")-style declarations via the new using declaration:

{
  using x = acquireSomeResource();
  ...
} // 'x' is disposed when exiting Block

A using declaration is constant, block-scoped declaration (like const), and tracks its binding for disposal at the end of the containing block, regardless as to whether from a normal or abrupt completion. This allows for explicit control over the lifetime of a resource. A resource is an object with a Symbol.dispose method that, when called, should perform synchronous cleanup activities.

This kind of declaration is extremely useful when managing the scoping and lifetime of multiple resources, as it guarantees ordered cleanup of resources (excluding process/thread termination):

{
  using x = createResource1();
  using y = createResource2();
  ...
} // disposes 'y', then disposes 'x'

In addition, this proposal introduces a new DisposableStack constructor which allows a user to collect multiple disposable resources in a single disposable container, which is extremely helpful when composing disposable classes consisting of other resources:

class MyResource {
  #x;
  #y;
  #disposables;

  constructor() {
    using stack = new DisposableStack(); // 'stack' will be disposed at end of constructor
    this.#x = stack.use(createResource1()); // Add resource for '#x' to 'stack'
    this.#y = stack.use(createResource2()); // Add resource for '#y' to 'stack'

    // if either 'createResource1()' or 'createResource2()' throws, 'stack' and its contents
    // will be disposed.

    this.#disposables = stack.move(); // move resources out of 'stack' and into '#disposables'
    
    // disposes 'stack' (now empty), but not '#x', '#y', or '#disposables'
  }
  
  ...
  
  [Symbol.dispose]() {
    this.#disposables.dispose(); // disposes '#y' and '#x'
  }
}

Not all resources can be disposed of in a synchronous manner, however. This proposal also introduces Symbol.asyncDispose and an AsyncDisposableStack API for asynchronously disposed resources. These two APIs are part of the main proposal and are intended to advance to Stage 3. However, syntax for an asynchronous using declaration has been postponed to a follow-on proposal due to its possible dependency on async do expressions, though it will likely look something like the following:

async function f() {
  // body-scoped 'async using'
  async using x = someAsyncDisposableResource1();
  
  // block-scoped 'async using' requires an explicit 'await' marker:
  await async do {
    async using y = someAsyncDisposableResource2();
    ...
  }; // performs (roughly) 'await y?.[Symbol.asyncDispose]()
  
} // performs (roughly) 'await x?.[Symbol.asyncDispose]()

In summary, the following features are at Stage 2 and will be proposed for advancement to Stage 3 at the upcoming TC39 plenary:

• using declarations — block-scoped, constant, synchronous disposal.
• Symbol.dispose — built-in symbol, indicates disposal method.
• Symbol.asyncDispose — built-in symbol, indicates async disposal method.
• DisposableStack — disposable container.
• AsyncDisposableStack — async disposable container.

The following features are at Stage 2 and have been postponed temporarily due to dependency on other proposals:

• async using declarations — block-scoped, constant, asynchronous disposal.

---

Apologies for using neither the bug nor feature request templates. Neither seemed to suit the needs of this specific request.

[mscdex]

I'm not sure I see the real use case for something like this (generally speaking). As far as node goes, most things are async, which would not benefit from this kind of language change as far as I can tell. The proposed async disposal would be more useful I think.

[rbuckton]

I'm not sure I see the real use case for something like this (generally speaking). As far as node goes, most things are async, which would not benefit from this kind of language change as far as I can tell. The proposed async disposal would be more useful I think.

There are numerous things in the Node APIs that could be synchronously disposable, such as ChildProcess (via kill), Worker, via kill, dns resolvers, many things with a destroy() method, etc. In addition, synchronous disposables will be useful with worker threads and the Fixed Shape Objects/Shared Structs proposal, especially with something like an Atomics.Mutex/Atomics.ConditionVariable.

Asynchronous disposables will also be fully supported in the future as well, with syntactic support lagging behind the rest of the proposal due to a potential dependency on async do {}. As such, I'm interested in discussing the application of both sync and async disposal with respect to NodeJS APIs.

[mscdex]

There are numerous things in the Node APIs that could be synchronously disposable, such as ChildProcess (via kill), Worker, via kill, dns resolves, many things with a destroy() method, etc.

I'm still not following. Those would all be asynchronous tasks, requiring that the object still exist (e.g. when kill() is called, DNS request is responded to, etc.) beyond the enclosing braces.

[rbuckton]

I'm still not following. Those would all be asynchronous tasks, requiring that the object still exist (e.g. when kill() is called, DNS request is responded to, etc.) beyond the enclosing braces.

kill() itself is not async, though it is true that there are things that could keep a child process alive past the block. in which case a [Symbol.dispose]() method on a ChildProcess might throw if kill() returns false.

And aside from kill(), I could imagine something like RefHolder existing within the NodeJS API:

// node:util
class RefHolder {
  #value

  constructor(value) {
    this.#value = value;
    this.#value.ref();
  }

  [Symbol.dispose]() {
    const value = this.#value;
    this.#value = null;
    value?.unref();
  }
}
exports.RefHolder = RefHolder;

// main.js
import { RefHolder } from "node:util";

async function f() {
  const timer = setInterval(100, someCallback);
  using holder = new RefHolder(timer);
  ...
  // 'timer' keeps the node process alive until `f` exits, at which point `timer.unref()` is called.
}

Though, I admit that could potentially be handled by DisposableStack:

async function f() {
  using stack = new DisposableStack();
  const timer = stack.adopt(setInterval(100, someCallback), timer => timer.unref());
  ...
  // 'timer' keeps the node process alive until `f` exits, at which point `timer.unref()` is called.
}

[rbuckton]

There are numerous things in the Node APIs that could be synchronously disposable, such as ChildProcess (via kill), Worker, via kill, dns resolves, many things with a destroy() method, etc.

I'm still not following. Those would all be asynchronous tasks, requiring that the object still exist (e.g. when kill() is called, DNS request is responded to, etc.) beyond the enclosing braces.

Also, I apologize if my initial explanation was unclear. A synchronously disposed resource is one whose disposal completes synchronously (inasmuch as the user is involved). Synchronously disposed resources can still be used in async functions, since it does not matter if the intervening code between the using declaration and the end of its containing block is synchronous or asynchronous.

[mscdex]

Let's take the example of a child process. kill() would generally be called either outside of the containing scope (e.g. a helper function spawning a process and returning it) or inside a child scope, for example:

const proc = spawn('foo');
proc.stdout.on('data', (data) => {
  if (data.includes(0xFF))
    proc.kill();
});

If we try to adapt child process spawning with this new using statement, such as:

function foo() {
  using proc = spawn('foo');
  proc.stdout.on('data', (data) => {
    if (data.includes(0xFF))
      proc.kill();
  });
}

then realistically what could we possibly dispose if we're still needing to process child process output?

The synchronous child process APIs already have implicit cleanup since they block the thread and any ChildProcess object that may be created temporarily would already be unusable by the time the spawnSync()/execSync() finishes.

The same goes for all the other examples where the object use is typically asynchronous (e.g. streams and dns).

About the only use case I can really see for something like using would be for test runners, where you want to have cleanup steps between each test, and even then I'm not sure that's enough justification for adding to the language (especially considering testing libraries have done without it for a long time now and AFAIK has not been a major paint point for them).

[rbuckton]

Let's take the example of a child process. kill() would generally be called either outside of the containing scope (e.g. a helper function spawning a process and returning it) or inside a child scope, for example:

const proc = spawn('foo');
proc.stdout.on('data', (data) => {
  if (data.includes(0xFF))
    proc.kill();
});

If we try to adapt child process spawning with this new using statement, such as:

function foo() {
  using proc = spawn('foo');
  proc.stdout.on('data', (data) => {
    if (data.includes(0xFF))
      proc.kill();
  });
}

then realistically what could we possibly dispose if we're still needing to process child process output?

Yes, the example you presented would not be correctly leveraging the using declaration, because you are not waiting for proc to provide output. In this case, the lifetime of proc exceeds that of the block in which it resides. This is especially true if your intent was to return proc from this function, in which case it becomes the responsibility of the consumer to manage lifetime.

A using declaration is designed to constrain the lifetime of a resource to a specific block. A better example of this might be:

async function* foo() {
  using proc = spawn('foo');
  for await (const data of proc.stdout) {
    if (data.includes(0xff)) break;
    // do something with data
    // if an exception is thrown (and not caught) anywhere in the function, 'proc' is killed
  }
} // proc is killed if its still alive

However, there is more to the resource management than just using. Imagine, instead, that you are creating a class that spins up a background process and is used to send and receive messages for that process. Such composition might use DisposableStack to track the background process to ensure proper cleanup in the event an exception is thrown during construction, or when your class is disposed:

class Client {
  #proc;
  #logger;
  #disposables;

  constructor(logFile) {
    // The lifetime of 'stack' is scoped to the constructor body
    using stack = new DisposableStack();
    
    // The lifetimes of '#proc' and '#logger' are scoped to the lifetime of 'stack'
    this.#proc = stack.use(spawn('background_process')); // tracks '#proc' for disposal in 'stack'
    this.#logger = stack.use(new FileLogger(logFile)); // tracks '#logger' for disposal in 'stack' 

    // If 'FileLogger' is unable to create 'logFile', it may throw an Error. When the exception
    // causes the constructor to exit prematurely, 'stack' is disposed, which in turn would dispose
    // '#proc' (killing the background process)
    
    // If everything completes successfully, we move everything out of the 'stack' that is marked
    // for disposal and into a new 'DisposableStack', which we store in '#disposables'. Now when
    // 'stack' is disposed, '#proc' and '#logger' will remain active.
    
    // The lifetimes of '#proc' and '#logger' are now scoped to the lifetime of '#disposables'.
    // The lifetime of '#disposables' is scoped to the lifetime of the `Client` instance.
    this.#disposables = stack.move();
  }
  
  [Symbol.dispose]() {
    // When a 'Client' instance is disposed, its respective '#proc' and '#logger' values will
    // also be disposed, killing the process and closing the file if either remain open.
    
    this.#disposables.dispose();
  }
}

[mscdex]

Ok, so basically the non-DisposableStack case would more or less require the use of async/await to really be useful for synchronous disposal.

[rbuckton]

There are many use cases, and not all would necessitate async/await. For this case (ChildProcess/Worker), however, you would need to use it.

[bnoordhuis]

It seems like the discussion here ran its course? If not, I can convert it to a GH discussion.