RFC/roadmap for async fn / CancelToken integration

[machty]

Async Function / CancelTokens

Ember Concurrency (EC) predates the TC39 proposal for Cancelable Promises and Cancel Tokens, which propose new API for canceling promises and in-progress async function calls.

The two main ideas of the proposal are that:

1. Cancelation should be considered a third resolution state of promises, in addition to the preexisting "fulfilled" and "rejected" states (as opposed to treating cancelation as a "rejected" state).
2. Initiating cancelation is achieved using Cancel Tokens, which are created externally and passed into the cancelable operation, which progressively checks if the token has requested cancellation. This puts the power into the async function/operation to decide if/when it is a good time to cancel.

EC has taken a slightly different approach to cancelation since its inception, and the purpose of this mini-RFC is to propose a roadmap and future APIs for bridging the gap between EC APIs, async functions, and cancel tokens.

EC's Approach

EC: Cancellation as Third State

Similar to the cancelable promise proposal, EC has embraced the idea that cancelation is a third state; since version 0.7.0, invoking a child task in a try/catch block no longer invokes the catch block if that child task is canceled. This is a major ergonomic win since the alternative (and API prior to 0.7.0) is having to check if the "error" thrown was a Cancellation. The only remnant of "cancellation as error state" is when treating a TaskInstance (the object returned from myTask.perform()) as a promise, i.e. if you call .then() on a TaskInstance, you must handle Cancelation as an error event, since (prior to the TC39 proposal) there was no other way to model cancellation. But if you stay within the EC Task API (you avoid .then() in favor of yield someTask.perform()), you don't have to think about catching/handling cancelation.

EC: Initiating Cancellation

TC39's proposal takes a non-preemptive/cooperative approach to cancellation, such that async operations are, by default, uncancellable, but can opt into cancelability by 1. adjusting their APIs to accept cancel tokens and 2. checking for canceled tokens at specific points in the async operation.

EC takes a preemptive approach to cancellation, such that tasks (async operations) are, by default, cancellable at any point where a yield occurs, and presently there is no way for a task to prevent cancelation at an inopportune point.

Both of these approaches have their pros and cons, but in my experience maintaining EC, no one has yet complained about EC's preemptive approach, arguably because EC docs discourage explicit/external cancelation in favor of using the declarative Task Modifier API, which encourages the user to write tasks in a clean, well-structured manner that all but eliminates the need for messy explicit/external cancellation.

If EC API had adopted a more cooperative/non-preemptive approach, a task that presently looks like

showLoadingIndicator: task(function * () {
    this.set('status', "Loading");
    yield timeout(2000);
    this.set('status', "Still Loading...");
    yield timeout(2000);
    this.set('status', null);
})

might instead be written like this

  showLoadingIndicator: task(async function (cancelToken) {
    this.set('status', "Loading");
    await timeout(2000);
    cancelToken.throwIfRequested();
    this.set('status', "Still Loading...");
    await timeout(2000);
    cancelToken.throwIfRequested();
    this.set('status', null);
  })

While cancellation tokens might be the correct low-level primitive for bringing cancellation to JavaScript, I'd hope that the JavaScript community wouldn't immediately start writing APIs that forced users to 1) pass around tokens around 2) write token.throwIfRequested() after every task resumption (then again, see async.cancelToken=... below)

That said, EC doesn't really have an answer for "how can I prevent this task from being abruptly cancelled if it's in the middle of some sensitive operation?" Fortunately, I think there is a middle-ground approach that EC can take.

Problems EC needs to solve

1. With the introduction of Cancel Tokens, can EC Tasks be driven by async functions instead of generator functions? If so, how?
2. How might classic generator-function-based tasks prevent untimely cancellation? In other words, how might EC tasks opt into cooperative/non-preemptive mode?

Note: from now on this document will refer to Generator Function-Driven Ember Concurrency Tasks as GFECTs, and the async function variant as AFECTs

EC Tasks: async functions

Ideally, it should be possible to convert a GFECT to an AFECT:

  startCounting: task(function * () {
    while (true) {
      this.incrementProperty('count');
      yield timeout(1000);
    }
  })

How might the above be written using async functions?

First off, let's clarify the similarities/differences between EC tasks written with generator functions and vanilla async functions.

1. Both let you write async operations using a familiar sequential style of syntax; instead of nesting callbacks with somePromise.then(), you can use yield somePromise (EC) or await somePromise. The semantics between EC's yield and async functions' await are exactly the same.
2. As mentioned before, async functions are non-preemptive; in order for an async function to be cancellable, it must periodically check the cancel state of a token that was passed into it. In order for the infinite loop pattern above to work, the async function version must not only check for periodic cancelation, but must also pass the token along to any child async operations; otherwise, in the example above, startCounting might be canceled, but in the async function version, the timeout() won't know about the cancelation until it returns, so it too must be passed the cancelation token so that it doesn't delay the cancelation of startCounting.
3. EC tasks expose a lot of useful state about the progress of a task, from isRunning/isIdle, to the number of concurrently running task instances, etc; this is ultimately why the API for running a task is this.get('someTask').perform() and not this.someTask() -- the only way to expose this state is for a task to be an object with a .perform() method rather than a function exporting state to some other object or having properties on the function instance itself mutate over time.

So, with these constraints in mind, an async function version of the above might look like:

  myTask: task(async function(cancelToken) {
    while (true) {
      this.incrementProperty('count');
      await timeout(1000, { cancelToken });
    }
  })

This implies that:

1. The EC timeout() function now accepts additional options, one of which is cancelToken; internally it'll race between the timer promise and the token's cancelation.
2. The cancelToken passed to myTask must come from... somewhere?

The present API for invoking a task is this.get('myTask').perform(...args), and whatever arguments are supplied to perform() are passed directly to the task generator function. EC must continue to support a TaskInstance being externally cancelable via a .cancel() method, which has the following implications:

1. In order for an AFECT (async function Ember Concurrency task) to be cancelable, the async function must accept a CancelToken as an argument.
2. AFECTs can optionally ignore / disregard / not use the cancel token; the effect of this will be described below in "Overlapping Cancel States".
3. EC must have a convention for supplying cancel tokens to a Task's .perform() method so that a) vanilla async functions can invoke both AFECTs and GFECTs, passing along a CancelToken if one is present, b) EC can generate a token to pass to a task async function that will fire when taskInstance.cancel() is called, c) EC can generate a combined cancelToken using CancelToken.race([passedToken, autoGeneratedToken])

CancelToken as Last Arg

warning: wishy-washy hand-wave-y stuff

Perhaps the simplest API to impose would be for EC to always supply a cancelToken as the last argument to AFECTs, and to inspect the last argument to .perform() as a potential MaybeCancelToken.

A MaybeCancelToken describes the circumstances under which the performed task can be cancelled, and can either be/represent:

1. cancelToken: this task instance cancels upon CancelToken.race([cancelToken, autoGeneratedToken]), where autoGeneratedToken is one that fires when taskInstance.cancel() is called.
2. cancelOnlyOn(cancelToken): this EC task can only be cancelled by the cancelToken; taskInstance.cancel() will no op
3. uncancellable(): this taskInstance isn't cancellable; calling taskInstance.cancel() will essentially noop
4. (no cancel token): EC generates autoGeneratedToken to the AFECT.

Overlapping Cancel States

Imagine you have a .restartable() AFECT:

doStuff: task(async function(/* cancelToken */) {
  await uncancelableOperation();
}).restartable(),

Since .restartable() can't immediately cancel doStuff (since the cancelToken is completely disregarded), we need to decide whether:

1. EC waits for prior doStuff to complete before running a new instance
2. EC lets the taskInstances overlap

Perhaps the default should be to prevent overlap (possibly with a warning in dev mode?) and to consider the second task instance to be in a "queued" state (similar to pending task instances when using .enqueued()). Overlap can be opt in using a new .overlap() task modifier.

Note that there is actually a case with present-day EC API where this overlap can happen:

doStuff: task(function * () {
  try {
    yield someOperations();
  } finally {
    yield timeout(10000);
  }
}).restartable(),

EC presently considers this undefined behavior (though I believe it overlaps).

await.cancelToken and GFECT converse

The CancelToken spec acknowledges the verbosity of constantly running token.throwIfRequested() and offers the await.cancelToken = cancelToken API as a convenience to implicitly race each await against cancelToken.

Conversely, EC could provide a GFECT-specific API to opt into non-preemptive mode to prevent untimely cancelation:

import { preventCancel } from 'ember-concurrency';
// ...
doStuff: task(function * () {
  let cancelToken = yield preventCancel();
  
  // during this 5s period, this task instance
  // is uncancellable
  yield timeout(5000);

  // can call cancelToken.throwIfRequested()
  // for cooperative cancellation  

  return cancelableAsyncFn(cancelToken);
}).restartable(),

What about .isDestroyed / structured concurrency?

An unfortunate consequence of a non-preemptive EC API is that Ember's API is very preemptive; a component can be unrendered at any point and Ember (nor any other framework I know of) is going to wait for unfinished tasks to complete before doing so. So you probably don't want to have AFECTs on Components unless you're extremely careful not to this.set() when the async function resumes after the component potentially unrendered. Careful usage of tokens should prevent this, but without the guarantee of preemption that today's GFECTs provide, it's probably not a good idea to put AFECTs on components.

Are AFECTs worth it?

Not sure; on the one hand, it seems bad to fragment the world with two pretty semantically different approaches to implementing tasks, but on the other hand, the dichotomy between preemptive / non-preemptive tasks is unavoidable, and it may or may not be nice, when deciding how to implement a task, to think of GFECT=preemptive, AFECT=non-preemptive.

Either way, it seems like GFECTs would be benefit from an API for opting into non-preemption using a CancelToken-based API so that they might seamlessly integrate with other CancelToken-based APIs and async functions.