Support for indefinitely-long queues.

[mbostock]

It’d be nice to create a queue simply to manage the invocation of tasks, without worry about collecting their results. This way, the queue wouldn’t grow in size based on the total number of tasks, but only on the number of pending and active tasks.

See Infinite Queue for an example (that eventually runs out of memory). But this isn’t a great example because it only defers a task when a previous task finishes, so it’d be pretty trivial to rewrite it as six independent serialized repeating tasks without needing a queue.

One possible API would be a queue.awaitNone method. Unlike queue.await and queue.awaitAll, you’d still be allowed to call queue.defer afterwards.

var q = d3.queue(1).awaitNone();

(function deferTask() {
  q.defer(task);
  setTimeout(deferTask, d3.randomExponential(1 / 60));
})();

But would queue.awaitNone take a callback that receives an error? Should an error prevent any future execution of tasks, like it does with a normal queue? (Is that what you want with an indefinitely-long queue?)

[mbostock]

I think this is sufficiently different that you’d want a different class. With queue.await and queue.awaitAll, the allowed call pattern is {defer*, await?} or {defer*, awaitAll?}. With queue.awaitNone, it is the opposite: {awaitNone, defer*}. However, there would be no way to prevent {defer*, awaitNone, defer*}, i.e., to prevent queue.defer calls before queue.awaitNone is called. And in that case, you may have accrued results from deferred tasks, so what do you do (discard the results? throw an error? repeatedly invoke the awaitNone callback with each result?).

Seems like a better approach would to be to specify this mode at the time the queue is constructed, e.g., a d3.queueVoid constructor. (Not a good name…)

This also complicates the implementation quite a bit, since now you need a linked list to store the waiting and active tasks lists, so that you can efficiently discard references to tasks as they complete. I’ve implemented that in the linked-list branch, but it increases the library size somewhat.

I think given the work involved and the non-obvious immediate value of this feature, I’m going to close this request. It might be worth revisiting in the future though!