Skip to content

Commit

Permalink
Basic fiber-aware futures library
Browse files Browse the repository at this point in the history
This includes a packaged copy of the "futures" implementation I've been
using with fibers. The library is quite simple but makes working with
fibers much more manageable.
  • Loading branch information
laverdet committed Jul 27, 2011
1 parent f144c86 commit 95c1b61
Showing 1 changed file with 260 additions and 0 deletions.
260 changes: 260 additions & 0 deletions future.js
@@ -0,0 +1,260 @@
"use strict";
require('./fibers');
var util = require('util');
module.exports = Future;
Function.prototype.future = function() {
var fn = this;
return function() {
return new FiberFuture(fn, this, arguments);
};
};

function Future() {}

/**
* Wrap a node-style async function to return a future in place of using a callback.
*/
Future.wrap = function(fn, idx) {
idx = idx === undefined ? fn.length - 1 : idx;
return function() {
var args = Array.prototype.slice.call(arguments);
if (args.length > idx) {
throw new Error('function expects no more than '+ idx+ ' arguments');
}
var future = new Future;
args[idx] = future.resolver();
fn.apply(this, args);
return future;
};
};

/**
* Wait on a series of futures and then return. If the futures throw an exception this function
* /won't/ throw it back. You can get the value of the future by calling get() on it directly. If
* you want to wait on a single future you're better off calling future.wait() on the instance.
*/
Future.wait = function wait(/* ... */) {

// Normalize arguments + pull out a FiberFuture for reuse if possible
var futures = Array.prototype.slice.call(arguments, 0), singleFiberFuture;
for (var ii = 0; ii < futures.length; ++ii) {
if (futures[ii] instanceof Future) {
// Ignore already resolved fibers
if (futures[ii].isResolved()) {
futures.splice(ii, 1);
--ii;
continue;
}
// Look for fiber reuse
if (!singleFiberFuture && futures[ii] instanceof FiberFuture && !futures[ii].started) {
singleFiberFuture = futures[ii];
futures.splice(ii, 1);
--ii;
continue;
}
} else if (futures[ii] instanceof Array) {
// Flatten arrays
futures.splice.apply(futures, [ii, 1].concat(futures[ii]));
--ii;
continue;
} else {
throw new Error(futures[ii] + ' is not a future');
}
}

// Resumes current fiber
var fiber = Fiber.current;
if (!fiber) {
throw new Error('Can\'t wait without a fiber');
}

// Resolve all futures
var pending = futures.length + (singleFiberFuture ? 1 : 0);
function cb() {
if (!--pending) {
fiber.run();
}
}
for (var ii = 0; ii < futures.length; ++ii) {
futures[ii].resolve(cb);
}

// Reusing a fiber?
if (singleFiberFuture) {
singleFiberFuture.started = true;
try {
singleFiberFuture.return(
singleFiberFuture.fn.apply(singleFiberFuture.context, singleFiberFuture.args));
} catch(e) {
singleFiberFuture.throw(e);
}
--pending;
}

// Yield this fiber
if (pending) {
Fiber.yield();
}
};

Future.prototype = {
/**
* Return the value of this future. If the future hasn't resolved yet this will throw an error.
*/
get: function() {
if (!this.resolved) {
throw new Error('Future must resolve before value is ready');
} else if (this.error) {
throw this.error;
} else {
return this.value;
}
},

/**
* Mark this future as returned. All pending callbacks will be invoked immediately.
*/
"return": function(value) {
if (this.resolved) {
throw new Error('Future resolved more than once');
}
this.value = value;
this.resolved = true;

var callbacks = this.callbacks;
if (callbacks) {
delete this.callbacks;
for (var ii = 0; ii < callbacks.length; ++ii) {
try {
callbacks[ii](undefined, value);
} catch(ex) {
console.log(ex.stack || ex);
process.exit(1);

This comment has been minimized.

Copy link
@metamatt

metamatt Mar 20, 2013

Contributor

I'm curious if you could explain the rationale behind process.exit() here. I know this was a long time ago, but essentially this implementation of "return" and "throw" live on in node-fibers today.

In the context I'm using node-fibers and fiber-futures, I'm seeing the following behavior:

  • I have a fiber which is going to perform multiple operations
  • In that fiber, I Future.wrap some async function and call it and Future.wait for the result
  • This all goes as planned. My fiber yields inside Future.wait, then when the async function completes it calls the future's resolver which boils down to the cb function defined locally inside Future.wait, which calls fiber.run on the fiber, and the fiber keeps running.
  • So far so good. But now the fiber keeps running, and any exception it ever throws in the future (until it yields again) will be delivered back to the fiber.run call site, which is in cb, which is running under the try/catch handler here, which exits the entire process.

I feel like I'm probably greatly misunderstanding the intent or how this is supposed to be used.

To my understanding, it would be better to remove this exception handler, let the exception bubble up out of return, and land wherever it lands... OK, that's hard to pin down, where it would land (probably uncaughtException), but arguably no worse than what would happen with willy nilly exceptions in the callback style in the absence of fibers, and (also arguably) killing the whole process is overly draconian.

To be a little more specific, the actual scenario I hit that led me to investigate this is using the Mocha test framework to test some code using node-fibers and fiber-futures, and I noticed that exceptions thrown from inside a fiber often abort the whole test suite, and the cause is as described here. This might be a special case because Mocha does install an uncaughtException handler that's contextually aware (i.e. it knows what test is currently running and maps uncaught exceptions to mean a failure of that specific test), and outside of this Mocha context, throwing exceptions in an async context where you don't know who's going to catch them is not a good idea. Still, I'm curious what you'd recommend, if not the removal of this exception handler.

This comment has been minimized.

Copy link
@laverdet

laverdet Mar 20, 2013

Author Owner

Yeah sure I'm not sure I can justify it being there. Feel free to send over a PR, there's another try/catch below in "throw" that does the same thing if you want to remove that one too.

This comment has been minimized.

Copy link
@metamatt

metamatt Mar 20, 2013

Contributor

Thanks. Yup, in my codebase I've just removed both of those and will play with that for a few days, and if I remain convinced that's the right approach, I'll send a PR.

Two questions for you on the right way to make this change

  • do you buy my reasoning that while exceptions here are likely an invitation to undefined behavior (hard to define where they're going to land, and likely on uncaughtException), that's no worse than the status quo without fibers
  • is it useful to catch, log, and rethrow these exceptions (both here and in "throw"), or just remove the exception handler entirely? The argument for catch+log+rethrow, I think, is that if this is inviting undefined behavior, the logging might be a clue in the right direction. The argument against is that it's just noise and there's no reason to assume it's useful, since there's absolutely no way to reason about what code might be running inside that call to callbacksii.

Thanks again.

This comment has been minimized.

Copy link
@laverdet

laverdet Mar 20, 2013

Author Owner

Wait yeah thinking about it a little bit more I remember the reasoning. Basically the client of your code is now causing you to crash, because their handler is throwing an error. The solution here is process.nextTick(), which will safely land the exception in uncaughtException rather than in future.return.

This comment has been minimized.

Copy link
@metamatt

metamatt Mar 21, 2013

Contributor

Nice thinking. That makes sense.

This comment has been minimized.

Copy link
@metamatt

metamatt Mar 22, 2013

Contributor

I coded up and tested a PR along those lines which I'm sure you'll see before you read this.

While I've got your attention, I have a question which is only tangentially related but hopefully this is an ok place to ask it: is there a way (either with a nice supported API or with deep hackery) to get the call stack for the run() site of the currently executing fiber? All the ways I know of getting a backtrace in Node (console.trace(), Error.captureStackTrace(), debugger backtrace command), applied to a fiber, bottom out at the fiber boundary. Since the run() call site is intimately bound with the behavior of the fiber -- it's where unhandled exceptions will go, it's where execution will resume when the fiber yields or returns -- I'd like to know how to find it.

This ability would have sped up my understanding of the problem that led me to this PR, for example.

Perhaps you could point me in the direction of either a way to do this or where to learn what's involved? Or am I missing something and there's an easy way?

This comment has been minimized.

Copy link
@laverdet

laverdet Mar 22, 2013

Author Owner

Yeah I'll check out the PR soon, at first glace it looks good.

The answer to your question, like many things in JS, involves monkey-patching. This isn't a feature by default because there is an associated performance cost but you can easily add it like this:

var Fiber = require('fibers');
Fiber.prototype.run = function(run) {
  return function fn(arg) {
    this.caller = {};
    Error.captureStackTrace(this.caller, fn);
    run.call(this, arg);
  };
}(Fiber.prototype.run);

Fiber(function() {
  console.log(Fiber.current.caller.stack);
}).run();

This comment has been minimized.

Copy link
@metamatt

metamatt Mar 22, 2013

Contributor

Makes sense. Thanks again.

}
}
}
},

/**
* Throw from this future as returned. All pending callbacks will be invoked immediately.
*/
"throw": function(error) {
if (this.resolved) {
throw new Error('Future resolved more than once');
} else if (!error) {
throw new Error('Must throw non-empty error');
}
this.error = error;
this.resolved = true;

var callbacks = this.callbacks;
if (callbacks) {
delete this.callbacks;
for (var ii = 0; ii < callbacks.length; ++ii) {
try {
callbacks[ii](error);
} catch(ex) {
console.log(ex.stack || ex);
process.exit(1);
}
}
}
},

/**
* Returns whether or not this future has resolved yet.
*/
isResolved: function() {
return this.resolved === true;
},

/**
* Returns a node-style function which will mark this future as resolved when called.
*/
resolver: function() {
return function(err, val) {
if (err) {
this.throw(err);
} else {
this.return(val);
}
}.bind(this);
},

/**
* Waits for this future to resolve and then invokes a callback.
*/
resolve: function(cb) {
if (this.resolved) {
cb(this.error, this.value);
} else {
(this.callbacks = this.callbacks || []).push(cb);
}
return this;
},

/**
* Resolve only in the case of success
*/
resolveSuccess: function(cb) {
this.resolve(function(err, val) {
if (err) {
return;
}
cb(val);
});
return this;
},

/**
* Propogates errors to an another future or array of futures.
*/
proxyErrors: function(futures) {
this.resolve(function(err) {
if (!err) {
return;
}
if (futures instanceof Array) {
for (var ii = 0; ii < futures.length; ++ii) {
futures[ii].throw(err);
}
} else {
futures.throw(err);
}
});
return this;
},

/**
* Differs from its functional counterpart in that it actually resolves the future. Thus if the
* future threw, future.wait() will throw.
*/
wait: function() {
Future.wait(this);
return this.get();
},
};

/**
* A function call which loads inside a fiber automatically and returns a future.
*/
function FiberFuture(fn, context, args) {
this.fn = fn;
this.context = context;
this.args = args;
this.started = false;
var that = this;
process.nextTick(function() {
if (!that.started) {
that.started = true;
Fiber(function() {
try {
that.return(fn.apply(context, args));
} catch(e) {
that.throw(e);
}
}).run();
}
});
}
util.inherits(FiberFuture, Future);

0 comments on commit 95c1b61

Please sign in to comment.