Skip to content
Browse files

Added q/util and q/queue, with step documentation.

  • Loading branch information...
1 parent ec28073 commit 0592935fc2fa469aaf53de5a4dc18bc40b53ced7 @kriskowal kriskowal committed Jan 5, 2011
Showing with 671 additions and 18 deletions.
  1. +16 −0 CHANGES
  2. +257 −1 README
  3. +13 −0 examples/delay.js
  4. +43 −0 examples/shallow-deep.js
  5. +16 −0 examples/step1.js
  6. +17 −0 examples/step2.js
  7. +22 −0 examples/step3.js
  8. +0 −15 examples/test.js
  9. +5 −1 lib/q.js
  10. +57 −0 lib/q/queue.js
  11. +220 −0 lib/q/util.js
  12. +5 −1 package.json
View
16 CHANGES
@@ -1,4 +1,20 @@
+Deprecations:
+ - I plan in the next backward-incompatible revision to move
+ the `defined` method from `q` to `q/util`. Accordingly,
+ please begin using the version of `defined` exported from
+ the latter module.
+
+0.2.1
+ - The `resolve` and `reject` methods of `defer` objects now
+ return the resolution promise for convenience.
+ - Added `q/util`, which provides `step`, `delay`, `shallow`,
+ `deep`, and three reduction orders.
+ - Added `q/queue` module for a promise `Queue`.
+ - Added `q-comm` to the list of compatible libraries.
+ - Deprecated `defined` from `q`, with intent to move it to
+ `q/util`.
+
0.2.0 - BACKWARD INCOMPATIBLE
- Changed post(ref, name, args) to variadic
post(ref, name, ...args). BACKWARD INCOMPATIBLE
View
258 README
@@ -15,14 +15,154 @@ For Node:
$ node examples/test.js
-The Q Ecosystem:
+APPLIED INTRODUCTION
+--------------------
+
+Skipping past what an asynchronous promise is and how to use
+them directly for a moment, compare the usage of this
+library to Tim Caswell's excellent `step` library.
+
+ https://github.com/creationix/step
+
+The `q/util` module, included here, provides a `step`
+function similar to Tim's. It takes any number of functions
+as arguments and runs them in serial order. Each function
+returns a promise to complete its step. When that promise
+is deeply resolved (meaning there are no more unfinished
+jobs in its object graph), the resolution is passed as the
+argument to the next step.
+
+ var Q = require("q/util");
+ var FS = require("q-fs");
+
+ Q.step(
+ function () {
+ return FS.read(__filename);
+ // __filename is NodeJS-specific
+ },
+ function (text) {
+ return text.toUpperCase();
+ },
+ function (text) {
+ console.log(text);
+ }
+ );
+
+In Node, this example reads itself and writes itself out in
+all capitals. Notice that any value can be treated as an
+already resolved promise, since the second and third steps
+return a string and `undefined` respectively.
+
+You can also perform actions in parallel. This example
+reads two files at the same time and returns an array of
+promises for the results. Since the second step has more
+than one argument, the results array gets unpacked into the
+variadic arguments.
+
+ var Q = require("q/util");
+ var FS = require("q-fs");
+
+ Q.step(
+ function () {
+ return [
+ FS.read(__filename),
+ FS.read("/etc/passwd")
+ ];
+ },
+ function (self, passwd) {
+ console.log(__filename + ':', self.length);
+ console.log('/etc/passwd:', passwd.length);
+ }
+ );
+
+The number of tasks performed in each step is not limited.
+You can just as well return an array of promises of
+indefinite length. This example reads all of the files in
+the same directory as the program and notes the length of
+each.
+
+ var Q = require("q/util");
+ var FS = require("q-fs");
+
+ Q.step(
+ function () {
+ return FS.list(__dirname);
+ },
+ function (fileNames) {
+ return fileNames.map(function (fileName) {
+ return [fileName, FS.read(fileName)];
+ });
+ },
+ function (files) {
+ files.forEach(function (pair) {
+ var fileName = pair[0];
+ var file = pair[1];
+ console.log(fileName, file.length);
+ });
+ }
+ );
+
+All of these examples use the `q-fs` module, which is
+packaged separately. You can try these programs,
+`step{1,2,3}.js` in the `examples/` directory of this
+package.
+
+When working with promises, exceptions are generally only
+thrown to indicate programmer errors. Promise-returning
+API`s generally `reject` their promises to indicate that the
+promise will never be resolved/fulfilled. As such, the
+above programs will terminate when the first step rejects a
+the returned promise, which can happen if there is an error
+while reading or listing a file. The rejection can be
+observed because the `step` function returns a `promise`
+that will be eventually resolved by the return value of the
+last step.
+
+ var completed = Q.step(...);
+
+We use the `when` method to observe either the resolution or
+the rejection of the promise.
+
+ Q.when(completed, function callback(completion) {
+ // ok
+ }, function errback(reason) {
+ // error
+ });
+
+If a rejection is not explicitly observed, it gets
+implicitly forwarded to the promise returned by `when`.
+
+This is the implementation of `step` in terms of the `when`
+method and the `deep` resolver method.
+
+ function step() {
+ return Array.prototype.reduce.call(
+ arguments,
+ function (value, callback) {
+ return Q.when(deep(value), function (value) {
+ if (callback.length > 1) {
+ return callback.apply(undefined, value);
+ } else {
+ return callback(value);
+ }
+ });
+ },
+ undefined
+ );
+ }
+
+
+The Q Ecosystem
+---------------
q-fs https://github.com/kriskowal/q-fs
basic file system promises
q-http https://github.com/kriskowal/q-http
http client and server promises
q-util https://github.com/kriskowal/q-util
promise control flow and data structures
+ q-comm https://github.com/kriskowal/q-comm
+ remote object communication
teleport https://github.com/gozala/teleport
browser-side module promises
...
@@ -31,6 +171,7 @@ The Q Ecosystem:
THE HALLOWED API
+----------------
when(value, callback_opt, errback_opt)
@@ -238,11 +379,126 @@ error(reason)
when calls where you want to trap the error clause and throw it
instead of attempting a recovery or forwarding.
+
enqueue(callback Function)
Calls "callback" in a future turn.
+THE UTIL MODULE
+---------------
+
+The Q utility module exports all of the Q module's API but
+additionally provides the following functions.
+
+ var Q = require("q/util");
+
+
+step(...functions)
+
+ Calls each step function serially, proceeding only when
+ the promise returned by the previous step is deeply
+ resolved (see: `deep`), and passes the resolution of the
+ previous step into the argument or arguments of the
+ subsequent step.
+
+ If a step accepts more than one argument, the resolution
+ of the previous step is treated as an array and expanded
+ into the step's respective arguments.
+
+ `step` returns a promise for the value eventually
+ returned by the last step.
+
+
+delay(timeout, eventually_opt)
+
+ Returns a promise for the eventual value after `timeout`
+ miliseconds have elapsed. `eventually` may be omitted,
+ in which case the promise will be resolved to
+ `undefined`. If `eventually` is a function, progress
+ will be made by calling that function and resolving to
+ the returned value. Otherwise, `eventually` is treated
+ as a literal value and resolves the returned promise
+ directly.
+
+
+shallow(object)
+
+ Takes any value and returns a promise for the
+ corresponding value after all of its properties have
+ been resolved. For arrays, this means that the
+ resolution is a new array with the corresponding values
+ for each respective promise of the original array, and
+ for objects, a new object with the corresponding values
+ for each property.
+
+
+deep(object)
+
+ Takes any value and returns a promise for the
+ corresponding value after all of its properties have
+ been deeply resolved. Any array or object in the
+ transitive properties of the given value will be
+ replaced with a new array or object where all of the
+ owned properties have been replaced with their
+ resolution.
+
+
+reduceLeft(values, callback, basis, this)
+reduceRight(values, callback, basis, this)
+reduce(values, callback, basis, this)
+
+ The reduce methods all have the signature of `reduce` on
+ an ECMAScript 5 `Array`, but handle the cases where a
+ value is a promise and when the return value of the
+ accumulator is a promise. In these cases, each reducer
+ guarantees that progress will be made in a particular
+ order.
+
+ `reduceLeft` guarantees that the callback will be called
+ on each value and accumulation from left to right after
+ all previous values and accumulations are fully
+ resolved.
+
+ `reduceRight` works similarly from right to left.
+
+ `reduce` is opportunistic and will attempt to accumulate
+ the resolution of any previous resolutions. This is
+ useful when the accumulation function is associative.
+
+
+THE QUEUE MODULE
+----------------
+
+The `q/queue` module provides a `Queue` object where
+infinite promises for values can be dequeued before they are
+enqueued.
+
+
+put(value)
+
+ Places a value on the queue, resolving the next gotten
+ promise in order.
+
+get()
+
+ Returns a promise for the next value from the queue. If
+ more values have been enqueued than dequeued, this value
+ will already be resolved.
+
+close(reason_opt)
+
+ Causes all promises dequeued after all already enqueued
+ values have been depleted will be rejected for the given
+ reason.
+
+closed
+
+ A promise that, when resolved, indicates that all
+ enqueued values from before the call to `close` have
+ been dequeued.
+
+
Copyright 2009, 2010 Kristopher Michael Kowal
MIT License (enclosed)
View
13 examples/delay.js
@@ -0,0 +1,13 @@
+
+var Q = require("q");
+
+var delay = function (delay) {
+ var d = Q.defer();
+ setTimeout(d.resolve, delay);
+ return d.promise;
+};
+
+Q.when(delay(1000), function () {
+ console.log('Hello, World!');
+});
+
View
43 examples/shallow-deep.js
@@ -0,0 +1,43 @@
+
+var Q = require("./util");
+
+var eventually = function (eventually) {
+ return Q.delay(1000, eventually);
+};
+
+var x = Q.shallow([1, 2, 3].map(eventually));
+Q.when(x, function (x) {
+ console.log(x);
+});
+
+var x = Q.shallow({
+ "a": eventually(10),
+ "b": eventually(20)
+});
+Q.when(x, function (x) {
+ console.log(x);
+});
+
+var x = Q.shallow({
+ "a": [1, 2, 3].map(eventually)
+});
+Q.when(x, function (x) {
+ console.log(x);
+});
+
+var x = Q.deep({
+ "a": [1, 2, 3].map(eventually)
+});
+Q.when(x, function (x) {
+ console.log(x);
+});
+
+var x = Q.deep([
+ {
+ "a": [1, 2, 3].map(eventually)
+ }
+]);
+Q.when(x, function (x) {
+ console.log(x);
+});
+
View
16 examples/step1.js
@@ -0,0 +1,16 @@
+
+var Q = require("q/util");
+var FS = require("q-fs");
+
+Q.step(
+ function () {
+ return FS.read(__filename);
+ },
+ function (text) {
+ return text.toUpperCase();
+ },
+ function (text) {
+ console.log(text);
+ }
+);
+
View
17 examples/step2.js
@@ -0,0 +1,17 @@
+
+var Q = require("q/util");
+var FS = require("q-fs");
+
+Q.step(
+ function () {
+ return [
+ FS.read(__filename),
+ FS.read("/etc/passwd")
+ ];
+ },
+ function (self, passwd) {
+ console.log(__filename + ':', self.length);
+ console.log('/etc/passwd:', passwd.length);
+ }
+);
+
View
22 examples/step3.js
@@ -0,0 +1,22 @@
+
+var Q = require("q/util");
+var FS = require("q-fs");
+
+Q.step(
+ function () {
+ return FS.list(__dirname);
+ },
+ function (fileNames) {
+ return fileNames.map(function (fileName) {
+ return [fileName, FS.read(fileName)];
+ });
+ },
+ function (files) {
+ files.forEach(function (pair) {
+ var fileName = pair[0];
+ var file = pair[1];
+ console.log(fileName, file.length);
+ });
+ }
+);
+
View
15 examples/test.js
@@ -1,15 +0,0 @@
-
-var Q = require("q");
-
-var delay = function (delay) {
- var deferred = Q.defer();
- setTimeout(deferred.resolve, delay);
- return deferred.promise;
-}
-
-var hi = Q.when(delay(1000), function () {
- return "Hello, World!";
-});
-
-Q.when(hi, console.log);
-
View
6 lib/q.js
@@ -138,13 +138,14 @@ function defer() {
forward.apply(undefined, [value].concat(pending[i]));
}
pending = undefined;
+ return value;
};
return {
"promise": freeze(promise),
"resolve": resolve,
"reject": function (reason) {
- resolve(reject(reason));
+ return resolve(reject(reason));
}
};
}
@@ -280,6 +281,9 @@ function ref(object) {
},
"post": function (name /*...args*/) {
var args = Array.prototype.slice.call(arguments, 1);
+ var method = object[name];
+ if (!method) throw new Error("No such method " + name + " on object " + object);
+ if (!method.apply) throw new Error("Property " + name + " on object " + object + " is not a method");
return object[name].apply(object, args);
}
}, undefined, function valueOf() {
View
57 lib/q/queue.js
@@ -0,0 +1,57 @@
+
+// Copyright (C) 2010 Google Inc.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+/**
+ * An infinite queue where (promises for) values can be dequeued
+ * before they are enqueued.
+ *
+ * Based on a similar example in Flat Concurrent Prolog, perhaps by
+ * Ehud (Udi) Shapiro.
+ *
+ * @author Mark S. Miller
+ */
+
+exports.Queue = Queue;
+function Queue() {
+ var ends = Q.defer();
+ var closed = Q.defer();
+ return {
+ "put": function (value) {
+ var next = Q.defer();
+ ends.resolve({
+ "head": value,
+ "tail": next.promise
+ });
+ ends.resolve = next.resolve;
+ },
+ "get": function () {
+ var result = Q.get(ends.promise, "head");
+ ends.promise = Q.get(ends.promise, "tail");
+ return Q.when(result, null, function (reason) {
+ closed.resolve();
+ return Q.reject(reason);
+ });
+ return result;
+ },
+ "closed": closed.promise,
+ "close": function (reason) {
+ var end = {"head": Q.reject(reason)};
+ end.tail = end;
+ ends.resolve(end);
+ return closed.promise;
+ }
+ };
+}
+
View
220 lib/q/util.js
@@ -0,0 +1,220 @@
+
+var Q = require("q");
+
+var has = Object.prototype.hasOwnProperty;
+
+// replicate the Q API
+for (var name in Q) {
+ if (has.call(Q, name)) {
+ exports[name] = Q[name];
+ }
+}
+
+/**
+ * Calls each step function serially, proceeding only when
+ * the promise returned by the previous step is deeply
+ * resolved (see: `deep`), and passes the resolution of the
+ * previous step into the argument or arguments of the
+ * subsequent step.
+ *
+ * If a step accepts more than one argument, the resolution
+ * of the previous step is treated as an array and expanded
+ * into the step's respective arguments.
+ *
+ * `step` returns a promise for the value eventually
+ * returned by the last step.
+ *
+ * @param {Array * f(x):Promise}
+ * @returns {Promise}
+ */
+exports.step = function () {
+ return Array.prototype.reduce.call(
+ arguments,
+ function (value, callback) {
+ return Q.when(deep(value), function (value) {
+ if (callback.length > 1) {
+ return callback.apply(undefined, value);
+ } else {
+ return callback(value);
+ }
+ });
+ },
+ undefined
+ );
+};
+
+/**
+ * Returns a promise for the eventual value after `timeout`
+ * miliseconds have elapsed. `eventually` may be omitted,
+ * in which case the promise will be resolved to
+ * `undefined`. If `eventually` is a function, progress
+ * will be made by calling that function and resolving to
+ * the returned value. Otherwise, `eventually` is treated
+ * as a literal value and resolves the returned promise
+ * directly.
+ *
+ * @param {Number} timeout
+ * @returns {Promise * undefined} a promise for `undefined`
+ * that will resolve after `timeout` miliseconds.
+ */
+exports.delay = function (timeout, eventually) {
+ var deferred = Q.defer();
+ setTimeout(deferred.resolve, timeout);
+ if (typeof eventually === "undefined") {
+ return deferred.promise;
+ } else if (typeof eventually === "function") {
+ return Q.when(deferred.promise, eventually);
+ } else {
+ return Q.when(deferred.promise, function () {
+ return eventually;
+ });
+ }
+};
+
+/**
+ * Takes any value and returns a promise for the
+ * corresponding value after all of its properties have
+ * been resolved. For arrays, this means that the
+ * resolution is a new array with the corresponding values
+ * for each respective promise of the original array, and
+ * for objects, a new object with the corresponding values
+ * for each property.
+ */
+exports.shallow = shallow;
+function shallow(object) {
+ return consolidate(object, function (value) {
+ return value;
+ });
+}
+
+/**
+ * Takes any value and returns a promise for the
+ * corresponding value after all of its properties have
+ * been deeply resolved. Any array or object in the
+ * transitive properties of the given value will be
+ * replaced with a new array or object where all of the
+ * owned properties have been replaced with their
+ * resolution.
+ */
+exports.deep = deep;
+function deep(object) {
+ return consolidate(object, deep);
+}
+
+function consolidate(object, deep) {
+ return Q.when(object, function (object) {
+ if (object === null || object === undefined) {
+ return object;
+ } else if (Array.isArray(object)) {
+ return reduceLeft(object, function (values, value) {
+ return Q.when(deep(value), function (value) {
+ return values.concat([value]);
+ });
+ }, []);
+ } else if (typeof object === "object") {
+ var result = {};
+ var synchronize;
+ for (var name in object) {
+ if (has.call(object, name)) {
+ (function (name, value) {
+ synchronize = Q.when(synchronize, function () {
+ return Q.when(deep(value), function (value) {
+ result[name] = value;
+ });
+ });
+ })(name, object[name]);
+ }
+ }
+ return Q.when(synchronize, function () {
+ return result;
+ });
+ } else {
+ return object;
+ }
+ });
+}
+
+/**
+ * The reduce methods all have the signature of `reduce` on
+ * an ECMAScript 5 `Array`, but handle the cases where a
+ * value is a promise and when the return value of the
+ * accumulator is a promise. In these cases, each reducer
+ * guarantees that progress will be made in a particular
+ * order.
+ *
+ * `reduceLeft` guarantees that the callback will be called
+ * on each value and accumulation from left to right after
+ * all previous values and accumulations are fully
+ * resolved.
+ */
+var reduceLeft = exports.reduceLeft = reducer('reduce');
+
+/**
+ * `reduceRight` works similarly to `reduceLeft` but from
+ * right to left.
+ */
+var reduceRight = exports.reduceRight = reducer('reduceRight');
+
+function reducer(direction) {
+ return function (values, callback, basis, that) {
+ return Q.when(that, function (that) {
+ return Q.when(values, function (values) {
+ return values[direction](function (values, value) {
+ return Q.when(values, function (values) {
+ return Q.when(value, function (value) {
+ return callback.call(that, values, value);
+ });
+ });
+ }, basis);
+ });
+ });
+ }
+}
+
+/**
+ * `reduce` is opportunistic and will attempt to accumulate
+ * the resolution of any previous resolutions. This is
+ * useful when the accumulation function is associative.
+ */
+exports.reduce = reduce;
+function reduce(values, callback, accumulator, that) {
+ var accumulators = [];
+ if (arguments.length > 2)
+ accumulators.push(accumulator);
+ var reduction = Q.defer();
+
+ Q.when(Q.shallow(UTIL.map(values, function (value) {
+ return Q.when(value, function (value) {
+ accumulators.push(value);
+ reduce();
+ });
+ })), function () {
+ // assert accumulators.length == 1
+ reduction.resolve(accumulators.shift());
+ }, function (reason) {
+ reduction.reject({
+ "child": reason
+ });
+ });
+
+ function reduce() {
+ if (accumulators.length < 2)
+ return;
+ Q.when(callback.call(
+ that,
+ accumulators.shift(),
+ accumulators.shift()
+ ), function (value) {
+ accumulators.push(value);
+ reduce();
+ }, function (reason) {
+ reduction.reject({
+ "message": "error in reduction",
+ "child": reason
+ });
+ });
+ }
+
+ return reduction.promise;
+}
+
View
6 package.json
@@ -1,7 +1,7 @@
{
"name": "q",
"description": "defer/when-style promises (CommonJS/Promises/B)",
- "version": "0.2.0",
+ "version": "0.2.1",
"homepage": "http://github.com/kriskowal/q/",
"author": "Kris Kowal <kris@cixar.com> (http://github.com/kriskowal/)",
"contributors": [
@@ -19,6 +19,10 @@
}
],
"main": "lib/q.js",
+ "modules": {
+ "util": "./lib/q/util.js",
+ "queue": "./lib/q/queue.js"
+ },
"repository": {
"type": "git",
"url": "http://github.com/kriskowal/q.git"

0 comments on commit 0592935

Please sign in to comment.
Something went wrong with that request. Please try again.