Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Browse files

first commit

  • Loading branch information...
commit ac108199bf34a11a8b7d43e71bedd1405b0d48dd 0 parents
@pgte authored
Showing with 138 additions and 0 deletions.
  1. +15 −0 article.json
  2. +123 −0 article.md
15 article.json
@@ -0,0 +1,15 @@
+{
+ "title": "Asynchronous Iterations",
+ "description": "How to synchronize asynchronous function calls",
+ "date": "Fri Aug 26 2011 03:08:50 GMT-0700 (PST)",
+ "tags": ["flow", "asynchronous"],
+ "authors": [
+ { "name": "Pedro Teixeira",
+ "url": "https://github.com/pgte"
+ }
+ ],
+ "difficulty": 6,
+ "categories": [
+ ["Asynchronous Programming"]
+ ]
+}
123 article.md
@@ -0,0 +1,123 @@
+# Asynchronous Iteration
+
+Some patterns are hard to grasp, specially when programming asynchronously like you have to when you're doing I/O on Node. For example, let's suppose you had to program the following routine:
+
+> Insert a collection of objects on the database and then, when finished, call a callback.
+
+If you had to write this in a synchronous fashion you would do something like this:
+
+ function insertCollection(collection) {
+ for(var i = 0; i < collection.length; i++) {
+ db.insert(collection[i]);
+ }
+ }
+
+## Parallel Execution
+
+Since we are using Node, `db.insert` performs I/O and is therefore asynchronous. We have to turn this into an asynchronous function.
+
+This following implementation is __wrong__:
+
+ function insertCollection(collection, callback) {
+ for(var i = 0; i < collection.length; i++) {
+ db.insert(collection[i], function(err) {
+ if (err) { throw err; }
+ });
+ }
+ callback();
+ }
+
+The problem with this one is obvious: `callback` is called right after launching all the `db.inserts` on the background, not leaving them a chance to finish. By the time callback gets called, none of the inserts has terminated.
+
+Another approach (__which is also wrong__) would be this one:
+
+ function insertCollection(collection, callback) {
+ for(var i = 0; i < collection.length; i++) {
+ (function(i) {
+ db.insert(collection[i], function(err) {
+ if (err) { return callback(err); }
+ if (i == (collection.length - 1)) {
+ callback();
+ }
+ });
+ })(i);
+ }
+ }
+
+There is some temptation to think *we have to call when the last insert calls back*, but this is plain wrong. The first insert can still be executing when the last one callsback. You never know the order in which I/O operations finish.
+
+A safer approach is to do something like this:
+
+ function insertCollection(collection, callback) {
+ var inserted = 0;
+ for(var i = 0; i < collection.length; i++) {
+ db.insert(collection[i], function(err) {
+ if (err) {
+ return callback(err);
+ }
+ if (++inserted == collection.length) {
+ callback();
+ }
+ });
+ }
+ }
+
+You should only callback when __all__ of the inserts have terminated.
+
+This approach is still unsafe, since there is the chance that, if an error occurs, the `callback` function gets called more than once. We should put a guard against that.
+
+Also, we are calling an external function that may throw an error synchronously: we better also guard agains that. A good approach is to centralize the control on a function we can name `done`:
+
+ function insertCollection(collection, callback) {
+ var inserted = 0,
+ terminated = false;
+
+ function done() {
+ if (! terminated) {
+ terminated = true;
+ callback.apply({}, arguments);
+ }
+ }
+
+ for(var i = 0; i < collection.length; i++) {
+ try {
+ db.insert(collection[i], function(err) {
+ if (err) {
+ return done(err);
+ }
+ if (++inserted == collection.length) {
+ done();
+ }
+ });
+ } catch(err) {
+ done(err);
+ }
+
+ }
+ }
+
+This `done` function assures that we are not invoking the `callback` function more than once.
+
+## Serialized Execution
+
+Sometimes you may need to control the flow and / or the order of the execution. You may want the inserts to be perfectly ordered in this case, or you may want to stop inserting if an error occurs so you can recover more easily. Or you may even want to serialize your database requests so you can spare your database from being hit too hard.
+
+If that's the case, you can do something like this:
+
+ function insertCollection(collection, callback) {
+ var coll = collection.slice(0); // clone collection
+
+ (function insertOne() {
+ var record = coll.shift(); // get the first record of coll and reduce coll by one
+ db.insert(record, function(err) {
+ if (err) { return callback(err); }
+ if (coll.length == 0) {
+ callback();
+ } else {
+ insertOne();
+ }
+ }
+ })();
+ }
+
+Here we are declaring the function `insertOne` and then invoking it immediately after. When an insert is finished we call it again unless we don't have anything else to insert, in which case we callback saying the operation terminated.
Please sign in to comment.
Something went wrong with that request. Please try again.