Installation

Promises relies on the FuncTools library. In order to hack on Promises or use it run the following:

$ git clone git://github.com/ShiftSpace/promises.git
$ cd promises
$ git submodule init
$ git submodule update

Rationale

The following is written with MooTools in mind however the Promises
library has few requirments, it would be quite simple to port it to
another JavaScript framework as long as the framework supports
user defined events.

To understand the rationale behind Promises let’s look at a few
problems that crop up in many AJAX programs. Imagine that you have six
resources on your web service: a, b, c, d, e, f. For simplicity’s sake
let’s imagine that these are just json files with the following
format. For example a.json might look something like the following:

{data: "a"}

Now suppose that you want to make requests for all 6 resources and you
would like to produce the string “abcdef”. If you’ve done some AJAX
programming the following requirements should be pretty clear.

1. You need some kind of accumulator to hold the final string.
2. The requests must be made in correct order.

You code will look something like the following:

function get(rsrc, callback) 
{
  new Request({
    method: 'get',
    url: rsrc+'.json',
    onComplete: callback
  }).send();
}

var result = '';
get('a', function(response) {
  result += JSON.decode(response).data;
  get('b', function(response) {
    result += JSON.decode(response).data;
    get('c', function(response) {
      result += JSON.decode(response).data;
      get('d', function(response) {
        result += JSON.decode(response).data;
        get('e', function(response) {
          result += JSON.decode(response).data;
          get('f', function(response) {
            result += JSON.decode(response).data;
            console.log(result);
          });
        });
      });
    });
  });
});

Wow this is not very pretty. Before moving on, consider how we would
write this logic if we were not making asynchronous requests:

      
function get(rsrc)
{
  return rsrc;    
}

function add(a, b)
{
  return a + b;
}

var result = add(get("a"), 
                 add(get("b"), 
                     add(get("c"), 
                         add(get("d"), 
                             add(get("e"), get("f"))))));

Clearly you don’t need such a long winded style if you just want to
add some characters together- you can concatenate strings with +. But
let’s pretend for now that add is a function that combines the value
of two resources in an interesting way. While this is
considerably easier on the eyes, for the most part this doesn’t look
so much different structurally than the request version.

Some might argue that you should handle this scenario with a bulk
operations API. I think this is like arguing that an OpenGL FFI should
support bulk operations- you’re simply moving the complexity to the
server-side code. It is not being eliminated.

Another approach is that you could use something like Clientcide’s
Request.Queue, but this also doesn’t really do what we want. We have
to push Requests onto the queue in the order we want them to
execute. And we still need to create an accumulator via a
closure.

Compare this to the example above where the order of JavaScript
function argument evaluation ensures the proper order and no
accumulator via a closure is required.

Traditional AJAX requests don’t compose

For a second example let’s examine something a little more
sophisticated, this time let’s consider how we would like to write the
program logic- that is, not in terms of requests. Assuming get and add
are as before:

var a = get("a");
var value1 = add(add(a, get("b")), add(add(get("c"), get("d")), a));
var value2 = add(a, get("f"));

In this example, value1 should be “abcda”, value2 should be “af”.

Converting this to a sequence of requests presents some
challenges. The simple (and least desirable solution) is to wait for
all the resources to load and put the final result together in last resource
request. Why is this bad?

1. This code cannot be decomposed. You have to use it as one big
   frozen block of functionality. Meh.
2. The computation of value2 must wait for value1! Meh.

You could wait for value a and then put the computation of value1 and
value2 inside of the callback for the request for a. But suppose you
decide that you want to use the value of b in the computation of
value2 as well. Now you need to wait for a and b. In MooTools this
can be accomplished with the Groups class. But this also means you’ll
need to change the get function to not call Request.send().

Why is that? You do not want to miss that onComplete event under any
circumstances- the event handler has to be in place before you make
the request. So get can only return the Request object, it cannot send
it. You now have to send the requests yourself after you’ve set up your
Group on all the requests that you care about.

But say that while you are prototyping you realize the computation of the
value2, “af”, is needed elsewhere. Damn. You can reuse the code you’ve
written here but not without duplicating code- the waiting logic for a and b
that is shared by both the computation of value1 and value2. Wow, as your
requirements become more and more complex your code will become harder
and harder to maintain.

This clearly illustrates that traditional style AJAX requests cannot
be composed.

Which leaves one to wonder? Can anything be done about this or are we
doomed to write complex hard to maintain spaghetti callback
code. Before we consider the solution take note of yet another downside
to callback oriented AJAX requests.

Suppose a developer using your library wishes to extend your code with
the intention of making their own code extendable. Because your
function interface provides a callback to deliver the result of the
computation, his/her code will also have to provide a callback
argument in his/her function interface in order to deliver a result to
users of their library extension. Damn, callbacks on function interfaces
are viral!

Hmm…

Now that we understand the problem space, how does one go about
solving this? Is it just impossible for requests to be written in such a
way that we can compose them?

A JavaScript-y Solution

While there have been several attempts to do such a thing many of the
solutions require the programmer to adopt a library that is painfully
“non-webby” or one that transforms your JavaScript into a fairly
undebuggable form- Flapjax and Narrative Javascript are two examples.
This isn’t to dog on them, they are both very, very cool projects
with a lot to learn from (I’d never heard of continuations until
Narrative JavaScript).

Here’s where Promises comes in. Promises provide a simple
JavaScript-ish solution to this problem that is easy to port to other
JavaScript libraries (Prototype, jQuery, etc.). You’ll need to rethink
how you organize your AJAX logic but I think that it presents
significant advantages for code that is request heavy.

So what are Promises? Promises are objects that represent unrealized
values. They encapsulate the request. When you request the value of
resource foo, a Promise will fetch that resource and fire an event
realized with its realized value when the request is complete.

Now by itself this doesn’t sound very useful. It’s just a wrapper around
Request that doesn’t do anything. This is where decorators come in. I
suggest reading the following article about decorators in
Python- [http://www.artima.com/weblogs/viewpost.jsp?thread=240808].
If you’re not familiar with them, the following may not make much
sense. They aren’t complicated so I suggest going over that first
before continuing.

Basically a decorator simply wraps a functions with new functionality.
Decorators can be used for handy things like type checking arguments
to a function or JSON.encoding all of a function’s return values.
They are about as a powerful a construct as you can have without
full blown Lisp macros ;)

In addition to the Promise class the Promises library provides a
decorator called promise that does a few interesting things:

1. If a function returns a Request instance it will automatically be
   converted into a Promise. This does not mean a request will be
   made. Promises are “lazy”. They will not be realized unless they
   cross the function interface of another promise decorated function.
2. A function’s arguments will be checked to see if any unrealized Promises
   were passed in. If so, the function blocks until those Promises have
   been realized. If a realized Promise returns an unrealized Promise
   as a value, the function will continue to block.
3. This means all arguments inside of a promise decorated function will
   be guaranteed to be realized. The upside is that you call these
   functions with non-Promise arguments and they will work just fine!
4. Reread 3. This means you can sanely unit test AJAX code!

A Silly Example

Enough talk, here’s an example:

var get = function(rsrc) 
{
  return new Request({
    method: 'get',
    url: rsrc+'.json'
  });
}.decorate(promise)

var fnA = function(a, b)
{
  var p = get(b);
  return p.op(function(value) { return a + value; });
}.decorate(promise)

var show = function show(arg)
{
  console.log(arg);
}.decorate(promise)

show(fnA(fnA(fnA(fnA(fnA(get('a'), 'b'), 'c'), 'd'), 'e'), 'f'))

Notice we have no callbacks. Yes this code makes 6 simultaneous
Requests. We don’t have to queue them. When we run this code it will
print “abcdef” to the JavaScript console. In traditional AJAX
programming there is just no way to express the above succintly.

The get function, because it has been decorated with promise, will
convert the request into a Promise instance. It will not call the
request until this instance crosses the function interface of a
function that has also been decorated by promise- fnA for example.

fnA(get('a'), 'b')

This sends the resulting Promise instance of get('a') to
fnA which will force it to be realized. However since fnA is now blocking
it returns a Promise for its final return value! Note that the
above logic means that a lot of Promise instances are generated. Note
that even when the result of fnA is realized, that realized value is
yet another Promise. Yikes!

It’s OK. The promise decorator is smart enough to see that when the
blocking function finally returned a value this value is itself an
unrealized Promise, it will continue to block until it can get a real
value. Whew.

Also note the use of Promise.op. Sometimes you’ll need to modify the
value of a Promise. You of course can’t do this directly, there is no
value yet! However Promise.op lets you queue up operations to be
applied to the value. The quick facts:

1. Promise.op allows you to modify the value of a Promise.
2. If a Promise is unrealized Promise.op returns the Promise.
3. If realized, Promise.op returns the value after the op is applied.
4. Before the Promise fires its realized event all ops that were
   queued up will be applied first.

This might seem overly complex, but consider the following real world
example.

Real World Example

Without Promises

var MyClass = new Class({

  initialize: function(el, options)
  {
    this.setOptions(...);
    
    if(this.options.cssFiles)
    {
      var reqs = this.options.cssFiles.map(this.getFile.bind(this));
      var group = new Group(reqs);
      group.addEvent('onComplete', function() {
        this.show(reqs.map(function(req) { return req.responseText; }));
      }.bind(this);
      reqs.each(function(req) { req.send(); });
    }
    else
    {
      this.show();
    }
  },

  getFile: function(url)
  {
    return new Request({
      url: url,
      method: 'get'
    });
  },

  show: function(cssFiles)
  {
    if(css) this.addCssFiles(cssFiles);
    ...
  }

});

With Promises

 
var MyClass = new Class({

  initialize: function(el, options)
  {
    this.setOptions(...);
    
    var p = null;
    if(this.options.cssFiles)
    {
       p = new Promise(this.options.cssFiles.map(this.getFile));
    }
    
    this.show(p);
  },
  
  getFile: function(url)
  {
    return new Request({
      url: url
      method: 'get'
    });
  }.decorate(promise),
  
  show: function(cssFiles)
  {
     if(css) this.addCSS(cssFiles);
     ...
  }.decorate(promise)
  
);

The Promises version here has several advantages.

1. We don’t need to use Group.
2. We don’t need to send the requests after the addition of the
   onComplete handler.
3. We don’t need to create a closure to map the requests to their
   responseText values.
4. We don’t need to call show in two different locations.
5. The call to show only blocks if we have files to load.

Now imagine that you decide that show needs to take another
asynchronous parameter- like an html fragment. In the original code
this means creating another closure, a separate send, and another
request → responseText operation.

if(this.options.cssFiles)
{
  var reqs = this.options.cssFiles.map(this.getFile.bind(this));
  var html = this.getFile("foo.html");                                // Change 1: need a closure
  reqs.push(html);                                                    // Change 2: add it to the requests list
  var group = new Group(reqs);
  group.addEvent('onComplete', function() {
    this.show(reqs.map(function(req) { return req.responseText; }),
    html.responseText);                                               // Change 3: need to convert to responseText
  }.bind(this);
  reqs.each(function(req) { req.send(); });
}

With Promises this will require only modifying a single line.

var p = null;
if(this.options.cssFiles)
{
  p = new Promise(this.options.cssFiles.map(this.getFile));
}

this.show(p, this.getFile("foo.html")); // the only change

Hopefully you’re beginning to see that regular requests cannot be composed
and Promises can.

Other Notes & Caveats

Debugging & Unit Tests

There are some debugging dangers to take note of when using
Promises. If you’ve done any programming in a language that supports
lazy data structures you know the kind of trouble you can get
to.

You might find that errors occur in places much further down the line
then you’d normally expect. I’ll try to improve the error reporting
capabilities of Promises, but in general I imagine Promises will be
most effective in a codebase that unit tests all promise decorated
functions.

Again, the benefit of Promises is that if your functions
pass unit tests with regular arguments, they are guaranteed to work
with values coming from asynchronous requests!

You can unit test your code as regular code not as asynchronous code.

Concurrency Issues

Be careful of modifying a promise via Promise.op after you’ve passed
it onto another function. JavaScript is mutable language- if you give
ops to a Promise after it’s crossed a function interface, you’ll find
that you get very inconsistent results. If you mostly treat Promises as
immutable values you’ll avoid monstrous bugs in your code.

Rules of thumb about modifying Promises via Promise.op:

1. At the creation site of a Promise instance you can modify the
   instance at will.
2. As soon as Promise is sent to another function, you call Promise.op
   at your own peril.

Another thing to watch out for is that code that uses Promises is not
deterministic. You have to be aware of that fact when writing functions
that call other function with Promises instead of fully realized
values. Consider the following flawed example:

var MyClass = new Class({

  loadResource: function(rsrc)
  {
    return new Request({
       url: ...,
       method: 'get'
    });
  }.decorate(promise),

  doA: funtion(rsrc)
  {
    this.x = rsrc;
  }.decorate(promise),

  doB: function(rsrc)
  {
    this.y = this.x + rsrc;              // assumption that doA completed first
  }.decorate(proimse),

  myMethod: function() 
  {
    this.doA(this.loadResource(...));    // calling this.doA with a Promise
    this.doB(this.loadResource(...));    // calling this.doB with a Promise
  }

})

doB should not depend on the behavior of doA at all. Without Promises
if you changed something in doA that change would appear in doB because
you called doA first. However, using Promises is very much like writing code
with threads. You have no such guarantees.

Lazy Values

Sometimes it’s very useful to be able to operate on values that don’t actually
exist yet. This is useful even when no remote request are involved. For example:

var add = function(a, b) { return a + b; }.decorate(promise);
var a = $P();
var b = $P();
var sum = add(a, b);

sum.value(); // -> undefined
a.setValue(2);
sum.value(); // -> undefined 
b.setValue(3);
sum.value(); // -> 5

Pretty cool.

Things get really interesting when you come to realize you can use this functionality
with the DOM :)