Skip to content
This repository

Tame #1942

Closed
wants to merge 247 commits into from
Maxwell Krohn

Here's a first stab at a tame pull request, comments welcome. See TAME.md for more details.

Rob Tsuk

I've got a nodejs script that uses child_process.exec() extensively that would be a excellent test case. I'll port it over to this approach on Monday.

Paul Miller

Curious about readability of real-world async coffeescript code.

Could you (or someone), please, show await / defer version of this function?

  compile: (files, callback) ->
    async.sortBy files, @sort, (error, sorted) =>
      return @logError error if error?
      async.map sorted, @map, (error, mapped) =>
        return @logError error if error?
        async.reduce mapped, null, @reduce, (error, reduced) =>
          return @logError error if error?
          @write reduced, (error) =>
            return @logError error if error?
            @log()
            callback @getClassName()
Maxwell Krohn

Just doing a simple translation would be the following, however, it seems as if you're leaking callback in the case of errors, no?

compile: (files, callback) ->
   await async.sortBy files, @sort, defer error, sorted
   return @logError error if error?
   await async.map sorted, @map, defer error, mapped
   return @logError error if error?
   await async.reduce mapped, null, @reduce, defer error, reduced
   return @logError error if error?
   await @write reduced, defer error
   return @logError error if error?
   @log()
   callback @getClassName()

Ryan Florence

This looks amazing.

This new (awesome) syntax clearly doesn't stick very well with CoffeeScript's general behavior of having the JavaScript really similar to the CoffeeScript. The transpiled JavaScript is wildly different looking code--nobody would ever write their JavaScript that way.

Here is the translated output (slightly hand-edited for clarity):

Are those comments like // await block f4() part of the output? I think they should be.

Would love some brainstorming on making the output a little more human readable. Similar to how things get stored in _ref in a comprehension, maybe assign all those function expressions and pass them around?

_await_f4 = function () {}
_continue_f4 = function () {}
(function() {
  (_await_f4())
  (_continue_f4())
  ... etc
});

Awesome work, can't wait to try this out.

Paul Miller

Looks nice! :+1: from me.

Added:
I would be even more happy if there was some DRY method to handle errors.

Stéphan Kochen

TAME.md should've been a link, so here you go.

@paulmillr: I've only read TAME.md, but I figure it'd look like:

compile: (files, autocb) ->
  await async.sortBy files, @sort, defer(error, sorted)
  return @logError error if error?
  await async.map sorted, @map, defer(error, mapped)
  return @logError error if error?
  await async.reduce mapped, null, @reduce, defer(error, reduced)
  return @logError error if error?
  await @write reduced, defer(error)
  return @logError error if error?
  @log()
  return @getClassName()

There's a slight difference in behavior, though. Every return invokes the callback for compile, while the original code just did it once at the end. In case of an error, the callback receives the return value of logError. But that's a side-effect of autocb and can be avoided by simply not using it.

This has left me with the question of how to use autocb and a callback with multiple arguments. It's very common in Node.js for callbacks to have (error, result) style callbacks.

Also, I figure any throw after the first await is no longer caught by a try-catch block outside of the function, much like 'regular' async code we have now? (e.g., in @paulmillr's example, a try-catch around the call to compile.)

+1

Maxwell Krohn

@stephank good point about autocb only taking zero or one parameter. The current coffee grammar isn't happy with return(a,b). Something like return [a,b] means something else. I was hoping to avoid any new keywords. Any other ideas? I'm stumped. Thanks!

Maxwell Krohn

@rpflorence thanks the for comments. At the very least regular coffee code without await will compile as before. Of course that's not a great answer. Loops are definitely going to look different. The tamejs translation uses a fair number of temporaries, sort of like your suggestion, but that makes things look even worse, in my opinion.

showell

My vote is to keep this as a fork until CS gets line number support. I understand the use case of making async code nicer to write in JS, but it seems like a lot of extra complexity for the compiler. I wish the translation continued to happen at the JS part of the stack, and that the only CS hooks were more generic, like the ability to define a custom prefix for statements, or maybe some sort of generic annotation feature.

The reason that I'd postpone this until line number support is that I think the CS compiler could benefit from a little bit of rewriting, and the tame features are just gonna make it that much more difficult to do a rewrite.

The rewrite that I'd like to see on CS is make the parsing stage a little more generic about building an AST, and then let the code generation piece do more of the reassembly type stuff, such as marrying ifs with else.

In general, regardless of line number support, this is obviously a pretty major change to CS, and it seems like we should be conservative about pulling it in.

I'm happy to elaborate on the "generic hooks/annotation" idea if that has any resonance with other folks.

Aaron Shafovaloff

+1 !!!!!

Michael Ficarra
Collaborator

I want to echo a few of @showell's comments: we must be very cautious with this very large change, and the current relatively small compiler very badly needs a cleanup or a rewrite.

I'm going to go through the diff now, paying attention only to proper style or potential bugs. Before I do that, though, @maxtaco: does the syntax used here have any advantages over the syntax I proposed in #1710? I think the syntax from #1710 is more approachable and natural.

Michael Ficarra michaelficarra commented on the diff December 17, 2011
((48 lines not shown))
  48
+    setTimeout defer(), 100
  49
+    setTimeout defer(), 10
  50
+  console.log ("hello");
  51
+```
  52
+
  53
+Now for something more useful. Here is a parallel DNS resolver that
  54
+will exit as soon as the last of your resolutions completes:
  55
+
  56
+```coffeescript
  57
+dns = require("dns");
  58
+
  59
+do_one = (cb, host) ->
  60
+  await dns.resolve host, "A", defer(err, ip)
  61
+  msg = if err then "ERROR! #{err}" else "#{host} -> #{ip}"
  62
+  console.log msg
  63
+  cb()
5
Michael Ficarra Collaborator

do cb

Trevor Burnham Collaborator

I'm pretty sure func() is more widely used than do func, including within the compiler... see e.g. http://coffeescript.org/documentation/docs/lexer.html.

Michael Ficarra Collaborator

do is still a new syntax for CS, so it's reasonable that it wouldn't be all over the compiler yet. do should be used when you're not directly using the value of a function invocation, like when using a function for its side effects. @jashkenas: do you agree? Though this one's kind of different because it's implicitly part of a return, so either way would be acceptable in my eyes.

Jeremy Ashkenas Owner

No. I still have faint hopes of being able to axe do eventually... At the moment, it exists as a necessary evil to avoid the ((arg)->)(arg) pattern, and the ability for it to invoke an arbitrary expression is sort of a consequence of consistency, but I don't think it would be considered good style. This isn't a strong proscription -- just a note for patches against the compiler.

do cb

I've been using do exactly as @michaelficarra describes ('not directly using the value of a function invocation'), especially when an invocation appears at the end of a method, and it really clears up my code in my eyes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
TAME.md
((55 lines not shown))
  55
+
  56
+```coffeescript
  57
+dns = require("dns");
  58
+
  59
+do_one = (cb, host) ->
  60
+  await dns.resolve host, "A", defer(err, ip)
  61
+  msg = if err then "ERROR! #{err}" else "#{host} -> #{ip}"
  62
+  console.log msg
  63
+  cb()
  64
+
  65
+do_all = (lst) ->
  66
+  await
  67
+    for h in lst
  68
+      do_one defer(), h
  69
+
  70
+do_all process.argv.slice(2)
1
Michael Ficarra Collaborator

process.argv[2...]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
TAME.md
((102 lines not shown))
  102
+academic paper on tame](http://pdos.csail.mit.edu/~max/docs/tame.pdf)
  103
+suggests a technique called a *rendezvous*.  A rendezvous is implemented in
  104
+CoffeeScript as a pure CS construct (no rewriting involved), which allows a
  105
+program to continue as soon as the first deferral is fulfilled (rather than
  106
+the last):
  107
+
  108
+```coffeescript
  109
+tameRequire(node) # need full library via require() for rendezvous
  110
+
  111
+do_all = (lst, windowsz) ->
  112
+  rv = new tame.Rendezvous
  113
+  nsent = 0
  114
+  nrecv = 0
  115
+
  116
+  while nrecv < lst.length
  117
+    if nsent - nrecv < windowsz && nsent < n
1
Michael Ficarra Collaborator

and

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
TAME.md
((103 lines not shown))
  103
+suggests a technique called a *rendezvous*.  A rendezvous is implemented in
  104
+CoffeeScript as a pure CS construct (no rewriting involved), which allows a
  105
+program to continue as soon as the first deferral is fulfilled (rather than
  106
+the last):
  107
+
  108
+```coffeescript
  109
+tameRequire(node) # need full library via require() for rendezvous
  110
+
  111
+do_all = (lst, windowsz) ->
  112
+  rv = new tame.Rendezvous
  113
+  nsent = 0
  114
+  nrecv = 0
  115
+
  116
+  while nrecv < lst.length
  117
+    if nsent - nrecv < windowsz && nsent < n
  118
+      do_one(rv.id(nsent).defer(), lst[nsent])
1
Michael Ficarra Collaborator

do_one rv.id(nsent).defer(), lst[nsent]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
TAME.md
((106 lines not shown))
  106
+the last):
  107
+
  108
+```coffeescript
  109
+tameRequire(node) # need full library via require() for rendezvous
  110
+
  111
+do_all = (lst, windowsz) ->
  112
+  rv = new tame.Rendezvous
  113
+  nsent = 0
  114
+  nrecv = 0
  115
+
  116
+  while nrecv < lst.length
  117
+    if nsent - nrecv < windowsz && nsent < n
  118
+      do_one(rv.id(nsent).defer(), lst[nsent])
  119
+      nsent++
  120
+    else
  121
+      await rv.wait defer(evid)
1
Michael Ficarra Collaborator

await rv.wait defer evid

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
TAME.md
((144 lines not shown))
  144
+possible with just normal functional decomposition.  Therefore, we
  145
+don't allow direct `await` nesting.  With inline anonymous CoffeeScript
  146
+functions, you can concisely achieve interesting patterns.  The code
  147
+below launches 10 parallel computations, each of which must complete
  148
+two serial actions before finishing:
  149
+
  150
+```coffeescript
  151
+f = (n,cb) ->
  152
+  await
  153
+    for i in [0..n]
  154
+      ((cb) ->
  155
+        await setTimeout defer(), 5*Math.random()
  156
+        await setTimeout defer(), 4*Math.random()
  157
+        cb()
  158
+      )(defer())
  159
+  cb()
1
Michael Ficarra Collaborator

do cb

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
TAME.md
((175 lines not shown))
  175
+        setTimeout defer(), 4*Math.random ()
  176
+      )(defer())
  177
+```
  178
+In the first example, recall, you call `cb()` explicitly.  In this
  179
+example, because the callback is named `autocb`, it's fired
  180
+automatically when the tamed function returns.
  181
+
  182
+If your callback needs to fulfill with a value, then you can pass
  183
+that value via `return`.  Consider the following function, that waits
  184
+for a random number of seconds between 0 and 4. After waiting, it
  185
+then fulfills its callback `cb` with the amount of time it waited:
  186
+
  187
+```coffeescript
  188
+rand_wait = (cb) ->
  189
+  time = Math.floor (Math.random()*5);
  190
+  if time == 0
1
Michael Ficarra Collaborator

if time is 0

edit: Again, this is all over. I won't make any further comments on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
TAME.md
((140 lines not shown))
  140
+
  141
+### Composing Serial And Parallel Patterns
  142
+
  143
+In Tame, arbitrary composition of serial and parallel control flows is
  144
+possible with just normal functional decomposition.  Therefore, we
  145
+don't allow direct `await` nesting.  With inline anonymous CoffeeScript
  146
+functions, you can concisely achieve interesting patterns.  The code
  147
+below launches 10 parallel computations, each of which must complete
  148
+two serial actions before finishing:
  149
+
  150
+```coffeescript
  151
+f = (n,cb) ->
  152
+  await
  153
+    for i in [0..n]
  154
+      ((cb) ->
  155
+        await setTimeout defer(), 5*Math.random()
1
Michael Ficarra Collaborator

Spaces around infix binary operators. I won't mention this one everywhere.

edit: specified which operators

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
TAME.md
((174 lines not shown))
  174
+        setTimeout defer(), 5*Math.random ()
  175
+        setTimeout defer(), 4*Math.random ()
  176
+      )(defer())
  177
+```
  178
+In the first example, recall, you call `cb()` explicitly.  In this
  179
+example, because the callback is named `autocb`, it's fired
  180
+automatically when the tamed function returns.
  181
+
  182
+If your callback needs to fulfill with a value, then you can pass
  183
+that value via `return`.  Consider the following function, that waits
  184
+for a random number of seconds between 0 and 4. After waiting, it
  185
+then fulfills its callback `cb` with the amount of time it waited:
  186
+
  187
+```coffeescript
  188
+rand_wait = (cb) ->
  189
+  time = Math.floor (Math.random()*5);
2
Michael Ficarra Collaborator

Try to avoid these gratuitous parentheses where precedence is pretty obvious.

Maxwell Krohn
maxtaco added a note December 17, 2011

Sorry, I copied it from the JS! Will fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
TAME.md
((179 lines not shown))
  179
+example, because the callback is named `autocb`, it's fired
  180
+automatically when the tamed function returns.
  181
+
  182
+If your callback needs to fulfill with a value, then you can pass
  183
+that value via `return`.  Consider the following function, that waits
  184
+for a random number of seconds between 0 and 4. After waiting, it
  185
+then fulfills its callback `cb` with the amount of time it waited:
  186
+
  187
+```coffeescript
  188
+rand_wait = (cb) ->
  189
+  time = Math.floor (Math.random()*5);
  190
+  if time == 0
  191
+   cb(0)
  192
+   return
  193
+  await setTimeout defer(), time
  194
+  cb(time) # return here, implicitly.....
3
Michael Ficarra Collaborator

cb time. Invocation shouldn't have parentheses unless necessary. I won't make any more comments on this.

Jeremy Ashkenas Owner

@michaelficarra This is helpful ... but it may be more useful for you to make changes in your fork of this branch. As line notes, they make it more difficult for the rest of us to read through this particular pull request.

Michael Ficarra Collaborator

Alright, will do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Michael Ficarra michaelficarra commented on the diff December 17, 2011
((222 lines not shown))
  222
+behavior of the `Code` block accordingly.
  223
+
  224
+These keywords represent the potential for these tame additions to
  225
+break existing CoffeeScript code --- any preexisting use of these
  226
+keywords as regular function, variable or class names will cause
  227
+headaches.
  228
+
  229
+### The Lowdown on defer
  230
+
  231
+The implementation of `defer` is interesting --- it's trying to
  232
+emulate ``call by reference'' in languages like C++ or Java.  Here is an 
  233
+example that shows off the four different cases required to make this
  234
+happen:
  235
+
  236
+```coffeescript
  237
+cb = defer x, obj.field, arr[i], rest..
1
Michael Ficarra Collaborator

rest...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
TAME.md
((302 lines not shown))
  302
+## Translation Technique
  303
+
  304
+The CoffeeScript tame addition uses a similar continuation-passing translation
  305
+to *tamejs*, but it's been refined to generate cleaner code, and to translate
  306
+only when necessary.  Here are the general steps involved:
  307
+
  308
+* **1** Run the standard CoffeeScript lexer, rewriter, and parser, with a 
  309
+few small additions (for `await` and `defer`), yielding
  310
+a standard CoffeeScript-style abstract syntax tree (AST).
  311
+
  312
+* **2** Apply *tame annotations*:
  313
+
  314
+   * **2.1** Find all `await` nodes in the AST.  Mark these nodes and their
  315
+   ancestors with an **A** flag.
  316
+
  317
+   * **2.2** Find all `for`, `while`, or `loop` nodes marked with **A**.
1
Michael Ficarra Collaborator

until?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
TAME.md
((312 lines not shown))
  312
+* **2** Apply *tame annotations*:
  313
+
  314
+   * **2.1** Find all `await` nodes in the AST.  Mark these nodes and their
  315
+   ancestors with an **A** flag.
  316
+
  317
+   * **2.2** Find all `for`, `while`, or `loop` nodes marked with **A**.
  318
+   Mark them and their descendants with an **L** flag.
  319
+
  320
+   * **2.3** Find all `continue` or `break` nodes marked with an **L** flag.
  321
+   Mark them and their descendants with a **P** flag.
  322
+
  323
+* **3** ``Rotate'' all those nodes marked with **A** or **P**:
  324
+
  325
+   * **3.1** For each `Block` node _b_ in the `AST` marked **A** or **P**:
  326
+
  327
+      * **3.1.1** Find _b_'s first child _c_ marked with with **A** or **P**.
1
Michael Ficarra Collaborator

"with with" -> "with"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Maxwell Krohn

@michaelficarra I like the syntax in #1710, and I'd be open to changing the syntax to make it more Coffee-flavored. But I don't see how it would work with something like a Rendezvous or a Pipeliner in the tame_advanced test file. Both cases have a defer outside an await.

Thanks for taking such a careful read through the code.

@michaelficarra or @jashkenas, should I still make these changes/fixes in my branch?

Jeremy Ashkenas
Owner

Yes, if you continue to make changes and fixes in this branch, the pull request will continue to be updated.

Michael Ficarra
Collaborator

@maxtaco:

I'd be open to changing the syntax to make it more Coffee-flavored. But I don't see how it would work with something like [...]

I'll have to look into it once I'm more familiar with the contents of the pull request.

should I still make these changes/fixes in my branch?

Go ahead and make the changes that I've mentioned so far so that I can delete all those comments and then send you a pull request for anything that's left.

I did want to note that the autocb magics repulsed me on my first read-through. I don't think magic behaviour like that is ever appropriate. We should look into possibly using some sigil instead (I know, I know, Perlisms...) to annotate the fact that an argument behaves as an autocb.

Yoz Grahame

Another +1 to the comment from @showell. I'm just a CS user, not someone who's worked with the source. While I would dearly love yield/await in CS (trust me, I'm desperate for something like this), it's more important to my day-to-day development work that CS code be easily debuggable. The harder it is to trace JS errors back to the originating CS code, the less useful CS becomes.

@showell : Would these generic hooks be anything like the long-dormant CS extension ideas? See issue #257. I'd love to hear whether CS insiders are open to waking this one up again.

Josh Perez

What I currently like most about CoffeeScript is the comprehensible JavaScript it outputs. I realize using await and defer is optional, but I'd still like to see JavaScript more consistent with what I'm used to seeing from CoffeeScript.

Overall I believe the complexity this introduces outweighs what it's trying to fix which isn't necessarily broken. Is there a cheaper price we can pay for this sugar?

Maxwell Krohn

@michaelficarra autocb was tricky to get right and probably still has bugs in corner cases. We've found though that it's quite useful in practice.

Devon Govett

Firstly, @maxtaco, this is awesome work, however I don't see it as something that should be part of CoffeeScript, and here's why. Firstly, it changes the semantics of the language significantly, and diverges from its JS roots too much for my taste. Secondly, it introduces new keywords that may collide with variable names in existing CoffeeScript codebases. Thirdly, it is too complex a solution to the problem in my opinion. When reading the examples, it wasn't immediately clear to me what the code did. And finally, the generated code is too far from what I would have written myself, which has been the goal of CoffeeScript from the beginning.

Don't get me wrong, this is awesome work, and I can see a need sometimes, but I don't think this is the solution, if any, that should be adopted by CoffeeScript.

-1

Michael Ficarra
Collaborator

@maxtaco: It wasn't the behaviour magic that I disliked, but the specification magic. Magic identifiers are bad. Bad bad bad bad bad. Sigils that mark an identifier as having a magic behaviour are much more acceptable.

Devon Govett

Also, correct me if I'm wrong, but it doesn't look like you handle lots of common cases, making this solution quite limited.

Do these work?

# 1. Arrays and objects
arr = [
    await loadImage("test.png", defer())
    "another value"
]

obj =
    img: await loadImage("test.png", defer())
    foo: "bar"

# 2. Chaining
await loadImage("test.png", defer()).render()

# 3. Function nesting
render(await loadImage("test.png", defer()))

# 4. Expressions
lunch = (await eat(food, defer()) for food in ['toast', 'cheese', 'wine'])

# etc.

If not, then it looks like this proposal breaks how functions can be used in current CoffeeScript. I realize that you cannot do this with existing asynchronous CoffeeScript with callbacks, but the proposal is trying to make async code look synchronous, and does not work in these cases where a normal synchronous function would. There are simply too many cases to cover if you want to make asynchronous code behave like synchronous code without actual coroutines at the VM level.

Devon Govett

Also see #241, #287, #350, #656, and #1822 for previous discussion on these issues.

Ricardo Tomasi

-∞

Is the transformation complexity worth it? I think this belongs in an external library, as it already is. Having clear javascript output is one of the most valuable things in CoffeeScript.

Is this being considered for acceptance?

Maxwell Krohn

@michaelficarra ah, I see what you're saying (I had to look up what a sigil was). Well sure, if you guys have a better idea, let me know.

As for the voting, looks like it's about +10, and negative infinity, so unless we find a higher order of positive infinity, this is all moot.

Maxwell Krohn

@devongovett I talk a little bit about this in TAME.md. It's possible to have await blocks behave like expression in simple cases like this:

x = await foo defer _

But things will get much messier in a case like this

await baz ((await foo defer _) + (await boo defer _)), defer _

I haven't implemented this, but my guess is it will complicate the internals and generated code enormously to have await blocks work like expressions everywhere. So I chose to have them never act as expressions. It's either awaits as statements, or everything you used to do before.

Edit: I was wrong. It's easy to implement and it now works.

Devon Govett

Exactly. Yeah, I think if you're going to make some async stuff seem sync, then you gotta do it all. :/

Kamil Tusznio

-1.

Great work, but not something that should be in CS core for reasons mentioned by others.

Matt Perpick

-1.

This is very cool stuff but @devongovett's counter examples are too compelling.

Paul Miller

I've rethought my opinion about defer / await, it's a complex thing and non-beautiful thing. :-1:

IMO, CoffeeScript needs something like F# async workflows. They're quite simple.

Even monads are simpler than this.

Zach Smith

-1

definitely no

Alexey Petrushin

Can't it be optional? I mean if You don't use defer&await - the generated JS code will be simple and transparent as before. But if You decide to use it - it will be generated in continuation-passing style.
Isn't it acceptable solution?

And, to not to complicate CS compiler code, maybe it may be done as pluggable pre/postprocessor for CoffeeScript AST?
Tame.js already works this way converting JS -> JS, with CoffeeScript we already have AST, so why not write await&defer as pluggable CS AST -> CS AST transformer, and make it optional CS module? It should be possible, no?

P.S.
As a positive side effect it may be a good precedent for creating CoffeeScript AST transformers API and other post/pre processing stuff :)

Alexander Trefz

i think the async workflows that @paulmillr referenced, look way way simpler than the current proposal.

-1

I agree with devongovett; while this is an interesting piece of code, that definitely would make a great plugin/library, I don't think it should be in CS, for all the reasons that devongovett mentioned.

Patryk Zawadzki

Why do we need new language keywords for that? (Of course there currently is no real way to force a "join" but that's something we should have in JS, not tack in with glue in CS)

async = require("asynchronous");
dns = require("dns");

do_all = (lst) ->
  pool = new async.Pool dns.resolve, ([host], [err, ip]) ->
    msg = if err then "ERROR! #{err}" else "#{host} -> #{ip}"
    console.log msg
  for h in lst
    pool.process h
  pool.join()

do_all process.argv[2...]
Maxwell Krohn

@devongovett and others: I woke up this morning and realized I was wrong, it's easy to make await statements work as expressions. I want to clean things up a bit, but it seems to work for now, and doesn't break anything. A few examples added to TAME.md, and many more to come. This is a bit of a rush-job but I need to leave for the day.

showell

@maxtaco How long do you think it would take to build a a prototype project on top of your patch? It would be nice to see a small project written it, maybe something on the order of 500 lines.

Devon Govett

@maxtaco interesting... do you cover all the cases I listed? I'm sure there are lots more as well. Even if you do, however, that still does not fix the other issues I and others have mentioned. However, I am still open to seeing an example as @showell suggested. :)

Jeremy Ashkenas
Owner

Just a quick note -- there's some work to be done here on getting the tame branch to compile more efficiently in the absence of tamed code. At the moment, there's about a 4x slowdown when building on the "tame" branch, versus "master".

~/Desktop/Projects/coffee-script(master) > bin/cake bench
Lex      128 ms (22956 tokens)
Rewrite  103 ms (25600 tokens)
Parse    213 ms
Compile  162 ms (137401 chars)
total    606 ms

~/Desktop/Projects/coffee-script(master) > git co tame
Switched to branch 'tame'

~/Desktop/Projects/coffee-script(tame) > bin/cake bench
Lex      179 ms (28199 tokens)
Rewrite  959 ms (31589 tokens)
Parse    817 ms
Compile  299 ms (173647 chars)
total   2254 ms
Zach Smith

From the examples posted, it seems that we're adding new keywords to save ourselves from indenting. Seems like a lot of code that will have to be maintained for a very small improvement, if it's even an improvement at all.

Maxwell Krohn
Maxwell Krohn

@jashkenas The big slowdown was due to the multiple walks of the AST in the translation passes. I put in code to short-circuit those walks if no tamed code. That's now checked in, and things are back in line, but still about 25% to 50% slower than master. More work to be done, of course.

Dave Tonge

+1 for this change. I think it would be great to have this as part of the language rather then as a separate library. Providing the code that doesn't use "tamed" code would look the same as before.

Chris Coyne

Hi all --

A few people above asked for sample code using CoffeeScript with maxtaco's await/defer additions.I've been using it for the last couple weeks, as Max was developing it. We have a team of 4 at OkCupid building an entire site on it.

We're building a large recommendation engine based on a few million users' interests on OkCupid. I won't get too far into what we're making, but a core component is a service (recommendd) that can take a collection of anything people might use to describe themselves or their interests (e.g., "vegetarian," "JavaScript", "polyamorous", and "arrested development") and predict other interests/tags that might apply to them.

Because we're building a heavy load service, the webservers are effectively coordination points. A request comes in, and they fire off all kinds of rpc's to a middle layer that sits in front of the databases. We're writing it all in CoffeeScript - the webservers and the middle layers (actually, front end JS too) - and with coffee-script's new tame functionality it's been going very smoothly.

Here is something I wrote this morning, although it's part of a currently incomplete class. I just want to point out how easy await & defer make it to change things. I originally started without logged-in user custom code, and adding some additional logic required no ripping things apart. This code would be a nightmare in a flow control library such as async.

  # --------------------------------------------------------------------------

  getRecommendations: (search_params, cb) ->

    # Do 2 things at once:
    #  - check if we have a logged in user, get their info
    #  - fire off distributed requests for search queries
    # ------------------------------------------------------------------------

    await
      interest_meta = {}
      @isLoggedIn defer logged_in, user_info
      for interest in search_params.interests
        @getMetaInfo interest.text, defer interest_meta[interest.text]

    #
    # Do more things at once:
    #   - get a taste profile for the user (only if logged in)
    #   - get taste profiles for each legit search interest
    # ------------------------------------------------------------------------

    await
      taste_profiles = {}
      @getUserTasteProfile user_info.id, defer user_taste_profile if logged_in
      for interest, meta of interest_meta
        @getTasteProfile meta.id, defer taste_profiles[meta.id] if meta

    #
    # Get the recommendations, combining all the taste profiles
    # ------------------------------------------------------------------------

    await @getRecsFromTasteProfiles taste_profiles, user_taste_profile, defer recommendations

    #
    # We have recs, but just [id, score] pairs. Let's:
    #   - look up info on each interest
    #   - save that this user got these recs (if logged in)
    # ------------------------------------------------------------------------

    full_recommendations = []

    await
      for v,i in recommendations
        @getInfo v[0], defer full_recommendations[i]
      @rememberRecommendations user_info.id, recommendations, defer() if logged_in

    cb full_recommendations

  # --------------------------------------------------------------------------

A few quick things:

  • I'm fairly new to Coffee, so you guys will have to forgive my violations of preferred style when analyzing await/defer.
  • I tried to add some helpful comments inline
  • Error handling is very easy with await/defer, but none of my tamed functions here pass back errors because they're member functions that collect them in the class, and this function will fall through gracefully with an empty array, and the error will be used elsewhere

In case anyone's wondering, if our recommendation engine takes "JavaScript" and "complaining" as interests, here's the output:

  • Ruby
  • PHP
  • open source
  • HTML
  • CSS

(Ruby gets thrown to the top once I add "complaining.")

One last thing: this addition to CoffeeScript will rally a lot of people I know to use it. People that are sold on async programming but hate the callback soup that JS forces us to write. This might sound dramatic, but with await and defer, I think CoffeeScript could become the best and most popular option for web programming.

Matt Hickford

Here's my two cents. I've coded a few toys in CoffeeScript. I'd like to write asynchronous code I can maintain. I'm jealous of the await feature in C# 5. I was excited to read about TameJS but disappointed to learn I can't use it in my CoffeeScript.

I don't know Javascript. Personally I'm as concerned about how the Javascript looks as I am how machine code looks. I'm happy for a machine to do the work of a man. My programming language does memory management, can it do callback management too?

+1 . If you don't like it, don't use it!

ps. The exposition at http://tamejs.org/ explains how parallel code written with await and defer can be easier to tinker with than code in callbacks.

Chris Coyne

@xcoderzach - Tame does not exist simply to avoid indenting. When you've introduced plenty of looping, logic and async into your code, CoffeeScript with await and defer stays very elegant. It's the prettiest event-handling code I've ever seen.

My above code getRecommendations simultaneously gets user session data and begins its lookup process. It's totally readable despite the fact that it has 7 remote procedures and 3 loops. By comparison, I tried to write this function using straight up coffee. Maybe I don't know coffee well enough.

Actually, an interesting challenge for CoffeeScript without await:

  • write my above getRecommendations function
  • make a significant change to it without ripping it all up.

I think the best thing about tame is how easy it is to make changes. Imagine I decide getRecommendations should behave differently for logged out users. For guests, I want it to retrieve some canned results and reply with that.

Recall, this is my first version, posted above.

  # --------------------------------------------------------------------------
  # BEFORE
  # --------------------------------------------------------------------------

  getRecommendations: (search_params, cb) ->

    await
      interest_meta = {}
      @isLoggedIn defer logged_in, user_info
      for interest in search_params.interests
        @getMetaInfo interest.text, defer interest_meta[interest.text]

    await
      taste_profiles = {}
      @getUserTasteProfile user_info.id, defer user_taste_profile if logged_in
      for interest, meta of interest_meta
        @getTasteProfile meta.id, defer taste_profiles[meta.id] if meta

    await @getRecsFromTasteProfiles taste_profiles, user_taste_profile, defer recommendations

    full_recommendations = []

    await
      for v,i in recommendations
        @getInfo v[0], defer full_recommendations[i]
      @rememberRecommendations user_info.id, recommendations, defer() if logged_in

    cb full_recommendations
    # --------------------------------------------------------------------------

Now, to perform the login check first, I just move it to its own await, and I can put everything else inside a conditional.

  # --------------------------------------------------------------------------
  # AFTER
  # --------------------------------------------------------------------------

  getRecommendations: (search_params, cb) ->

    await @isLoggedIn defer logged_in, user_info

    if not logged_in
      await @getCommonRecommendations defer common_recommendations
      cb common_recommendations

    else 
      await
        interest_meta = {}
        for interest in search_params.interests
          @getMetaInfo interest.text, defer interest_meta[interest.text]

      await
        taste_profiles = {}
        @getUserTasteProfile user_info.id, defer user_taste_profile
        for interest, meta of interest_meta
          @getTasteProfile meta.id, defer taste_profiles[meta.id] if meta

      await @getRecsFromTasteProfiles taste_profiles, user_taste_profile, defer recommendations

      full_recommendations = []

      await
        for v,i in recommendations
          @getInfo v[0], defer full_recommendations[i]
        @rememberRecommendations user_info.id, recommendations, defer()

      cb full_recommendations
      # --------------------------------------------------------------------------

Again, this was really easy thanks to await and defer.

Devon Govett

I agree that this is very nice in theory, but I don't like the way defer works. I think its weird to have assignments at the end of a line, e.g.

@getMetaInfo interest.text, defer interest_meta[interest.text]

It might be nicer to make it like this:

interest_meta[interest.text] = @getMetaInfo interest.text, defer

as it is clearer. Obviously this breaks down for > 1 argument to the callback, but I suppose the destructuring operator could help here. e.g. instead of this:

await @isLoggedIn defer logged_in, user_info

this:

[logged_in, user_info] = await @isLoggedIn defer

I think this is clearer.

And what about if the await block is only one line, making it optional? For example, you could simplify the previous example to:

[logged_in, user_info] = @isLoggedIn defer

Some of the other issues still stand, but those are just some thoughts on how the syntax could be made better anyway. :)

Devon Govett

Also, just for discussion purposes, I did a quick port of @malgorithms function above to plain CoffeeScript with no libraries. Obviously I couldn't test this because I don't have the referenced functions so let me know if I made a mistake but here is what I came up with:

# --------------------------------------------------------------------------

getRecommendations: (search_params, cb) ->

    # Do 2 things at once:
    #  - check if we have a logged in user, get their info
    #  - fire off distributed requests for search queries
    # ------------------------------------------------------------------------

    interest_meta = {}
    logged_in = user_info = null
    do step_one = =>
        done = 0
        @isLoggedIn ->
            [logged_in, user_info] = arguments
            step_two() if ++done is 2

        interests = search_params.interests.slice()
        do proceed = =>
            unless interest = interests.shift()
                step_two() if ++done is 2
                return

            @getMetaInfo interest.text, (info) =>
                interest_meta[interest.text] = info
                proceed()

    #
    # Do more things at once:
    #   - get a taste profile for the user (only if logged in)
    #   - get taste profiles for each legit search interest
    # ------------------------------------------------------------------------

    taste_profiles = {}
    user_taste_profile = null            
    step_two = =>
        done = 0
        if logged_in
            @getUserTasteProfile user_info.id, (p) ->
                user_taste_profile = p
                step_three() if ++done is 2

        keys = Object.keys(interest_meta)        
        do proceed = =>
            unless interest = keys.shift()
                step_three() if ++done is 2
                return

            if meta = interest_meta[interest]
                @getTasteProfile meta.id, (p) ->
                    taste_profiles[meta.id] = p
                    proceed()
            else
                proceed()

    #
    # Get the recommendations, combining all the taste profiles
    # ------------------------------------------------------------------------

    full_recommendations = []            
    step_three = =>
        @getRecsFromTasteProfiles taste_profiles, user_taste_profile, (recommendations) ->
            #
            # We have recs, but just [id, score] pairs. Let's:
            #   - look up info on each interest
            #   - save that this user got these recs (if logged in)
            # ------------------------------------------------------------------------

            i = 0
            do proceed = =>
                return next() unless v = recommendations[i]
                @getInfo v[0], (info) ->
                    full_recommendations[i++] = info
                    proceed()

            next = =>
                if logged_in
                    @rememberRecommendations user_info.id, recommendations, ->
                        cb full_recommendations
                else
                    cb full_recommendations

# --------------------------------------------------------------------------

Not sure how it would have been different had I used a library like async, so if someone is so inlined to do a port, have at it! :)

Michael Ficarra
Collaborator

@devongovett: check out #1710. Is that syntax more to your liking? It doesn't break down for >1 arguments or require us to special-case destructuring assignment.

Maxwell Krohn

@devongovett I think this code makes the calls to @getMetaInfo, @getTasteProfile and @getInfo, respectively, in serial. @malgorithms's code fires them in parallel. Your code would look something like this in tame coffee:

await
  interest_meta = {}
  @isLoggedIn defer logged_in, user_info
  ((autocb) -> for interest in search_params.interests
    await @getMetaInfo interest.text, defer interest_meta[interest.text]
  )(defer())
# .. etc ...

Interesting though, the unrolled loop structure in your code looks a lot like what the modded coffee compiler would output for my snippet just above.

Maxwell Krohn

The alternative syntaxes I've seen detract from the power of the language, preventing some things we do in practice, such as timeouts:

{timeout} = require 'tamelib'
info = []
host = "pirateWarezSite.ru"
await dns.lookup host, timeout(defer(err, ip), 100, info)
if not info[0] then console.log "#{host}: timed out; no reply in 100ms"
else if err then    console.log "#{host}: error: #{err}"
else                console.log "#{host} -> #{ip}"

and TCP-style windowing of network calls (make lst.length calls with only windowsz outstanding at any one given time):

do_all = (lst, windowsz) ->
  pipeliner = new Pipeliner windowsz
  for x in lst
    await pipeliner.waitInQueue defer()
    make_one_call pipeliner.defer(), x
  await pipeliner.flush defer()
Devon Govett

@maxtaco Hmmm, ok. I misunderstood. It looked to me like the for loop would do those calls in serial because of the mentions of defer. Something to clarify maybe?

@michaelficarra hmm, maybe, but I'd prefer to keep it as close to existing coffeescript as possible, and using the existing coffeescript syntax makes it clear as to what it is doing. = means setting a value, etc.

Zach Smith
stuff = () ->
  await
    @isLoggedIn defer logged_in, user_info
  return logged_in

A seasoned (java|coffee)scripter would understand that this, of course, wouldn't work. But wouldn't this be confusing to people new to coffeescript coming from another language?

I'm assuming this doesn't work as expected, right? Or does it do some sort of callback magic?

Zach Smith

I made something similar that doesn't change the language.

https://github.com/xcoderzach/VanillaAwaitDefer

And it could be extended to have a shorthand for short oneliners, something like

awaitOne (d) -> afterStuff(d()), (arguments, just, go, here) ->  
Simon Free

It seems to me that many of the people who object to this don't want to see these new magic keywords forced into the language for everyone to deal with, possibly breaking existing code with symbol clashes, etc.

That's the only thing I can really object to, myself, though I don't think it would really break too much. To make everyone happy you could make the Tame rejigger only apply if a magic require is given at the top of the source file, or something like that.

Other than that, I definitely think this is something that will really set CoffeeScript apart and make it a really awesome choice for building concurrent applications.

+1 for what it's worth.

Patryk Zawadzki

This class of problems would be much better solved with cooperative multitasking.

All of the above examples could be easily solved with just one new keyword: yield.

What it would do is interrupt the execution and reschedule the rest of code at the end of the event queue (similar to calling setTimeout with a very low delay). It would allow you to spin waiting loops while allowing other code to run.

The problem is multitasking should be a part of JS, not CS. Otherwise it's very easy to write code that is impossible to debug once compiled.

Hypothetical await and defer replacement could then look like that:

dns = require("dns");

class Pool
  constructor: (@deferrable, @postprocess_cb) ->
    @workers = []
    # ...
  join: ->
    while @workers.length
      yield
  process: [args...] ->
    # ...
    @workers.push # ...

pool = new Pool dns.resolve, ([host], [err, ip]) ->
  msg = if err then "ERROR! #{err}" else "#{host} -> #{ip}"
  console.log msg
for host in host_list
  pool.process host
pool.join()
console.log 'Done!'
Paul Miller

+1 for @patrys yield proposition.

ES.next have generators, so it would be great if coffee script had supported them too.

Chris Coyne

@devongovett I think the simplest way to think of the relationship between await and defer is this:

await execution does not continue outside this block until all the functions generated by defer have been executed.

defer returns a function that collects callback args.

This idea of defer, that it's returning a function, is why the placement of defer with vars makes sense to me instead of as an assignment. But that said I don't feel strongly about that issue.

To illustrate what I mean about placement: let's say you're stuck working with someone else's function that requires two callback functions as arguments. One for when it has initialized, reporting some port, and another when it's totally ready to accept connections. And you want to proceed after both have called back, because you know the service is ready to go.

Here's what calling the function would look like with named params:

startService arg1, arg2, port_binding_cb, init_cb

In practice, when you go to call this function, you're aware that arguments 3 and 4 are supposed to be functions, so you look to defer to provide those functions.

await startService arg1, arg2, defer(port), defer()

# continuing your code here, you now know the port
# and you know the service is fully listening

I also find your assignment syntax very appealing, just saying that defer is returning a function -- and that function is supposd to be passed as an argument to startService. So it's abstracting away in a slightly weird way for me to remove that.

Chris Coyne

@patrys @paulmillr I believe a requirement of CoffeeScript is that it outputs JavaScript that existing browsers/JS engines can handle.

Chris Coyne

@xcoderzach - can you write the getRecommendations function above, using your vanilla await and defer?

Maxwell Krohn

@patrys the threads vs. events war is an unwinnable stalemate, so I stay out of it. Also, you kind of just proposed:

await
  node.threadSupport defer()
  for b in browsers
    for v in b.versions
      v.threadSupport defer()

At least you can wait in parallel.

In all seriousness, I prefer events to threads because events are first class objects that you can manipulate, store in arrays, and chain together. That gives a lot expressiveness that makes programming fun to me. Others, I know, disagree.

Patryk Zawadzki

@malgorithms and I believe it's not worth it because of the complexity it introduces. I am not proposing a solution, I am telling you that (in my opinion) the proposed solution—even if the best that is currently possible—is bad.

Chris Coyne

@xcoderzach @devongovett @maxtaco @michaelficarra @jashkenas @patrys - for my own education and to further this discussion intelligently, I figured I'd provide a wrapper for writing getRecommendations. So anyone can fill in their own function and run it on the command line.

Maybe one of these is true:

Case 1. That getRecommendations can be written and edited easily without Tame
Case 2. That getRecommendations is a bad function, that it shouldn't really be possible in CoffeeScript. :-)

I have a hard time accepting case 2, because I think the beauty of async programming is that you can fire off all kinds of calls and have practically no overhead while you're waiting for callbacks.

I don't want to be presumptuous that any of you even care enough to try this, but if you want to contribute to this CoffeeScript discussion, it might be a great exercise. At the very least, maybe it'll be fun?

Just edit the file to replace the function with your own, non-tamed version. Output should look something like this.

open remote calls: logged_in (1)
open remote calls: logged_in (1) get_meta (1)
open remote calls: logged_in (1) get_meta (2)
open remote calls: logged_in (1) get_meta (1)
open remote calls: logged_in (0) get_meta (1)
open remote calls: logged_in (0) get_meta (0)
open remote calls: logged_in (0) get_meta (0) get_user_taste (1)
open remote calls: logged_in (0) get_meta (0) get_user_taste (1) get_taste (1)
open remote calls: logged_in (0) get_meta (0) get_user_taste (1) get_taste (2)
open remote calls: logged_in (0) get_meta (0) get_user_taste (1) get_taste (1)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (1)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (1)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (1)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (2)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (3)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (4)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (5)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (6)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (7)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (8)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (9)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (10)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (11)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (11) rememberRecommendations (1)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (10) rememberRecommendations (1)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (9) rememberRecommendations (1)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (9) rememberRecommendations (0)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (8) rememberRecommendations (0)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (7) rememberRecommendations (0)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (6) rememberRecommendations (0)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (5) rememberRecommendations (0)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (4) rememberRecommendations (0)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (3) rememberRecommendations (0)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (2) rememberRecommendations (0)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (1) rememberRecommendations (0)
open remote calls: logged_in (0) get_meta (0) get_user_taste (0) get_taste (0) get_recs (0) get_info (0) rememberRecommendations (0)
-------------
Done. Got 11 results in 3115ms

Every RPC takes a random amount of time from 0 to 1000ms, and a user is randomly chosen as logged in or not.

Here is the gist:

https://gist.github.com/1501840

Wael M. Nasreddine

I would really love this feature, hopefully this will either make it into CS core or as a separate library, either way is fine by me

+++1

Ricardo Tomasi

@malgorithms an onlooker has absolutely no idea what the magical awaits and defers in getRecommendations should translate to in Javascript. They "translate" to a specific library implementation.

edit: recommender using async: https://gist.github.com/1502809/e95a86fc7b2b84ddf7001012a1a920b99c214390

Chris Coyne

@ricardobeat - my proposed exercise wasn't to translate await and defer in JavaScript or CoffeeScript. My exercise was simply to write getRecommendations however you want. You don't have to, of course, but I'm curious what it would look like if you did. (In other words, forget await and defer for now.)

Zach Smith

Ok, here is my rewrite of getRecomendations

{serialAwait} = require("./await_defer")
class Recommender

  getRecommendations: (search_params, cb) ->

    # Do 2 things at once:
    # - check if we have a logged in user, get their info
    # - fire off distributed requests for search queries
    # ------------------------------------------------------------------------

    await = serialAwait()

    await (defer) =>
      @isLoggedIn defer "logged_in", "user_info"
      for interest in search_params.interests
        @getMetaInfo interest.text, defer "interest_meta[" + interest.text + "]"

    #
    # Do more things at once:
    # - get a taste profile for the user (only if logged in)
    # - get taste profiles for each legit search interest
    # ------------------------------------------------------------------------

    await ({interest_meta, logged_in, user_info}, defer) =>
      @getUserTasteProfile user_info.id, defer "user_taste_profile" if logged_in
      for interest, meta of interest_meta
        @getTasteProfile meta.id, defer "taste_profiles[" + meta.id + "]" if meta


    #
    # Get the recommendations, combining all the taste profiles
    # ------------------------------------------------------------------------

    await ({user_taste_profile, taste_profiles}, defer) =>
      @getRecsFromTasteProfiles taste_profiles, user_taste_profile, defer "recommendations"

    #
    # We have recs, but just [id, score] pairs. Let's:
    # - look up info on each interest
    # - save that this user got these recs (if logged in)
    # ------------------------------------------------------------------------

    await ({logged_in, user_info, recommendations}, defer) =>
      for v,i in recommendations
        @getInfo v[0], defer "full_recommendations[" + i + "]"
      @rememberRecommendations user_info.id, recommendations, defer() if logged_in
    , ({full_recommendations}) ->
      cb full_recommendations

  # --------------------------------------------------------------------------


If you want to run it, clone this repo and run coffee recommend.coffee
https://github.com/xcoderzach/VanillaAwaitDefer

The main difference is that it's functional, and that you have to explicitly ask for results in the next function,
which I actually prefer, since the variables aren't coming out of nowhere.

Also, it gets the same output that you pasted, as long as the "random user" is actually logged_in

Edit: To be clear, I would probably use something similar to this, maybe with different names for the functions

Although, my main reason for writing it in the await/defer style, is to show that this exact functionality can be achieved in plain coffeescript with a small amount of functional programming.

I will agree it is a very nice syntax, trying to rewrite this problem "in my normal style" results in a lot of nested callbacks

Ricardo Tomasi

@malgorithms not sure I understand your comment. If await and defer are to get into coffeescript, they would have to be translated to something - if we disconsider implementation we're just back to talking imaginary language features. I rewrote it using the async lib as per the link in my previous comment.

@xcoderzach that's awesome.

Chris Coyne

Hey @xcoderzach - that's really interesting! I'm excited to see this example. A couple questions. (1) I'm a little confused about the string additions, is there some kind of eval going on? (2) how do you handle while loops?

I'm going to play with your code a little bit. Re: while loops, what I mean is what's the VanillaAwaitDefer equivalent of this?

apples = []
while numThatAreTasty(apples) < 10
   await getAnApple defer apples[apples.length]

And this?

# same thing, but checking apples for quality is an async call
num_tasty = 0
apples = []
while num_tasty < 10
   await getAnApple defer apples[apples.length]
   await countTasty apples, defer num_tasty

I'm very pleased with where this conversation is going...thanks!

ah hell, here are the apple examples as an entire program to mess with:

# -------------------------------------------------------------
# while example 1:
#  we have an rpc that provides us apples. get them
#  until we have at least 10 good ones and print them all out
# -------------------------------------------------------------

getAnApple = (cb) -> cb Math.random()

numThatAreTasty = (some_apples) ->
  c = 0
  for apple in some_apples 
    c++ if apple > 0.5
  return c

apples = []
while numThatAreTasty(apples) < 10
   await getAnApple defer apples[apples.length]

console.log apples


# -------------------------------------------------------------
#
# Now that all that is done... :-)
#
# while example 2:
#  we have an rpc that provides us apples, and a second one
#  that analyzes the apples for quality; same idea
# -------------------------------------------------------------

getAnApple = (cb) -> cb Math.random()
countTasty = (some_apples, cb) ->
  c = 0
  for apple in some_apples 
    c++ if apple > 0.5
  cb c

apples = []
num_tasty = 0

while num_tasty < 10
   await getAnApple defer apples[apples.length]
   await countTasty apples, defer num_tasty

console.log apples

@ricardobeat ah thanks for the async example! I didn't see the link. It doesn't work right, though, in the logged in user case, at the end.

Koushik Dutta

Nice! I actually did the same thing, except in JavaScript itself (for node.js/v8):
koush/node@b04dc42

The response I got was that it should not extend JavaScript, but CoffeeScript instead. And I am reading some comments here saying it should not be in CoffeeScript, because it deviates from JavaScript. :) sigh

Maxwell Krohn

@devongovett all of those test cases now work. Awaits as expressions are tricky, both for the implementer and the user, but they're working in the cases I've tried.

Maxwell Krohn

Rebased to 642fcbb. Sorry for any confusions in the pull request log.

Matt Baker

I'm biased, because I wrote a small flow control library based on TameJS that I like a lot and use frequently. That said, I think this is similar to the new keyword, in a negative way: keywords are less flexible than existing first-class language features. You can already do this kind of stuff with libraries, and it's relatively clean; here's how @devongovett's examples could look in Heavy Flow, for example:

# 1. Arrays and objects
arr = [
    flow.wait (finish) -> loadImage("test.png", finish)
    "another value"
]

obj =
    img: flow.wait (finish) -> loadImage("test.png", finish)
    foo: "bar"

# 2. Chaining
imageLoaded = flow.wait (finish) -> loadImage("test.png", finish)
imageLoaded.finished -> imageLoaded.data().render()

# 3. Function nesting
imageLoaded = flow.wait (finish) -> loadImage("test.png", finish)
imageLoaded.finished -> render(imageLoaded.data())

# 4. Expressions
lunch = ((flow.wait (finish) -> eat(food, finish)) for food in ['toast', 'cheese', 'wine'])

There are a couple places the syntax could be improved (although writing library-aware functions would make it considerably cleaner), but it's overall pretty close to the await/defer syntax, without introducing new keywords or contorted compilations. Again, I'm biased -- but using a library feels much more flexible than introducing new keywords.

weepy

@maxtaco, @michaelficarra, @patrys

would it be possible to emulate ES5 yield/generators using a similar approach to TameJS ?

Michael Ficarra
Collaborator

@weepy: ES5 doesn't have generators. Maybe you mean generators in Mozilla's JavaScript 1.7+?

weepy

Sorry I meant ES.next or harmony

and others added some commits August 14, 2011
Jeremy Ashkenas clarifying Cake on the homepage. 4711220
Rod Knowlton Clarify `--watch` option on homepage. fa18f72
Michael Ficarra removed a gratuitous comma f325b0c
Jeremy Ashkenas adding a CNAME file for github pages. 7d3a5ff
Jeremy Ashkenas trying with just the .org a56cb02
Maxwell Krohn some starting work on plugging into the coffee script compiler and ru…
…ntime
dabc757
Maxwell Krohn Disable the debug warning for now, which definitely seems to indicate…
… that our transform is being picked up. Baby steps
5c93811
Maxwell Krohn a first crack at adding await into the grammar 57c164c
Maxwell Krohn some progress d3112d0
Maxwell Krohn progress in fixing a few bugs, and better understanding the parser. t…
…he bug was when turning a statement into a return statement
92c7b25
Maxwell Krohn some more progress with the pivot and translation stuff, still need t…
…o make a few mental leaps
835ffb5
Maxwell Krohn Fix some bugs, work out the kink in the debugging 3fc3ff4
Maxwell Krohn work out more of the CPS mechanics ec91963
Maxwell Krohn src/nodes.coffee 5a35a40
Maxwell Krohn fix my mistakes 2c2864d
Maxwell Krohn need this for bootstrapping 09d0b9e
Maxwell Krohn fix problems from yesterday 48df9e9
Maxwell Krohn factor out constants into a nice file 7d2434a
Maxwell Krohn a failed attempt to wrestle with spacing e1e2137
Maxwell Krohn got indentation for if else's working pretty well f3c82b3
Maxwell Krohn fixed a deep bug, in which we weren't recursing properly on the the r…
…otations. added a little optimization. started on While
9ba1fe0
Maxwell Krohn try different types of tame flags, i think this does it. i think. 6feb542
Maxwell Krohn i might back this out. there's probably a better way to do this. b4b6cba
Maxwell Krohn This seems like a cleaner and simpler way to tame the insides of a wh…
…ile loop.
590e72f
Maxwell Krohn get it to a working state b12f2f6
Maxwell Krohn fixed a bug in which we were overeager in the CPS rotations. 07e0a35
Maxwell Krohn Making a bit of progress on continue and intelligent tame or no tame …
…decisions
5c71f61
Maxwell Krohn gaoiefjasoidfjasodfi jasodif jasdof joi d80638c
Maxwell Krohn lots of progress on the jump situation, i think this is getting close…
… to solving it
628bebd
Maxwell Krohn Inline awaits now work 6f69b86
Maxwell Krohn Added some preliminary support for defer epxressions. Small modificat…
…ions to the lexer, parser and rewriter to get this working. Need to finish up the output path here, of course.
81c9d57
Maxwell Krohn One more tweak to get defer's working in callchains f8533fb
Maxwell Krohn dummy continuations in more places d625276
Maxwell Krohn little ditty df05680
Maxwell Krohn express args as slots. we're going to need to do the Defer conversion…
… still
6f0b583
Maxwell Krohn this won't work, but a bit of progress on slots b3e2297
Maxwell Krohn got a lot of the Deferral logic working as it does in regular tame 9b0965c
Maxwell Krohn start work on the dreaded For loop 6e7bedb
Maxwell Krohn more progress on For loops 1ca54c3
Maxwell Krohn First successful test of a little test script. Need to massage the lo…
…cal runtime a little bit better.
cd491d0
Maxwell Krohn better idea for dealing with the tame runtime 668db1e
Maxwell Krohn try a different place for these ab0b591
Maxwell Krohn try this, it might be a fail. e767f61
Maxwell Krohn yes yes yes! all embedded! c5d712e
Maxwell Krohn simplify, getting rid of _kw 9a093e2
Maxwell Krohn todos 992121c
Maxwell Krohn blah fb0b364
Maxwell Krohn Update TODO-tame.md a742dd3
Maxwell Krohn Update TODO-tame.md f634601
Maxwell Krohn no need for a temporary, finally figured it out, i was being stoopid 3856f07
Maxwell Krohn tweak and tune the header code 332fff2
Maxwell Krohn more work, got cakefiles runnins, can now think about how to test! af36682
Maxwell Krohn bleh 2e9df06
playing around a little bit ba63b4e
Maxwell Krohn take regtests in the hizzie! 42690b1
Maxwell Krohn and another regtest to test the finer points of continue / break 3fb2f51
Maxwell Krohn cleanup 242fad0
Maxwell Krohn support for switch statements 5147501
got for k,v in obj working! 7c5b57b
add regression tests for the new awesomeness. 6a21aaa
more testing goodness d2bc75b
Allow rv.id(4).defer(x) to work as in tamejs 71f34d7
more code from tamejs, this one from rendezvous. also, fix a broken r…
…egtest due to defer being a reserved coffee word.
d3f5a7c
Maxwell Krohn don't forget this stuff 5c92254
Maxwell Krohn some work in progress from san francisco. i think i've broken things …
…temporarily but i'm working on it...
9a0ee17
Maxwell Krohn now at least it can build itself. 4af8a4e
Maxwell Krohn now it's working when it can't find tameRequire 12def0e
Maxwell Krohn now we're making progress with tameRequire, which puts us in decent s…
…hape
6ba31cb
Maxwell Krohn another test, whoops, a failure! edd551c
Maxwell Krohn fixed it! got class functions to bind properly 33d5b70
Maxwell Krohn decruft some of my temporary workarounds. 0e8a9df
Maxwell Krohn comments and clean up d5a9e84
Maxwell Krohn comments. less code in the case of no assignemnt f048c42
Maxwell Krohn Don't indent here, was not needed. e286fdd
Maxwell Krohn clean up, don't comment the code dcd4849
Maxwell Krohn tame comments and stuff 9b43267
Maxwell Krohn Update TAME.md fa06523
Maxwell Krohn Update TAME.md 2e6abf7
Maxwell Krohn Update TAME.md 5a1abab
Maxwell Krohn some documenation 8f68655
Maxwell Krohn more docs 7a12162
Maxwell Krohn more docs 731c477
Maxwell Krohn docs 7a22a2f
Maxwell Krohn docs c22623f
Maxwell Krohn docs 2363ee1
Maxwell Krohn docs 481983a
Maxwell Krohn docs ee3e3cc
Maxwell Krohn docs f18a88f
Maxwell Krohn docs 699ef5a
Maxwell Krohn docs a9fdaf2
Maxwell Krohn docs ac31323
Maxwell Krohn docs af40eac
Maxwell Krohn docs be53840
Maxwell Krohn a sketch ea36c48
Maxwell Krohn rotation sketches d9ad4e7
Maxwell Krohn this was left out 49c74b3
Maxwell Krohn try it 772a841
Maxwell Krohn Update TAME.md 0a5ff10
Maxwell Krohn Update TAME.md 79debc6
Maxwell Krohn Update TAME.md 60a9ff2
Maxwell Krohn Update TAME.md 1fe9019
Maxwell Krohn Update TAME.md fc91bb7
Maxwell Krohn Update TAME.md da3305b
Maxwell Krohn Update TAME.md dcf38bb
Maxwell Krohn try this 38b314a
Maxwell Krohn try this 060e053
Maxwell Krohn x 0ab2438
Maxwell Krohn what the! rotations done e05497e
Maxwell Krohn last diagram b1f2186
Maxwell Krohn check in all doc work 0104d88
Maxwell Krohn some docs 06e1d1c
Maxwell Krohn fixed up the post-rotation graphic dbc6d4f
Maxwell Krohn fixes here too 9fe005c
Maxwell Krohn bug 01a488a
Maxwell Krohn Debugged tame rendezvous. added a bunch of test cases to make sure it…
…'s working properly, see tame_advanced.coffee
b8acda8
Maxwell Krohn more test cases e2cd933
Maxwell Krohn more stuff here 37269d5
Maxwell Krohn debug f2c561e
Maxwell Krohn autocb first try 187abb1
Maxwell Krohn more testing of autocb db680c6
Maxwell Krohn more sweet asss tests f0809af
Maxwell Krohn more docs dcad805
Maxwell Krohn fixes bb7a588
Maxwell Krohn fixup tameRequire docs and usage 2e23266
Maxwell Krohn more cleanups c6b1519
Maxwell Krohn images a912b74
Maxwell Krohn images 98aa88d
Maxwell Krohn images 6964d76
Maxwell Krohn images fc30b97
Maxwell Krohn more details 87e0b73
Maxwell Krohn docs c292afc
Maxwell Krohn docs b0833e7
Maxwell Krohn docs dc310f8
Maxwell Krohn docs 3e577d2
Maxwell Krohn clean this up a bit
Conflicts:

	lib/coffee-script/coffee-script.js
	lib/coffee-script/tame.js
0c2b467
Maxwell Krohn defer(splat...) now works. knock off some todo items 0e87d4c
Maxwell Krohn some cleanups 3a3dbe5
Maxwell Krohn some cleanups 21458a4
Maxwell Krohn docs 6e7b0b8
Maxwell Krohn docs d9aa496
Maxwell Krohn docs bcca004
Maxwell Krohn Update TAME.md 050c1f6
Maxwell Krohn Not italics, bold. d437966
Maxwell Krohn defend our position! 5d5e5d1
Maxwell Krohn nothing left todo here 3d16fb1
Maxwell Krohn no longer needed 05f589f
Maxwell Krohn everything parses again, just to check 8f5d360
Maxwell Krohn Finish up the rebase process e095831
Maxwell Krohn Keep track of if we lost any guys. Turns out we were! 394aea6
Maxwell Krohn A trip down the rabbit hole, and a journey back up. So there were thr…
…ee separate bugs, which cascaded:

1. The @ operator didn't work properly within await blocks.  This was an issue with the CpsCascade function, which didn't properly create a `Code` object with the `"boundfunc"` parameter.
2. `autocb` didn't work properly in functions that only had one line --- `await`. The problem is that no `Return` node was being grafted on.  We made an exception for that in the `Block::makeReturn` method.
3. Scoping wasn't right with `__tame_deferrals` -- that needs to be locally scoped, so make that change.
1adf300
Maxwell Krohn Various renames, in preparation for pull request 849cd98
Maxwell Krohn a lot more renaming, to make it clear what's tame-related 0a43c2a
Maxwell Krohn more cleanups, commenting and renames. bf3ec76
Maxwell Krohn this test broke my world fd041e1
Maxwell Krohn betteR! gotta run, but close! 0795b7d
Maxwell Krohn Fix a bug in which autocb's were being dropped in loops that return d4fe65b
Maxwell Krohn add more test cases. more fixups to the scoping system. e90ffb3
Maxwell Krohn add more tests to make sure that scoping is working. f4e14c0
Maxwell Krohn slightly improved test case bb30596
Maxwell Krohn Added the Pipeliner library, which exposed yet another problem with a…
…utocbs! This commit is a workaround, but i need to study it in more depth.
380f4a0
Maxwell Krohn clean up the autocb mess i think with a pretty elegant solution. Stil…
…l some more tweaking left
8f817ff
Maxwell Krohn also want this in the repor eb2636f
Maxwell Krohn more cleanup and tests. b6d5285
Maxwell Krohn yet more tests. this is the one that chris pointed out that had me re…
…evaluate scoping.
e60d3dc
Maxwell Krohn chris pointed out a bug with nested loops, which is now fixed. also, …
…i want to work on the empty body autocb problem
bb113f9
Maxwell Krohn fixed the naked autocb problem 1520ef1
Maxwell Krohn yet another autocb corner case! da98af5
Maxwell Krohn recompile all .js files 9f51683
Maxwell Krohn update links a527cb2
Maxwell Krohn whoops, errant comment c15f18c
Maxwell Krohn @michaelficarra suggested all of these bug and style fixes 58ea9e7
Maxwell Krohn more documenation d34f0a4
Maxwell Krohn fix efde659
Maxwell Krohn First stab at getting await blocks to act as expressions, almost work…
…ing, except for nested guys.
9b3d7a8
Maxwell Krohn It's working now with recursion. Add a test.
*NOTE* This feature is a bit of a rush job, but I need to run out for the day.  The point is, that it is pretty easy to make tame statements work as expressions. Things work as far as I can tell.
b857c00
Maxwell Krohn Slight updates to docs, but I need to run out for the day. 6f2d96e
Maxwell Krohn add a few more test cases, and fix a few bugs in expressions ce2bdca
Maxwell Krohn I think I see what's slow about parsing -- the tame AST passes. I thi…
…nk this fixes it, or at least works around it. Remove all tame from the core libraries
618305c
Maxwell Krohn back out these changes that i abandoned 47e6f8b
Maxwell Krohn A little bit more conservative. 04658a9
Maxwell Krohn order matters for getting the browser embedded script to work. fix it. 3e1cca8
Maxwell Krohn add a TameTailCall class so that tail calls aren't ad-hoc. Eventually…
… they'll return a value.
8247386
Maxwell Krohn working on a new implemention of tamed expressions. have the basics up
and running but it will probably get weird in certain ways.
d7d3441
Maxwell Krohn more fiddling 09c286f
Maxwell Krohn more progress on getting rotations to work with expressions. f73eb0e
Maxwell Krohn progress on the new test cases I'm just checking in. 3e4c4e4
Maxwell Krohn cleanups aeeb2f7
Maxwell Krohn this is cool, got some more test cases working. 9463080
Maxwell Krohn totally busted, can't figure out this bug. 88f9ded
Maxwell Krohn fixed the bugs, got tamed addition expressions working again. we were
mangling the prototype, which we fixed with a constructor in the Base
class. add yet more complication having to do with threading values
through tamed continuations.
7dad16e
Maxwell Krohn More work on expressions, got more things working like the tests incl…
…uded.
cc3c09f
Maxwell Krohn more things working, now nested awaits are properly serialized. 50ce7b6
Maxwell Krohn another test working! 999eeda
Maxwell Krohn more incredible tests 3a8fbfe
Maxwell Krohn more comments and commentary b8d3e72
Maxwell Krohn some progress --- got some loops working as we would expect, but stil…
…l can't handle paren'ed awaits, something's up with that.
a588590
Maxwell Krohn uggggggggggggggggggggggggggggggggggg b8f74f8
Maxwell Krohn holy shit, epic battle with the unwrap bugs has been won, for now at …
…least.
dc81455
Maxwell Krohn Get everything working after the rebase. I apologize, but it seems re…
…basing doesn't play nice with maintaining the commentary interleaving in the pull request.
2233f07
Maxwell Krohn fix a bug that cropped up as part of the rebase 3ca31ba
Maxwell Krohn built the browser, can't run the advanced tests though due to lack of
require.  maybe down the road we'll fix this.
12c1ae1
Maxwell Krohn no more! 9e80dbe
Maxwell Krohn start to hack up the gh-pages 35e32d5
Maxwell Krohn give it a go 32b2ced
Maxwell Krohn A few more test cases. fix a bug with TameTailCalls. Clean up that
code a bit.
82b79cf
Maxwell Krohn modified for bugfixies e2c6c58
Maxwell Krohn Another bug found in if's + expressions + Ops. Fixed, with a new test
case.
47db980
Maxwell Krohn Clean up tame short-circuiting 72faf5d
Maxwell Krohn More cleanups w/r/t fewer AST traversals db03e53
Maxwell Krohn some progress here, finally groked how it's working. need to figure o…
…ut something fancy to do with the boilerplate.
43e8478
Maxwell Krohn -b command line flag corresponds to --bare tame output 6c91d9e
Maxwell Krohn save a few line breaks to make the code pretty 180530e
Maxwell Krohn window.tame = .... 4e31c29
Maxwell Krohn beh, lost changes from the office. ea9b905
Maxwell Krohn need to do the same to __tame_k too 5523e40
Maxwell Krohn redo the progress i made last night 3932e71
Maxwell Krohn progress being made e21001b
Maxwell Krohn bugfix b5c2632
Maxwell Krohn pass tests with bugfix fixed e8ea119
Maxwell Krohn fix off-by-one eb259f1
Maxwell Krohn push this live to github, some progress.... b6086b1
Maxwell Krohn handle an unhandled case, with super() and no args. 6f4c885
Maxwell Krohn add a test case too to handle this bug ff457e3
Maxwell Krohn rebuild autogenerated files; fix merge errors 346d2a9
Maxwell Krohn Get the browser stuff working again c5df56d
Maxwell Krohn more tweaks, ready to push 1c84d03
Maxwell Krohn fix a bug that chris found, in nested for .. of loops 47d6dc2
Maxwell Krohn also have a way to avoid stack overflows if possible. 6e38a46
Maxwell Krohn Get ready to launch this so that it can coexist with legacy
coffee-script
0efbce3
Maxwell Krohn fix typo c95c24d
Maxwell Krohn whoops, make it correct 7286b53
Maxwell Krohn names, WTH do we call this thing? 45e3677
Maxwell Krohn more thought on names. d558331
Maxwell Krohn change it back until we figure out what to name it, still up in the air 0d2535b
Maxwell Krohn fix problems with the new extensions system 173a069
Maxwell Krohn Fix a bug in returning a forloop as a list comprehension from autocb.…
… BUT, it's still kinda broken
186c64e
Maxwell Krohn the rest of the fix from last night. 3941b3e
Maxwell Krohn version bump 1b929ab
Maxwell Krohn i think we're going to go with iced coffee script as the name 0f9756c
Maxwell Krohn start to support the .iced extension bdd81dd
Maxwell Krohn start to comment on the renaming 7484ac9
Maxwell Krohn Starting to rename everything, tame -> iced. 42b7013
Maxwell Krohn more conversion of tame -> iced 02cecb9
Maxwell Krohn version bump ba5b6db
Maxwell Krohn don't abuse parameters to functions. pass Parameters, not Values 8323d3c
Maxwell Krohn recompile after big rebase, all tests pass. a80740f
Maxwell Krohn Version bump 9eaaf88
Maxwell Krohn

I wanted to file an update on this PR, which appears moribund. First of all, I'd like to thank everyone on the list for their comments, scrutiny, and contributions. I think they've helped to make these proposed additions to Coffee better, and more in line with the philosophy of the CoffeeScript project. So thanks!

Our next move is to start promoting our fork of CoffeeScript with tamed features. We've changed names to IcedCoffeeScript. So basically s/tame/iced/g everywhere. Here's where you can find out more about it:

New features are:

  • "Iced"-aware stack traces, for really figuring out what went wrong
  • Full expression support
  • Nuked tameRequire, in favor of reasonable default behavior and compile-time flags
  • Works and tested in the browser.
  • Interesting glossy page example

Let me make a quick appeal in favor of this software. The most damning complaint I've heard about this change to CoffeeScript is that is "too complicated." But I think that attitude is counter-productive. Compilers are supposed to do complicated things to save humans from having to do them (e.g. register allocation, type inference, etc). And anyone who's written server-side Node.js code knows that hand-rolled event-based code is complicated. Whenever you make a slight change to your logic (e.g. add an if clause, change from parallel to serial, etc), you need to rip everything up, likely introducing subtle bugs. The compiler should save the human this effort, and that's exactly what IcedCoffeeScript is doing. It's about much more than preventing "callback pyramids." It is that in the toy examples, but for real cases, it's much more. It's about keeping code readable, clean, correct, parallel (when required) and maintainable.

Comments welcome, and thanks!

Paul Miller

@maxtaco this looks very nice! I'll try the thing soon.

Chris Coyne

Hooray!

At OkCupid we have an upcoming project that's already 113 IcedCoffeeScript files, including 13 different rpc servers which were all cleanly written with lots of awaits and defers. Everything from our recommendation engine to a pubsub system are pretty IcedCoffee.

The entire site we're building will be open source in a month or two. But I can attest to the fact that Iced is working great.

cc

Michael Ficarra
Collaborator

@maxtaco: I would say this pull request is in no way moribund. Though, like many other issues that people assume are not getting fixed, it just isn't a top priority. There's a lot of open issues, many of them more important than this. Don't you agree that we should get the issues with our current compilations fixed before adding new features?

Paul Miller

@maxtaco just tried it, it's still painful to write a handler of error in callback every time

  _handle: (file) ->
    return if @invalid.test sysPath.basename file
    await fs.realpath file, defer error, path
    return helpers.logError error if error?
    await fs.stat file, defer error, stats
    return helpers.logError error if error?
    @_handleFile file if stats.isFile()
    @_handleDir file if stats.isDirectory()
Maxwell Krohn

@paulmillr I agree some sort of improvement would be good here, but I'm not sure what it should be. Seems like most languages would use exceptions to handle this cleanly. I've thought a bit about moving try..catch through the CPS transform but so far haven't. And either way, fs.realpath would need a wrapper to throw an error rather than cb'ing with an error. If I had to write the above, I would probably do something like this:

_handle: (file) ->
  unless @invalid.test sysPath.basename file
    await fs.realpath file, defer error, path
    await fs.stat file, defer error, stats unless error
    if      error               then helpers.logError error
    else if stats.isFile()      then @_handleFile file
    else if stats.isDirectory() then @_handleDir file

This way, you can handle all errors in one fell swoop, and also, if you changed _handle to take a callback, it could be fired just once at the end of the function. Rewriting the equivalent with vanilla CS would get ugly,

Maxwell Krohn

@michaelficarra Sorry, maybe my wording was too extreme. First of all, glad to know people are still enthused about the idea; and second, thanks for all of the awesome (ongoing) work on CS. To better rephrase what I was trying to say, it doesn't seem to me as if there's enough agreement to motivate such a big change to CoffeeScript. But we're going to try to drum up some grassroots support over in our branch. In the mean time, I'll keep rebasing to capture the changes in the trunk.

@jashkenas at some point mentioned a CS2 might happen in the future. One suggestion I had for the internals is to change the AST layout in the code so that it's possible to traverse the AST and modify it as you go. For instance, to be able to change an If into a Block during a traversal. The way it's set up now, that's pretty cumbersome, since the pointers from the parent node are flattened into an temporary array that cannot be meaningfully manipulated from the children. Being able to mutate AST nodes on traversal would make some of the IcedCoffeeScript patches simpler.

Jeremy Ashkenas
Owner

One suggestion I had for the internals is to change the AST layout in the code
so that it's possible to traverse the AST and modify it as you go. For instance,
to be able to change an If into a Block during a traversal. The way it's set up
now, that's pretty cumbersome...

@maxtaco: Patches that clean up the AST, but have no external effects ... are always welcomed warmly. I'd be glad to merge an AST refactor that makes it easier for maintain Iced.

Paul Miller

@maxtaco I think that making it work with try/catch is an awesome idea and with it async programming would be much more easy and fun.

What do you think about expecting callbacks to receive two arguments, as node.js convention says? e.g. callback(error, result). It would be even more awesome than in F#:

F#

open System.Net
open Microsoft.FSharp.Control.WebExtensions

let urlList = [ "Microsoft.com", "http://www.microsoft.com/"
                "MSDN", "http://msdn.microsoft.com/"
                "Bing", "http://www.bing.com"
              ]

let fetchAsync(name, url:string) =
    async { 
        try
            let uri = new System.Uri(url)
            let webClient = new WebClient()
            let! html = webClient.AsyncDownloadString(uri)
            printfn "Read %d characters for %s" html.Length name
        with
            | ex -> printfn "%s" (ex.Message);
    }

let runAll() =
    urlList
    |> Seq.map fetchAsync
    |> Async.Parallel 
    |> Async.RunSynchronously
    |> ignore

runAll()

IcedCoffeeScript

links =
  Microsoft: 'microsoft.com'
  MSDN: 'msdn.microsoft.com'
  Bing: 'www.bing.com'

fetch = (url, callback) ->
  request = await http.get host: url, defer response
  result = ''
  request.on 'error', (error) ->
    callback error
  response.on 'data', (chunk) ->
    result += chunk.toString()
  response.on 'end', ->
    callback null, result.length

await
  for title, url in links
    try
      fetch url defer result
      console.log "Read #{result.length} characters for #{title}"
    catch error
      console.log "Fetch error in #{title}: #{error}"
Devon Govett

@maxtaco why doesn't your repo accept issues? Here's a quick one I found:

fn = ->
  await setTimeout defer(), 1000
  alert arguments[0]

fn 'hi'

The bug is that you aren't handling arguments properly. You might have to store a reference to it or something like you already do with this. This example ends up alerting null after 1 second rather than 'hi'.

Otherwise, this is shaping up quite nicely. I especially like the stack trace features you added. I'm sure the compiled JS could be cleaned up some though, and I will probably have suggestions on that soon. :)

Andrew Kelley

:+1: this belongs in coffee-script

One suggestion: Maybe we can render the simple case simply. For example, if I do something like this:

# Search for 'keyword' on twitter, then callback 'cb'
# with the results found.
search = (keyword, cb) ->
  host = "http://search.twitter.com/"
  url = "#{host}/search.json?q=#{keyword}&callback=?"
  await $.getJSON url, defer json
  cb json.results

There's no reason to add all that complicated stuff. Iced could use a simple callback, like this:

var search;

search = function(keyword, cb) {
  var host, url;
  host = "http://search.twitter.com/";
  url = "" + host + "/search.json?q=" + keyword + "&callback=?";
  return $.getJSON(url, function(json) {
    return cb(json.results);
  });
};

I know that could be a nightmare to implement in the compiler, though.

Andrew Kelley

@maxtaco, if you do iced --runtime window, it should only declare iced once so that several files can use it. Right now it's the same thing as inline, except it uses window.iced instead of a local variable.

Andrew Kelley

@maxtaco, and here's a failing test case:

atest "nested await", (cb) ->
   a = 10
   b = 11
   await 
     foo(a, defer x)
     await foo(b, defer y)
     foo(y, defer z)

   cb(x == a and y == b == z, {})
Maxwell Krohn

@devongovett, I've enabled issue reporting, didn't realize it wasn't on by default. Thanks for that bug report, I'll puzzle it over.

@superjoe30, my original thought on nested awaits is that they should cause a compiler error. I don't think they're very useful and you can always introduce intermediary anonymous functions to be explicit about what's going on. But your example can make sense. Issue the first foo and the second in parallel, and issue the third after the second completes. Exit the await once the first and third have completed. This seems weird to me, but you're right, it ought to work or complain that it can't.

I'll make two issues over in the branch.

Alan Gutierrez

-1 I've embraced Streamline + CoffeeScript. I prefer that syntax. Adding this to the core of CoffeeScript would be a such a disappointment.

weepy

Is there not something more basic/fundemental that can be added to CoffeeScript that would enable users to build async frameworks upon? It seems to me very much again CoffeeScript to include libs like tame/streamline in the core. I would have thought/hoped that there would be some kind syntax that could be added to enable them though ?

Alan Gutierrez

@weepy To clarify, I'm not arguing for Streamline over Tame. I'm saying that having geared by development to Streamline, with CoffeeScript, and builds that take care of everything, I don't want to see Tame become the default because someone petitioned to have it rolled into the core.

weepy
weepy
  • yes I got that ^_^

and to be clear I wasn't arguing for either to be in core - just for
something more 'fundemental'. E.g in Kaffeine, a function with !
allows it to be called async:

res.send({
  count: User.count!,
  puzzles: User.puzzles!("today")
})
=>
User.count(function(_0) { User.puzzles("today", function(_1) { res.send({
    count: _0,
    puzzles: _1
  }) }) })

or

sum = add!(3, 4).length
=>
add(3, 4, function(_0) { sum = _0.length })

Now this is pretty neat, and allows various expression to be used in
as though they were sync. However it is only fairly simple sugar and
won't properly handle parellel style like:

for i, val in A
  $.getJSON! val
Andrew Kelley

I'm starting to see this from the perspective that Iced should be a library. I don't think you'd even have to change the syntax that much, really.

Jeremy Ashkenas
Owner

Hey folks. Just ran across some prior art for this kind of thing -- which I don't recall seeing before -- StratifiedJS' take:

https://github.com/onilabs/coffee-script

... if you're interested in this ticket, it's worth a look.

Satoshi Murakami
Collaborator
Maxwell Krohn

Stratified definitely predated the original tamejs stuff by some months. I think it's doing CPS conversions and solving the callback pyramid problem. This is a taste issue, but I prefer the iced syntax for two reasons: (1) fewer keywords/syntax changes; (2) callbacks are represented explicitly. That callbacks are represented explicitly gives the ability to get interesting control flow patterns that Stratified accomplishes with further keywords. Take for instance the iced.Rendezvous class which allows for the programmer to act after the first computation completes; or the icedlib.Pipeliner class which makes it easy to set up a windowed pipeline of computations; or the icedlib.timeout class which allows the programmer to "split" an arbitrary events into two racing events, allowing for a simple implementation of timeouts. In all cases, the libraries are explicitly playing around with callbacks, so the language internals don't have to.

Andy Joslin

+1, I really like this.

Matt Tagg

+1

No comments for six months, weird. Anyway I really feel Iced is a rather elegant and exciting addition to CS.

Being a superset, I don't think we'd loose much merging it back in, and we would gain a whole lot. CS is already a fragmentation in the JS ecosystem and would be great to unify the talents in the community. Additionally, having another fork further fragments the tooling, a real issue for users when deciding on suitability (which affects adoption).

my humble 2c :p

Nami-Doc
Collaborator

You may be interested in looking to #350

Alex Kocharin

CS is already a fragmentation in the JS ecosystem and would be great to unify the talents in the community.

CS should have modular system instead, so anybody could make extensions to CS that works with each other.

But I'd like to see tame here anyway.

Sokolov Yura

I used IcedCoffeeScript in prototype, and leave several await in production - all works great.
I found no any bug with Iced.
I remove await from linear-callback code cause CoffeScript give's already pretty syntax for callbacks.
But I leave await in cycles and conditions, cause they looks much prettier in this way.

Jorge Barata González

+1

Nami-Doc
Collaborator

You may be interested by #350

Paul Miller

I was the second commenter in this thread. Iced coffeescript does not solve fundamental problem of async js code: composition.

The problem is absolutely effectively solved by promises:

async.sortBy(files, sort)
  .then (sorted) =>
    async.map sorted, @map
  .then (mapped) =>
    async.reduce mapped, null, reduce
  .then (reduced) =>
    @write reduced

You can use promises today and the result code is cute.

Sokolov Yura

How does this iced code is mapped to promizes?

    console.log "prepare"
    await
      for file in files
        fs.unlink file, defer()
    console.log "done"
Paul Miller

@funny-falcon

Q.all(files.map (file) -> Q.nfcall fs.unlink, file).then(-> console.log 'done')
  1. You call fs.unlink with promise wrapper (Q.nfcall) on each file.
  2. You turn an array of promises to one promise with Q.all and then just use it simply.

Also, consider how your 4-line-coffee-code translates to 20 lines of js.

Sokolov Yura

@paulmillr
I know how my example translates to 20 lines of js - in a very genius way :-P

While I know, that iced-coffee-script will be certainly slower that your version,
I'll prefer it, cause it way more readable.

For simpler cases coffee-script already provides very pretty syntax for functions.
And for this rare case readability counts more than performance.

Nami-Doc Nami-Doc closed this June 08, 2013
Nami-Doc
Collaborator

Closing for #2762

Maxwell Krohn maxtaco referenced this pull request in maxtaco/coffee-script August 19, 2013
Closed

Would IcedCS ever be merged into Coffeescript? #85

Nami-Doc Nami-Doc referenced this pull request October 15, 2013
Closed

Coffee.coffee -> .js #3204

Andrew Kelley andrewrk referenced this pull request in kchmck/vim-coffee-script January 29, 2012
Closed

make `await` and `defer` keywords #73

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.