Merge branch 'master' of git://github.com/creationix/howtonode.org

app.js

@@ -4,6 +4,7 @@ var Connect = require('connect');
 module.exports = Connect.createServer(
   Connect.logger(),
   Connect.conditionalGet(),
+  Connect.favicon(),
   Connect.cache(),
   Connect.gzip(),
   require('wheat')(__dirname)

articles/capturing-packets-in-javascript.markdown

@@ -0,0 +1,124 @@
+Title: Capturing Packets in JavaScript with node_pcap
+Author: Matt Ranney
+Date: Wed Sep 29 2010 23:08:34 GMT-0700 (PDT)
+
+OK, I hear you. Capturing packets is hard and best left to kernel hackers, assembly language programmers, and black hat security
+researches. If you just want to make things for the web using node.js, why should you care?
+
+Pulling packets off the network can show you what your computers are saying to each other without disrupting the flow of or changing any applications. Packet capture is a fantastic debugging tool that will remove a lot of the mystery from writing and running network programs. The point of `node_pcap` is to provide a good HTTP debugging tool and a framework for doing your own network analysis.
+
+There are plenty of ways to do packet inspection these days, but none of them let you interact with your network traffic the way that node lets you write network programs: by writing a few event handlers in JavaScript. `node_pcap` not only let's you capture and process packets in JavaScript, but since it is built on node.js, data from the packets can be easily routed around to web browsers, databases, or whatever else you can think of.
+
+## Example
+
+Here's an example of capturing packets and sending them back to a web browser using WebSocket:
+
+<http://pcap.ranney.com:81/>
+
+If you still aren't convinced, check out how easy it is to write a simple "network grep" type of program using `node_pcap`:
+
+<capturing-packets-in-javascript/example.js>
+
+This program will look at all TCP packets that flow past the default network interface and run the regular expression `matcher` against the data section of the packet. If it matches, the data section will be printed.
+
+Still not convinced? I understand. This packet business can be astonishingly low level compared to the abstractions you are comfortable working with. If this doesn't seem awesome yet, it probably won't until you actually need it. When you can't figure out what your program is doing by just adding log messages, come back and check out what packet capture can do for you.
+
+`node_pcap` exposes packets as JavaScript objects, but it also comes with a few examples that are useful on their own. If you do nothing else, check out `http_trace` and `simple_capture`. Look at the source code and see how they work. It's really easy.
+
+## Installation
+
+Anyway, if you are still here, let's get this sucker installed. The first thing you'll need is `libpcap`. If you are on OSX 10.6, you already have it. If you are on a Linux system that uses `apt-get` to install things, you can get it like this:
+
+    sudo apt-get install libpcap-dev
+
+
+If you are on some other kind of system, I don't know the exact command to install `libpcap-dev`, but it is a very common library that's widely available.
+
+Once you have `libpcap` and node, you just need `npm`. Install `node_pcap` with `npm` like this:
+
+    npm install pcap
+
+This will install the pcap libraries and three executable.
+
+If you want to hack on the code, and I encourage you to do so, use `git` to clone the repository on github:
+
+    git clone git://github.com/mranney/node_pcap.git
+
+You'll still need to use `npm` to build and install the files where they need to go:
+
+    mjr:~/work/node_pcap$ npm install .
+
+To verify that things are working, run:
+
+    sudo simple_capture
+
+It should look something like this:
+
+    mjr:~$ sudo simple_capture
+    libpcap version 1.0.0
+    * en0 10.51.2.125/255.255.255.0
+    fw0 no address
+    en1 no address
+    lo0 127.0.0.1/255.0.0.0
+    00:1c:23:b9:e8:b5 -> ff:ff:ff:ff:ff:ff 10.51.2.10 ARP request 10.51.2.4
+    00:1e:c9:45:e8:30 -> ff:ff:ff:ff:ff:ff 10.51.2.1 ARP request 10.51.2.45
+    00:1a:92:c4:32:d1 -> ff:ff:ff:ff:ff:ff 10.51.2.179 ARP request 10.51.2.126
+
+Your traffic might not be ARP requests, but some packets should be flowing, and you should see one line per packet.
+
+Opening the capture interface on most operating systems requires root access, so most of the time that you run a program using `node_pcap` you'll need to use sudo.
+
+## http_trace
+
+`http_trace` is a tool that distills the packets involved in an HTTP session into higher level events. There are command line options to adjust the output and select different requests. Here's a simple example of looking for any requests that have "favicon" in the URL and showing request and response headers:
+
+<img src="capturing-packets-in-javascript/http_trace_1.jpg" style="float: none"></img>
+
+
+To see the full list of options do:
+
+    http_trace --help
+
+With no arguments, `http_trace` will listen on the default interface for any IPv4 TCP traffic on any port. If it finds HTTP on any TCP connection, it'll start decoding it. You might be surprised by how many HTTP connections your computer is making that you didn't know about, especially if you run OSX. Fire it up and see what you find.
+
+## Solving Problems
+
+Here's why you need all of this. Let's say you have a node program that makes an outgoing connection, but the outgoing connection doesn't seem like it is working. This reason in this case is that a firewall rule is filtering the traffic. Here's how to detect it:
+
+<img src="capturing-packets-in-javascript/http_trace_2.jpg" style="float: none"></img>
+
+The `--tcp-verbose` option will expose events for TCP connection setup, close, and reset. It'll also let you know about SYN retries and packets retransmissions. SYN retry happens when a new TCP connection is getting set up, but the other side isn't responding. Retransmissions occur when packets are dropped by the network, and TCP on either end of the connection resends data that has already sent. If data is moving slowly, but you don't appear to be out of CPU, turn on `--tcp-verbose` and see if you are getting retransmissions or SYN retries. If so, you can blame the network and not your node program.
+
+Another common case is when the data going over the network isn't quite the data you were expecting. Here's a simple example using curl from the command line. Let's say you wanted to send some JSON to your local CouchDB, but CouchDB keeps rejecting it.
+
+    mjr:~$ curl -X POST 'http://localhost:5984/test' -H "Content-Type: application/json" -d {"foo": "bar"}
+    {"error":"bad_request","reason":"invalid UTF-8 JSON"}
+
+That looks like pretty well-formed JSON, so what's going on here? Run `http_trace` with the --bodies option to dump the request and response body. Since this is a connection to `localhost`, we need to explicitly listen on the loopback interface.
+
+<img src="capturing-packets-in-javascript/http_trace_3.jpg" style="float: none"></img>
+
+Here we can see that the request body was simply, "{foo:", which is clearly not valid JSON. The problem in this case is that the shell and curl couldn't figure out what part of the command line arguments to use for the POST body, and they got it wrong. This works if quoted properly:
+
+    mjr:~$ curl -X POST 'http://localhost:5984/test' -H "Content-Type: application/json" -d '{"foo": "bar"}'
+    {"ok":true,"id":"b4385e0de2e74df4cdbf21cf6c0009d0","rev":"1-4c6114c65e295552ab1019e2b046b10e"}
+
+## Understanding Higher Level Protocols
+
+`node_pcap` can piece back together a TCP session from individual packets as long as it sees them all go by. It will emit events at TCP connection setup, teardown, and reset.
+
+On top of TCP, it can decode HTTP and WebSocket messages, emitting events for request, response, upgrade, data, etc.
+
+It looks sort of like this:
+
+<img src="capturing-packets-in-javascript/pcap_boxes.png" style="float: none"></img>
+
+You set up `node_pcap` to capture the packets you want, and then you can work with the captured data in JavaScript at whatever level is the most useful.
+
+## Work in Progress
+
+There are a lot of cases that `node_pcap` doesn't handle, and for these you'll need a more complete packet decoder like Wireshark. I'm trying to handle the common case of OSX/Linux, IPv4, TCP, HTTP, and WebSocket first, and then add support for other variants of the protocol stack.
+
+If you like this kind of stuff and want to help expand the protocols that node_pcap understands, patches are certainly welcome.
+
+I hope this software is useful and fun. Thanks for reading.

articles/capturing-packets-in-javascript/example.js

@@ -0,0 +1,15 @@
+var pcap = require("pcap"),
+    pcap_session = pcap.createSession("", "tcp"),
+    matcher = /safari/i;
+
+console.log("Listening on " + pcap_session.device_name);
+
+pcap_session.on('packet', function (raw_packet) {
+    var packet = pcap.decode.packet(raw_packet),
+        data = packet.link.ip.tcp.data;
+
+    if (data && matcher.test(data.toString())) {
+        console.log(pcap.print.packet(packet));
+        console.log(data.toString());
+    }
+});

articles/intro-to-jake.markdown

@@ -0,0 +1,122 @@
+Title: Intro to Jake - JavaScript build tool for Node.js
+Author: Matthew Eernisse
+Date: Wed Sep 29 2010 12:39:01 GMT+0530 (IST)
+
+Jake is a JavaScript build program for Node.js, with capabilities similar to GNU Make or Ruby's Rake. If you've ever built projects with Rake, you'll be very at home using Jake
+
+Jake has the following features:
+
+* Jakefiles are in standard JavaScript syntax
+* Tasks with prerequisites
+* Namespaces for tasks
+* Async execution of tasks
+
+## Installing
+
+**Requirements**: Jake requires Node.js. Of course.
+
+Get Jake:
+
+	git clone git://github.com/mde/node-jake.git
+
+Build Jake:
+
+	cd node-jake && make && sudo make install
+
+Or install with NPM:
+
+	npm install jake
+
+(Or, get the code, and `npm link` in the code root.)
+
+## Basic usage
+
+	jake [options] target (commands/options ...)
+
+You can see what other options Jake supports by doing `jake -h` or `jake --help` Probably the most important option starting off is the `-T` or `--tasks` option that lets you see what tasks are defined in a Jakefile.
+
+## Jakefiles and Jakefile syntax
+
+You define your build tasks in a Jakefile.js (usually in the root of your project) -- by default the `jake` command looks for a Jakefile in the current directory, but you can also point Jake at a specific file using the -f (--jakefile) flag.
+
+In your Jakefile, call `task` to define tasks. Call it with three arguments (and one more optional argument):
+
+	task(name, dependencies, handler, [async]);
+
+Where `name` is the string name of the task, `dependencies` is an array of the dependencies, and `handler` is a function to run for the task.
+
+Here's an example:
+
+<intro-to-jake/sample-jakefile.js>
+
+This is a build tool for Node, so of course we want to support async execution, right? The `async` argument is optional, and when set to true `(async === true)` indicates the task executes asynchronously. Asynchronous tasks need to call `complete()` to signal they have completed.
+
+Here's an example of an asynchronous task:
+
+<intro-to-jake/async-jakefile.js>
+
+Use `desc` to add a string description of the task.
+
+Use `namespace` to create a namespace of tasks to perform. Call it with two arguments:
+
+	namespace(name, namespaceTasks);
+
+Where is `name` is the name of the namespace, and `namespaceTasks` is a function with calls inside it to `task` or `desc` defining all the tasks for that namespace.
+
+Here's an example:
+
+<intro-to-jake/namespace-jakefile.js>
+
+In this example, the foo:baz task depends on both the 'default' and the 'foo:bar' task.
+
+Run these namespaced tasks with `jake [namespace]:[task]`. The above example would be run with:
+
+	jake foo:baz
+
+## 	Passing parameters to jake
+
+You can pass either positional or named parameters to Jake tasks (well, 'named parameters,' the JavaScripty way, in an Object).
+
+Single parameters passed to the jake command after the task name are passed along to the handler as positional arguments.
+
+So you can see this in action, let's set up an 'awesome' task that does nothing but print out the arguments it gets:
+
+	desc('This is an awesome task.');
+	task('awesome', [], function () {
+	  console.log(sys.inspect(Array.prototype.slice.call(arguments)));
+	});
+
+With this example, running `jake` like this:
+
+	jake awesome foo bar baz
+
+You'd get the following output:
+
+	[ 'foo', 'bar', 'baz' ]
+
+Paramters passed to the jake command that contain a colon (:) or equals sign (=) will be added to a keyword/value object that is passed as a final argument to the handler.
+
+With the above Jakefile, you could run `jake` like this:
+
+	jake awesome foo bar baz qux:zoobie frang:asdf
+
+And you'd get the following output:
+
+	[ 'foo'
+	, 'bar'
+	, 'baz'
+	, { qux: 'zoobie', frang: 'asdf' }
+	]
+
+As you might expect if you're used to Make or Rake, running `jake` with no arguments runs the default task.
+
+## Related projects
+
+James Coglan's "Jake": <http://github.com/jcoglan/jake>
+
+Confusingly, this is a Ruby tool for building JavaScript packages from source code.
+
+280 North's Jake: <http://github.com/280north/jake>
+
+This is also a JavaScript port of Rake, but it runs on the JVM-based Narwhal platform.
+

articles/intro-to-jake/async-jakefile.js

@@ -0,0 +1,7 @@
+desc('This is an asynchronous task.');
+task('asynchronous', [], function () {
+  setTimeout(function () {
+    console.log("Yay, I'm asynchronous!");
+    complete();
+  }, 1000);
+}, true);

articles/intro-to-jake/namespace-jakefile.js

@@ -0,0 +1,20 @@
+desc('This is the default task.');
+task('default', [], function () {
+  console.log('This is the default task.');
+  console.log(sys.inspect(arguments));
+});
+
+namespace('foo', function () {
+  desc('This the foo:bar task');
+  task('bar', [], function () {
+    console.log('doing foo:bar task');
+    console.log(sys.inspect(arguments));
+  });
+
+  desc('This the foo:baz task');
+  task('baz', ['default', 'foo:bar'], function () {
+    console.log('doing foo:baz task');
+    console.log(sys.inspect(arguments));
+  });
+
+});

articles/intro-to-jake/sample-jakefile.js

@@ -0,0 +1,5 @@
+desc('This is the default task.');
+task('default', [], function (params) {
+  console.log('This is the default task.');
+  console.log(sys.inspect(arguments));
+});

articles/object-graphs-2.markdown

@@ -0,0 +1,83 @@
+Title: Learning Javascript with Object Graphs (Part II)
+Author: Tim Caswell
+Date: Mon Oct 11 2010 08:24:25 GMT-0700 (PDT)
+Node: v0.2.3
+
+The first article using graphs to describe JavaScript semantics was so popular that I've decided to try the technique with some more advanced ideas.  In this article I'll explain three common techniques for creating objects.  They are constructor with prototype, pure prototypal, and object factory.
+
+My goal is that this will help people understand the strengths and weaknesses of each technique and understand what's really going on.
+
+## Classical JavaScript Constructors
+
+First let's create a simple constructor function with a prototype.  This is the closest thing to a class you're going to find in native JavaScript.  It's extremely powerful and efficient, but doesn't quite work like you would expect if coming from a language with classes.
+
+<object-graphs-2/classical.js#rectangle>
+
+Now let's define a new class of objects called Squares that inherit from Rectangles.  To do inheritance, the constructor's `prototype` has to inherit from the parent constructor's `prototype`.  Here we're overriding `getPerimeter` to make it slightly more efficient and to show how to override functions.
+
+<object-graphs-2/classical.js#square>
+
+Usage is straight-forward.  Just create an instance of each and call a function on each.
+
+<object-graphs-2/classical.js#test*>
+
+This is the resulting data structure.  Dashed lines mean object inheritance.  
+
+![classical](object-graphs-2/classical.dot)
+
+<br style="clear:left"/>
+
+Notice that there is little difference between the `rect` instance and `Square.prototype`.  They are both simply objects that inherit from `Rectangle.prototype`.  JavaScript is just a series of linked objects when you get down to it.  The only objects that are special are functions in that they take parameters and can hold executable code and point to scopes.
+
+## Pure Prototypal Objects
+
+Let's do the same example, but without using constructor functions.  This time we'll just use plain prototypal inheritance.
+
+Let's define a Rectangle prototype that the base pattern for all our objects.
+
+<object-graphs-2/prototypal.js#rectangle>
+
+Now let's define a sub-object called Square that overrides some of the properties to change the behavior.
+
+<object-graphs-2/prototypal.js#square>
+
+To create actual instances of these prototypes, we simply create new objects that inherit from the prototype objects and then set their local state manually.
+
+<object-graphs-2/prototypal.js#test*>
+
+Here is the resultant graph of objects.
+
+![classical](object-graphs-2/prototypal.dot)
+
+<br style="clear:left"/>
+
+This isn't quite as powerful as the constructor + prototype method, but is often much easier to understand since there is less indirection.  Also if you come from a language that has pure prototypal inheritance, you'll be happy to know it's possible in JavaScript too.
+
+## Object Factories
+
+One of my favorite methods for creating objects is to use a factory function.  The difference is that instead of defining a prototype object with all my shared functions and then creating instances of those, I simply call a function that returns a new object every time.  
+
+This example is a super simple MVC system.  The controller function takes in as parameters the model and view objects and outputs a new controller object.  All state is stored in the closure via the scope.
+
+<object-graphs-2/factory.js#controller>
+
+To use this, simply call the function with the desired parameters.  Notice how we can use these directly as event handlers (setTimeout) without having to first bind the function to the object.  Since it (the function) doesn't use `this` internally, there is no need to mess with the value of `this`.
+
+<object-graphs-2/factory.js#usage>
+
+    // Output
+    View now has 5
+    View now has 6
+    View now has 5
+    Saving value 5 somewhere
+    Now hiding view
+
+Here is the object graph that results from this code.  Notice that we have access to the two passed in anonymous objects via the hidden `[scope]` property of the functions.  Or in other words, we have access to `model` and `view` from the closure created by the factory function.
+
+![classical](object-graphs-2/factory.dot)
+
+<br style="clear:left"/>
+
+## Conclusion
+
+There is so much more I want to explore, but I like to keep these articles somewhat short and bite-size.  If there is demand, I'll write a part three explaining how to do ruby-style mixins and other advanced topics.