packet.idl

# Node Packet

An evented binary packet and structure parser for Node.js.

## Synopsis

Use tar as an example?

## Installing

## Rationale

Node Packet aspires to be the go to library for converting binary data into
JavaScript or CoffeeScript primitives.

The following rationale is offered for the following questions that may or may
not be asked frequenty.

### Why didn't you derive from EventEmitter?

EventEmitter is well suited for classes that have a set of API defined event
types. It does not at all suited for classes that emit of any number of event
types defined by the API user.

## Bit Pattern Language

### Big-Endian Byte Ordering

To define a big-endian byte ordering for a field, prefix the bit size with `b`.
**Mnemonic**: The letter `b` stands for big-endian.

```javascript
"b16"             // Big-endian 32 bit number.
"b8"              // Endianess of a single byte is irrelevant.
"l16, b8"         // Big-endian 16 bit integer followed by a byte.
```

### Little-Endian Byte Ordering

To define a little-endian byte ordering for a field, prefix the bit size with `l`.
**Mnemonic**: The letter `l` stands for little-endian.

```javascript
"l32"             // Little-endian 32 bit integer.
"l8"              // Endianess of a single byte is irrelevant.
"l16, b8"         // Little endian 16 bit integer followed by a byte.
```

### Skipping Bytes

You can skip over bytes your pattern with `x`. **Mnemonic**: The letter `x`
means to cross-out, which is kind of like skipping.

```javascript
"b8, x16, l16"    // A byte separated from a little-endian 16 bit integer by 16
                  // two bytes.
```

### Signed Versus Unsigned Integers

All numbers are assumed to be unsigned, unless prefixed by a negative symbol.
**Mnemonic**: The `-` symbol indicates the possiblity of negative numbers.

```javascript
"-b32"            // Big-endian 32 bit signed integer.
"-l32"            // Little-endian 32 bit signed integer.
"b32"             // Big-endian 32 bit unsigned integer.
```

### IEEE 754 Floating Point Numbers

The number type for JavaScript is the  64 bit IEEE 754 floating point. Node
Packet can read write 64 bit and 32 bit IEEE 754 floating point numbers.

To indicated that the type is a floating point number, use the `f` type suffix.
This is indicated with a `f` suffix.  **Mnemonic**: The letter `f` stands for
*floating-point*.

```javascript
"b64f"            // Big-endian 64 bit IEEE 754 double floating point number.
"l32f"            // Little-endian 32 bit IEEE 754 single floating point number.
```

The floating-point numbers can be stored in little-endian or big-endian byte order.

### Arrays of Raw Bytes

A value will be converted to a big-endian array of bytes if followed by an `a`
suffix. **Mnemonic**: The letter `a` stands for *array*.

```javascript
"l128a"           // Unsigned little-endian 128 bit integer as big-endian array
                  // of bytes.
```

Note that big-endian means that the most signifcant byte is at index `0` of the
array.

This can be surprising if you're expecting the the significance of the bytes
will increase with the index of the array, but then that's what little-endian is
all about. (Big-endian orders like Arabic numerals, while little-endian orders
like offsets into memory.)

If you'd prefer a little-endian array, you can reverse the array with the `@`
qualifier. **Mnemonic**: The `@` character is a swirl and we want to turn our
array around.

```javascript
"l128a@"          // Unsigned little-endian 128 bit integer as little-endian
                  // array of bytes.
```

### Bytes as Hex Strings

A value will be converted to a big-endian hex string if followed by an `h`
suffix. 

```javascript
"l128h"           // Unsigned little-endian 128 bit integer as a big-endian hex
                  // string.
```

Like the raw byte of arrays, the hex string is in big-endian order with the most 
significant bytes at the top. It is less likely that you'll want to reverse a
hex string, but if you, you can use the `@` qualifier. **Mnemonic**: The `@`
character is a swirl and we want to turn our string  around.

```javascript
"l128h@"          // Unsigned little-endian 128 bit integer as a little-endian
                  // hex string.
```

### Arrays of Common Types

It is often the case that a binary format contains an array of values. The most
common case are arrays of bytes represnting ASCII or UTF-8 strings.

Arrays are specified with an subscript and a count. **Menmonic**: The square
brackets are used as array subscripts in JavaScript, and used to declare array
length in other C dialect languages.

```javascript
"b32[4]"          // An array of four big-endian 32 bit numbers.
"b8[16]"          // An array of 16 bytes.
```

The array notation produces an array of the type before the subscript.

### Length Encoded Arrays

Length encoded arrays are specified by joining a count type and a value type
with a `/`. **Mnemonic**: Inspired by Perl's `pack`, this count and type
separator character is as good as any.

```javascript
"b8/b8"           // Length encoded byte array with a byte length.
"l16/b8"          // Length encoded byte array with 16 bit little-endian length.
```

### Zero Terminated Arrays

Zero terminated series are speified with a `z` qualifier. **Mnemonic**: The
letter `z` stands for zero.

```javascript
"l16z"            // Little-endian 16 bit numbers terminated by a zero value.
"b8z"             // Byte string terminated by zero.
```

### Transforms

Often there are transformations that you need to perform on an field to get
it to its final state. You may need to convert a byte array to string of a
particular character encoding, for example. This is done with a tranformation
functions which are specified with a transformation pipeline.

If the transformation is a fixed transformation, you can perform the
transformation by defining a pipeline. A pipeline defines one or more
tranformations that are invoked on the value after parsing and before
serialization. The transformations can accept scalar JavaScript parameters.

```javascript
function str(encoding, name, field, parsing, value) {
    if (parsing) {
        var buffer = new Buffer(array.length)
        for (var i = 0; i < value.length; i++) {
            buffer[i] = value[i];
        }
        var length = value.length
        if (field.terminator) {
            length += field.terminator.length;
        }
        reutrn buffer.toString(encoding, 0, length);
    } else {
        if (field.terminator) {
            value += field.terminator;
        }
        return new Buffer(value, encoding);
    }
}
```

Now you can use the transform in your pattern.

```javascript
"n8z|str('ascii')"      // An ascii string terminated by zero.
"n8z|str('ascii'), b16" // An ascii string terminated by zero followed by a
                        // big-endian 16 bit integer.
```

The `str` transform is defined by default. The transform names are purposely
terse to fit with the brevity of the pattern language.

## Reference

namespace: packet

Node Packet exports the ec2 namespace, which provides the {{Structure}},
{{Parser}} and {{Serializer}} classes.

class: Structure

  parameter: pattern

  The packet pattern.

A structure is an object that both reads from and writes to a buffer
syncrhonously. When reading, buffer must contain the entire contents of the
structure. When writing, the buffer must have enough space to accomdoate the
structure.

function: read

  parameter: buffer

  The byte buffer.

  parameter: offset optional

  The optional offset into the byte buffer. Defaults to `0`.

  parameter: callback

  Called with the parameters read from the byte buffer.

The read method accepts a buffer with an optional offset. The number of
arguments is determined by the structure packet pattern, and must match
the number of arguments expected by the packet pattern.

The callback will be called with the fields read from the buffer, with the
actual count of bytes read as the last parameter.

function: write

  parameter: buffer

  The byte buffer.

Write the arguments to the  buffer at the optional offset. The arguments are
determined by the structure bit pattern. Returns the number of bytes written.

function: sizeOf

Get the size of the structure for the given variable length values. A structure
can have 0 or more variable length values.

The `sizeOf` method does not expect and will not correctly calculate the size of
the structure if fixed size value are given.

class: Parser

function: packet

  parameter: name

  The name of the packet type.

  parameter: pattern

  The packet pattern.

  parameter: callback optional

  Called when a packet of this type has been read from a buffer.

Defines a named packet type optionally assigning a default response for the
packet type.

function: parse

  parameter: nameOrPattern

  Either the name of a named packet or a one off 

function: clone

Clone the packet parser to create a packet parser that shares the named packet
definitions but has its own parser state.

This allows a packet parser prototype to be used to efficently create initialized
instances.

class: Serializer