-
Notifications
You must be signed in to change notification settings - Fork 2
Getting Started
Spiff needs for you to have two things - a binary file to parse, and a description of the file format. The event dispatching mechanism in Spiff means that you can actually do whatever you want with the data (see TreeBuildingEventListener), but in most cases you'll probably also want some classes that will ultimately represent the contents of the file. The assumption throughout this guide is that you are wanting to bind the contents of the file into some class structure.
File formats are described in an .adf file (adf = Arbitrary Data Format - the extension is a convention rather than a necessity). Let's take a look at mapping a relatively simple file format - a 24-bit Bitmap (.bmp) file. You can read more about the .bmp file format at wikipedia
Here's the first 14 bytes of a bmp file
.setorder LITTLE_ENDIAN
.group(bitmapFileHeader) {
# expect to find the string 'BM' at the start
string('BM', US-ASCII) bfType
int bfSize
short bfReserved1
short bfReserved2
int bfOffBits
}
The most basic thing you can do is to specify a list of datatypes, in the order in which they occur in the file, followed by a name to give to that field. Names for the fields will be used to find corresponding fields in bound classes, so identifiers use the java convention (alphanumeric starting with an alpha).
Strings come in three flavours: fixed length, null terminated, and literals. In this case, we've used a literal - that is, we expect the string to have a fixed value ("BM" - 0x424D in hex). We've also explicitly specified that it will be encoded as US-ASCII. The encoding is an optional parameter, if it wasn't there the string would be decoded using the default encoding for your platform. The file includes a comment, preceded with a #. Comments can appear on their own line or at the end of a line, but multiline comments are not supported.
The whole of the header is wrapped in a group instruction. In an adf file, instructions that control the flow of execution, as opposed to reading data from the file, start with a period ("."). The group instruction is used to partition the data into logical objects. Usually, but not necessarily, a group will correspond to an object. In this case, you can imagine that we have a Bitmap class (not explicitly declared in the adf file - we'll get to that), which is composed of a BitmapFileHeader class, and that BitmapFileHeader class has 5 fields, corresponding to the pieces of data defined within it.
Note that at the very top of the file, we use a setorder instruction to specify that bytes in this file format will be little endian. The setorder instruction can be used at any point in the file to switch endianness, but in most (sane) cases, you'll just want to specify it at the top. If we don't specify it, the default is BIG_ENDIAN.
For the Bitmap file, the next 40 bytes are pretty much the same as the file header section:
.group(bitmapInfoHeader) {
int biSize
int biWidth
int biHeight
short biPlanes
short biBitCount
int biCompression
int biSizeImage
int biXPelsPerMeter
int biYPelsPerMeter
int biClrUsed
int biClrImportant
}
Now things get a little more interesting. In a Bitmap, the Info Header section may be followed by a colour table, for 1, 4, and 8 bit bit counts. For our example, a 24-bit bit count, that section is not present. We can use conditionals in the file to either try and read a colour table or not, depending on the value of the biBitCount field.
.if(biBitCount != 24) {
.set numberOfColours pow(2, biBitCount)
.repeat(numberOfColours) {
.group(rgbQuad) {
ubyte rgbBlue
ubyte rgbGreen
ubyte rgbRed
ubyte rgbReserved
}
}
}
The if instruction works exactly as you'd expect it to. The argument is an expression that evaluates to a boolean, using all the usual operators you'd find in Java. You can access the value of fields already parsed from the file just by using the name of the field. Note that names are not scoped or namespaced, so you should avoid duplicating names. Where a field is inside a loop, only the last executed value is kept.
If the bitmap is 1, 4 or 8 bit, the colour table will have 2, 16, or 256 colours accordingly. You can set arbitrary variables using the set instruction, followed by the name of the variable, and an expression to be evaluated. You can use functions from the java.lang.Math class in your expressions - in this case, we calculate the number of colours as 2 to the power of the number of bits (i.e. 2^8 == 256)
A repeat instruction, which presumably needs no explanation, is used to loop over the rgbQuad group the appropriate number of times. Inside the rgbQuad group are four ubyte instructions, which are unsigned bytes. There are also ushort and uint instructions. When binding to classes, unsigned values are cast to the next biggest datatype, so a ubyte becomes a short, ushort is an int, and uint is a long.
The final part of the file is the pixel data. We can parse the data for a 24-bit bitmap like this:
.repeat(biHeight) {
.mark rowStart
.repeat(biWidth) {
.group(pixelData) {
ubyte rgbBlue
ubyte rgbGreen
ubyte rgbRed
}
}
.mark rowEnd
.skip 4 - ((&rowEnd - &rowStart) % 4)
}
In a 24-bit bitmap, each pixel is represented by 3 unsigned bytes, one for each colour channel. Pixels are arranged in a (nearly) obvious manner, with consecutive bytes representing the pixels along one row, followed by the pixel data for the next row and so on. Rows are actually ordered bottom to top (i.e. the bottom row of pixels comes first in the file), but obviously you still need to parse the file in a sequential manner, and sort out the order in your application. In the snippet above, we simply use a repeat to loop over the rows of pixels, and an inner loop within that to loop over the columns. Inside that, we parse the pixel data in much the same way as the colour table.
The added gotcha is that each row of the bitmap file must be padded to a 4-byte boundary. So, if the bitmap is 23 pixels wide, there will be 69 bytes of data, which must then be padded with three extra 00 bytes to a total of 72 bytes, before the next row of data begins. By preceding field names in expressions with an ampersand ('&'), you can access the position in that file at which the field was last read, instead of its value. You can also use a mark instruction to mark arbitrary points in the file, which can then be referenced in expressions.
To solve the padding problem, we mark the position of the start of the row, read the pixel data for the row, and mark the end of the row. That means that we can calculate the size of the row, modulo 4, and subtract that from 4 to find out how many bytes we need to skip (note that rowEnd and &rowEnd would return the same value). Luckily, Spiff has a skip instruction, which skips a number of bytes relative to the current position.
Put all this together, and you have an adf file which describes how to parse a 24-bit bitmap file. Let's do something with it.
#Class binding Spiff uses an event dispatching mechanism to allow you to plug in listeners that can be notified when data is read from the file. You could write your own listeners, but Spiff comes with a ClassBindingEventListener, which allows you to bind data from the file into classes.
For this example, we'll have a Bitmap class which will hold the data.
public class Bitmap {
private BitmapFileHeader bitmapFileHeader;
private BitmapInfoHeader bitmapInfoHeader;
@BindingCollection(value="pixelData", type=PixelData.class)
private List<PixelData> thePixels;
}
The bitmapFileHeader and bitmapInfoHeader fields match the names we declared in the first two group instructions. Spiff will automatically bind those groups to the corresponding objects, and assume that the data within those groups will bind to data within those objects.
public class BitmapFileHeader {
@Binding("bfType")
private String bmMagicString;
private int bfSize;
private short bfReserved1;
private short bfReserved2;
private int bfOffBits;
}
Note that even private fields can be bound. In the BitmapFileHeader, we have a field that has a different name to it's corresponding declaration in the adf file. That's no problem, we can use a @Binding annotation to tell Spiff that this field in the class should be bound to the bfType field in the adf.
For the pixel data, we can bind multiple instances of a field to a collection. The @BindingCollection annotation lets Spiff know that it needs to add an instance to the list every time it encounters the corresponding group instruction.
That's really all you need. Then it's just a case of wiring it all together:
public static void main(String[] args) {
ClassBindingEventListener<Bitmap> eventListener = new ClassBindingEventListener<Bitmap>(Bitmap.class);
BinaryParser parser = new BinaryParser(eventListener);
parser.parse(new File("bitmap.adf"), new File("myBitmap.bmp"));
Bitmap theBitmap = eventListener.getResult();
}
All we do is to create a ClassBindingEventListener with the appropriate class, and construct a BinaryParser instance with that event listener. The parse() method takes two parameters, the adf file describing the format, and the file to parse. Once that's done, you can ask the ClassBindingEventListener to give you the result, and you should have a Bitmap, with all the relevant data populated. Could it be simpler?