Skip to content
Read and write binary records for Common Lisp
Common Lisp
Find file
Latest commit 9ec4204 Frode Copied to github
Failed to load latest commit information.
COPYING Copied to github
ChangeLog Copied to github
Makefile Copied to github
README Copied to github
README-bitfield Copied to github
binary-types.asd Copied to github
binary-types.lisp Copied to github
example.lisp Copied to github

README

######################################################################
## 
##    Copyright (C) 2001,2000, 2003
##    Department of Computer Science, University of Tromsø, Norway
## 
## Filename:      README
## Author:        Frode Vatvedt Fjeld <frodef@acm.org>
## Created at:    Wed Dec  8 15:35:53 1999
## Distribution:  See the accompanying file COPYING.
##                
## $Id: README,v 1.1.1.1 2004/01/13 11:13:13 ffjeld Exp $
##                
######################################################################

Binary-types is a Common Lisp package for reading and writing binary
files. Binary-types provides macros that are used to declare the
mapping between lisp objects and some binary (i.e. octet-based)
representation.

Supported kinds of binary types include:

 * Signed and unsigned integers of any octet-size, big-endian or
   little-endian. Maps to lisp integers.

 * Enumerated types based on any integer type. Maps to lisp symbols.

 * Complex bit-field types based on any integer type. Sub-fields can
   be numeric, enumerated, or bit-flags. Maps to lisp lists of symbols
   and integers.

 * Fixed-length and null-terminated strings. Maps to lisp strings.

 * Compound records of other binary types. Maps to lisp DEFCLASS
   classes or, when you prefer, DEFSTRUCT structs.

Typically, a complete binary record format/type can be specified in a
single (nested) declaration statement. Such compound records may then
be read and written with READ-BINARY and WRITE-BINARY.

Binary-types is *not* helpful in reading files with variable
bit-length code-words, such as most compressed file formats. It will
basically only work with file-formats based on 8-bit bytes
(octets). Also, at this time no floating-point types are supported out
of the box.

Binary types may now be declared with the DEFINE-BINARY-CLASS macro,
which has the same syntax (and semantics) as DEFCLASS, only there is
an additional slot-option (named :BINARY-TYPE) that declares that
slot's binary type. Note that the binary aspects of slots are *not*
inherited (the semantics of inheriting binary slots is unclear to me).

Another slot-option added by binary-types is :MAP-BINARY-WRITE, which
names a function (of two arguments) that is applied to the slot's
value and the name of the slot's binary-type in order to obtain the
value that is actually passed to WRITE-BINARY. Similarly,
:MAP-BINARY-READ takes a function that is to be applied to the binary
data and type-name when a record of that type is being read.  A
slightly modified version of :map-binary-read is
:MAP-BINARY-READ-DELAYED, which will do essentially the same thing as
:map-binary-read, only the mapping will be "on-demand": A slot-unbound
method will be created for this purpose.

A variation of the :BINARY-TYPE slot-option is :BINARY-LISP-TYPE,
which does everything :BINARY-TYPE does, but also passes on a :TYPE
slot-option to DEFCLASS (or DEFSTRUCT).  The type-spec is inferred
from the binary-type declaration. When using this mechanism, you
should be careful to always provide a legal value in the slot (as you
must always do when declaring slots' types). If you find this
confusing, just use :BINARY-TYPE.

Performance has not really been a concern for me while designing this
package. There's no obvious performance bottlenecks that I know of,
but keep in mind that all "binary" reads and writes are reduced to
individual 8-bit READ-BYTEs and WRITE-BYTEs. If you do identify
particular performance bottlenecks, let me know.

The included file "example.lisp" demonstrates how to use this
package. To give you a taste of what it looks like, the following
declarations are enough to read the header of an ELF executable file
with the form

   (let ((*endian* :big-endian))
     (read-binary 'elf-header stream)


;;; ELF basic type declarations
(define-unsigned word 4)
(define-signed sword  4)
(define-unsigned addr 4)
(define-unsigned off  4)
(define-unsigned half 2)

;;; ELF file header structure
(define-binary-class elf-header ()
  ((e-ident
    :binary-type (define-binary-struct e-ident ()
		   (ei-magic nil :binary-type
			     (define-binary-struct ei-magic ()
			       (ei-mag0 0 :binary-type u8)
			       (ei-mag1 #\null :binary-type char8)
			       (ei-mag2 #\null :binary-type char8)
			       (ei-mag3 #\null :binary-type char8)))
		   (ei-class nil :binary-type
			     (define-enum ei-class (u8)
			       elf-class-none 0
			       elf-class-32   1
			       elf-class-64   2))
		   (ei-data nil :binary-type
			    (define-enum ei-data (u8)
			      elf-data-none 0
			      elf-data-2lsb 1
			      elf-data-2msb 2))
		   (ei-version 0 :binary-type u8)
		   (padding nil :binary-type 1)
		   (ei-name "" :binary-type
			    (define-null-terminated-string ei-name 8))))
   (e-type
    :binary-type (define-enum e-type (half)
		   et-none 0
		   et-rel  1
		   et-exec 2
		   et-dyn  3
		   et-core 4
		   et-loproc #xff00
		   et-hiproc #xffff))
   (e-machine
    :binary-type (define-enum e-machine (half)
		   em-none  0
		   em-m32   1
		   em-sparc 2
		   em-386   3
		   em-68k   4
		   em-88k   5
		   em-860   7
		   em-mips  8))
   (e-version   :binary-type word)
   (e-entry     :binary-type addr)
   (e-phoff     :binary-type off)
   (e-shoff     :binary-type off)
   (e-flags     :binary-type word)
   (e-ehsize    :binary-type half)
   (e-phentsize :binary-type half)
   (e-phnum     :binary-type half)
   (e-shentsize :binary-type half)
   (e-shnum     :binary-type half)
   (e-shstrndx  :binary-type half)))


For a second example, here's an approach to supporting floats:

  (define-bitfield ieee754-single-float (u32)
    (((:enum :byte (1 31))
       positive 0
       negative 1)
      ((:numeric exponent 8 23))
      ((:numeric significand 23 0))))




The postscript file "type-hierarchy.ps" shows the binary types
hierarchy.  It is generated using psgraph from the CMU lisp
repository:

  (with-open-file (*standard-output* "type-hierarchy.ps"
                   :direction :output
                   :if-exists :supersede)
    (psgraph:psgraph 'binary-type
	             #'(lambda (p)
	                 (mapcar #'class-name
				 (aclmop:class-direct-subclasses
                                   (find-class p))))
                     #'(lambda (s) (list (symbol-name s)))
	             t))
Something went wrong with that request. Please try again.