Hammer Primer (beta-test)
Introduction to using the Hammer parser construction toolkit by Meredith L. Patterson
The Hammer repository is at github.com/UpstandingHackers/hammer
Help us "beta-test" this primer!
These lectures introduce Hammer, a kit for practical construction of secure parsers in C, C++, and other languages used in production programming. The point of Hammer is to save the time and effort of practical industry programmers who need to deal with parsing potentially hostile inputs (which, these days, is pretty much everyone coding any kind of Internet communications).
Hammer needs you to follow a particular programming/design pattern: the code that validates the inputs (and actually defines what the valid inputs are) comes first, before any actions on the data parsed out of these inputs. This is important, because acting on input data before the entire message has been fully parsed and recognized as correctly formed is the most common weakness that leads to memory corruption.
The lectures introduce necessary theory concepts as they go. No special knowledge of parsing is required, only an interest in how to do it right, i.e., with the lowest chance of the parser being exploited by crafted inputs. If you feel that any part of these lectures can be made more accessible by a brief introduction of a relevant concept, please let us know!
This is an early release, and yet Hammer has already moved forward since these lectures were recorded. Whenever today's Hammer works differently than described in the lectures, annotations and/or comments have been added to the video.
Please feel free to contribute writeups on any build configuration problems you encountered and your solutions! Similarly, please feel free to contribute examples of your own parsers.
Table of Contents
- Lecture 1 - Introduction
- Lecture 2 - JSON/HTTP
- Lecture 3 - JSON RPC
- Lecture 4 - Samba Base64 Parsing Crash
- Lecture 5 - Authorization
- Lecture 6 - State Machines
- Lecture 7 - Languages
- Lecture 8 - Installing Hammer
- Lecture 9 - Why Base64?
- Lecture 10 - Unit Tests
- Lecture 11 - Base64 Recognizer
- Lecture 12 - Base64 Decoder
- Lecture 13 - JSON Parser
Lecture 1 - Introduction
Abstract: Parser Combinators and Hammer are introduced, course outline. System requirements:
- Linux, or
- OS X with homebrew or macports, or
- Windows with virtual machine
Lecture 2 - JSON/HTTP
Abstract: JSON, JSON RPC introduced. Sent in HTTP POST request, so course will cover it as well.
Lecture 3 - JSON RPC
Abstract: More detailed look at JSON RPC.
Lecture 4 - Samba Base64 Parsing Crash
Abstract: Installation of samba and swat exploit demonstrated.
Lecture 5 - Authorization
Abstract: Authorization as a state machine.
Lecture 6 - State Machines
Abstract: More detailed look at State Machines.
Lecture 7 - Languages
Abstract: HTTP Basic Authentication described.
Lecture 8 - Installing Hammer
Abstract: How to install hammer and its dependencies.
Lecture 9 - Why Base64?
Abstract: Base64 encoding described, rules of base64. Recognizer of base64 will be created by coding the rules.
Lecture 10 - Unit Tests
Abstract: Base64 recognizer boilerplace and unit tests, how to compile a program with hammer.
Files in lecture_10/
Lecture 11 - Base64 Recognizer
Abstract: Turning the Base64 alphabet rules into a code description.
Files in lecture_11/
Lecture 12 - Base64 Decoder
Abstract: Converting the Base64 Recognizer into a Decoder.
Files in lecture_12/
Changes in Hammer since recording time: 3:50 -- HAction function signatures are now: HParsedToken* func(const HParseResult *p, void *user_data); That is, HAction functions must also take a user_data whether or not they use it. (This is
Lecture 13 - JSON Parser
Abstract: Creating a JSON Parser.
Notes: 13:43 -- act_json_object is changed to act_json_object_main abruptly. Ignore this, act_json_object is correct.
Files in lecture_13/