Lempo is a protocol analysis framework for Emacs, built on the idea that once you have a protocol decoded enough to dump it with fancy highlighting you also have most of the client implementation ready - or if you have a client implementation it’s trivial to sprinkle a bit of highlighting in for useful protocol debugging, and also can apply that with minimal changes for sniffing said protocol.
Emacs is a great environment for doing this kind of analysis - it’s trivial to update analyser functions within a running debug session, which leads to fast results.
Lempo provides methods for managing a debug buffer, to write into it, and pre-defined faces for somewhat consistent highlighting over different protocols. A package with simple debug functionality can just use lempo to create a debug buffer, and write to that. A package that wants to make this available also for sniffing only needs to provide a callback function to distribute incoming data over the existing debug logging functions. The required helper daemon for sniffing is automatically managed by lempo.
Currently the sniffing functionality is focused on serial devices, on UNIX based operating systems - but expanding that to Windows, or other physical data layers is trivial - a proof of concept for sniffing network traffic is included.
The recommended way to use this is to add the top level lisp directory to your load path, and require lempo-autoloads:
(add-to-list 'load-path "/path/to/lempo/lisp")
(require 'lempo-autoloads)This will register the interactive lempo-create function, and make lempo available to other modules using it. Without registered protocols lempo-create will be mostly useless - to use the builtin ones require lempo-pcap for the libpcap functionality, or lempo-tty for TTY debugging.
Once protocols are loaded M-x lempo-create can prompt for the mode to start, and set everything up. In your own projects you’d be defining custom modes, and possibly also have wrappers to the lempo entry points for a smoother UI integration. Depending on what you’re doing you may just use the default bridge helpers, or bring your own.
It may be sensible to bind lempo-menu to a custom keybinding - right now it is not of much use outside of debug buffers, but going forward generic bridge management and switching will move there as well.
M-x lempo-create starts an interactive setup:
- Select a registered protocol (e.g.
tty-bridgeorpcap-bridge). - For TTY protocols, select the physical device (e.g.
/dev/ttyUSB0). - For pcap, the device can be left empty and set later.
Lempo creates a debug buffer, starts the bridge helper daemon if needed, and shows the buffer. For TTY bridges a virtual TTY symlink is also created (e.g. /tmp/lempo-bridge-...) that your native client can connect to.
The debug buffer shows captured traffic with timestamps, direction markers, and protocol-specific highlighting. While in the buffer the following keybindings are active:
| Key | Action |
|---|---|
q | Quit buffer (optionally destroys the bridge) |
c | Clear buffer |
? / h | Open transient menu |
The transient menu offers all relevant control options for the active bridge.
The transient menu (M-x lempo-menu or ? in the debug buffer) provides quick access to common operations. Other useful commands are:
M-x lempo-list— list all active bridgesM-x lempo-toggle— toggle debug logging for the current bridgeM-x lempo-destroy— destroy a specific bridgeM-x lempo-quit— quit the debug buffer, optionally destroying the bridge
For common bridge runtime commands the following helper functions exist:
| Command | Description |
|---|---|
M-x lempo-start-capture | Begin packet capture |
M-x lempo-stop-capture | Stop packet capture |
M-x lempo-set-device | Change capture device (e.g. en0) |
M-x lempo-set-filter | Change BPF filter (e.g. "icmp and host 8.8.8.8") |
All of these operate on the current bridge, or with a prefix argument (C-u) prompt to select a bridge.
For bridge-specific commands that don’t have a dedicated helper (e.g. SETBAUD on TTY bridges), the generic M-x lempo-bridge-command provides completion from the bridge’s announced command set and prompts for optional arguments.
For programmatic use, call lempo-send-command directly:
(lempo-send-command bridge "SETBAUD" "115200")After loading lempo-pcap:
M-x lempo-create RET pcap-bridge RET ;; Device prompt can be left empty — the pcap bridge starts without one
Set the network interface and filter:
M-x lempo-set-device RET en0 M-x lempo-set-filter RET "icmp and host 8.8.8.8"
Start capture:
M-x lempo-start-capture
Captured ICMP packets appear in the debug buffer with highlighted Ethernet, IP, and ICMP headers. To stop:
M-x lempo-stop-capture
To change the filter, stop first, then reconfigure and restart:
M-x lempo-stop-capture M-x lempo-set-filter RET "tcp port 80" M-x lempo-start-capture
When finished, destroy the bridge:
M-x lempo-destroy
Note, the above describes the ideal state - right now some interactive plumbing is a bit broken. To do the above with the non-interactive functions (like in your scratch buffer), first get the UUID from inspecting the running bridge - easiest through the info option in the transient on a debug buffer.
The output will be something like Bridge: bridge-default-1921 (pcap-bridge-143) none -> none (status: running). pcap-bridge-143 is the bridge uuid. With that we can now get a bridge identifier, and configure settings as well as start capture:
(lempo-set-filter (lempo-get "pcap-bridge-143") "icmp")
(lempo-set-device (lempo-get "pcap-bridge-143") "en0")
(lempo-start-capture (lempo-get "pcap-bridge-143"))Lempo includes a checksum analyzer for reverse engineering binary protocols. Given a hex string, it automatically detects and identifies common checksum algorithms:
- Simple: XOR, Sum8, Sum16-LE, LRC, 2’s complement
- CRC-8: standard, polynomial 0x31, init 0xFF variants
- CRC-16: CCITT, CCITT/0000, MODBUS, IBM
Use it interactively with M-x lempo-analyze-checksum. If you have a region selected it will be used as the default input.
Input: deadbeef22 Result: XOR checksum at byte 4 (end), expected 22, computed 22
Note that simple checksums can coincidentally match other algorithms. For example, 504d33610080cf matches both XOR and CRC-8/0x07/FF by chance — verify with multiple packets before concluding.
For the full list of supported algorithms, see Appendix A.
Bridge helpers are small programs to sniff data or traffic without interrupting it or slowing it down, and provide it for consumption by Emacs at its own speed. The included bridge helpers can be built with make bridges from the top level directory in the source tree, or directly with CMake:
mkdir build && cd build
cmake ..
make -j4Dependencies: C++11 compiler, pthreads, util library (PTY support). Optional: libpcap for the pcap bridge. On Linux, the pcap bridge needs capabilities: sudo setcap cap_net_raw,cap_net_admin=eip /path/to/lempo-pcap-bridge.
The rest of this section is only interesting for testing bridge helpers outside of Emacs - for inside Emacs just make sure the binaries are available in the search path.
All bridge helpers take two arguments: A device as first argument, and bridge specific options as second argument. If the argument can also be set via runtime commands they may be replaced by an empty argument (i.e., lempo-bridge "" "" when both device and options can be set later).
At minimum a bridge will show a user friendly startup message and an overview of supported commands (if any). Once traffic is registered the bridge will dump lines showing received or sent frames in the format direction:length:data:
- `TX:16:504d336100800701…` - Client to device (16 bytes)
- `RX:12:504d336200000701…` - Device to client (12 bytes)
If the bridge supports commands they can be sent in the form CMD:identifier:command:arguments. Arguments are optional, the identifier can be an arbitrary string or number - its purpose is to help a client match responses to commands. They therefore don’t have to be unique.
The response to a command well be status:identifier:message, with status either being OK or ERR:
CMD:0:PING OK:0:PONG CMD:0:PENG ERR:0:Unknown command: PENG
Any lines not starting with CMD will be ignored - and any response not in the format described above is meant for human consumption, and can be ignored by analysis tools.
This bridge will:
- create a PTY and symlink it o
virtual_ttylink - bidirectionally relay all data between the PTY and actual device
Currently this bridge is set to a fixed baud rate of 115200, which can be adjusted in the source file.
> lempo-tty-bridge-simple /dev/cu.usbmodem0000000000001 lempo
Created PTY: /dev/ttys016
Created symlink: lempo -> /dev/ttys016
Opened device: /dev/cu.usbmodem0000000000001
CMDS:
Bridge running. Press Ctrl-C to stop.This bridge will do the same things as the simple TTY bridge, but also has a control channel for runtime configuration. This can be used to adjust the baud rate as needed.
> lempo-tty-bridge /dev/cu.usbmodem0000000000001 lempo
CMDS:SETBAUD,QUIT,SHUTDOWN
Created PTY: /dev/ttys016
Created symlink: lempo -> /dev/ttys016
Opened device: /dev/cu.usbmodem0000000000001
Bridge running. Press Ctrl-C to stop.This bridge relays data between two existing TTY devices without creating a PTY. It is intended for use with tty0tty virtual serial port pairs on Linux, where applications require an enumerable serial port (e.g. Java applications with a device dropdown). PTYs created by the regular bridge do not appear in serial port enumerations, so this relay bridge uses existing virtual devices that do.
Unlike the regular TTY bridge, this does not create a new PTY — it opens an existing virtual device (e.g. /dev/tnt1 from tty0tty) and relays bidirectionally to the physical device.
> lempo-tty-relay-bridge /dev/tnt1 /dev/ttyUSB0
CMDS:SETBAUD,QUIT,SHUTDOWN
Opened virtual device: /dev/tnt1
Opened physical device: /dev/ttyUSB0
Relay bridge running. Press Ctrl-C to stop.Your application connects to the other end of the virtual pair (/dev/tnt0).
This bridge can sniff network traffic via libpcap - and therefore needs that available for building. It accepts the usual bpf filter expressions, either on startup or set at runtime.
On Linux additional capabilities are required: sudo setcap cap_net_raw,cap_net_admin=eip /path/to/lempo-pcap-bridge
> lempo-pcap-bridge en0 "icmp and host 8.8.8.8"
CMDS:START,STOP,SETDEV,SETFILTER,QUIT,SHUTDOWN
Pcap bridge started.
Device: en0
Filter: icmp and host 8.8.8.8
Use START command to begin capture. Press Ctrl-C to stop.There are two modes of using this - debug logging from a client, or with sniffing. The second builds on the first one - so when developing a client it’s typically a good idea to just use both.
For just debug logging this is pretty much an opinionated way to handle a debug buffer for you, so
(defvar foo-debug-bridge nil
"Bridge associated with this debug buffer")
(defvar foo-debug-enabled nil
"Control debug logging")
(lempo-register-protocol
(make-lempo-protocol
:name 'foo
:buffer-name "*Foo Debug*"
:bridge 'foo-debug-bridge
:has-daemon nil
:debug-var 'foo-debug-enabled))This sets up a simple handler for foo-protocol to log into *Foo Debug*. bridge and debug-var are optional, but generally a good idea: If your protocol functions can be called from anywhere they won’t have association to a bridge via buffer local variables, so having that variable allows us to have bindings in all your functions that might call lempo which allow lempo to look up the correct bridge:
(defun foo-frobnicate ()
""
(let ((lempo-bridge foo-debug-bridge))
(when foo-debug-enabled
...
)))This example also shows using foo-debug-enabled for conditional execution based on a debug toggle. If variable references are passed in the structure lempo will automatically set those variables to sensible values.
The following snippet will create a new bridge for our protocol, or toggle one if it already exists:
(lempo-toggle-create nil 'foo)When enabled this will show the debug buffer.
(defvar foo-debug-bridge nil
"Bridge associated with this debug buffer")
(defvar foo-debug-enabled nil
"Control debug logging")
(lempo-register-protocol
(make-lempo-protocol
:name 'foo-bridge
:buffer-name "*Foo Debug %s*"
:bridge-callback #'foo-lempo-bridge-callback
:bridge 'foo-debug-bridge
:has-daemon t
:has-tty t
:daemon "lempo-tty-bridge"
:debug-var 'foo-debug-enabled))Protocol implementations write to the debug buffer using the helper functions lempo provides. The simplest way to log a message is with lempo-log-error, which inserts a timestamp and the message:
(lempo-log-error "Unexpected response: %s" response)For more control over formatting, use the insertion helpers directly:
(lempo-insert-timestamp) ; [HH:MM:SS.mmm]
(lempo-insert-send) ; → SEND (green)
(lempo-insert-recv) ; ← RECV (magenta)
(lempo--insert-separator) ; ──────────Hex data can be formatted with faces for consistent highlighting across protocols:
(insert (lempo--format-hex-bytes bytes 'lempo-data-face))The predefined faces are:
| Face | Use |
|---|---|
lempo-magic-face / lempo-sof-face | Magic bytes, start of frame |
lempo-len-face | Length fields |
lempo-cmd-face | Command bytes |
lempo-status-face | Status bytes |
lempo-data-face / lempo-payload-face | Data payload |
lempo-crc-face / lempo-lrc-face | CRC / LRC checksum |
lempo-annotation-face | Human-readable annotations |
lempo-error-face | Errors |
lempo-ethernet-face / lempo-ip-face / lempo-tcp-face / lempo-udp-face / lempo-icmp-face | Network headers |
For capturing and analysing protocols we want to pass data through with minimal delay - but also want the analyser to consume captured frames in its own speed. To make this possible we’re using a simple daemon to cache unprocessed frames before forwarding them, and processing them for analysing in a separate thread:
Client → Virtual PTY ←→ [Relay Thread] ←→ Real Device (TTY)
↓ (ring buffer)
Monitor Thread → stdout
The ring buffer should be set to a sensible size for the analysed protocol so we don’t encounter frame dropping under typical conditions.
The monitor thread should dump complete frames to stdout in the format direction:length:data. direction may be RX or TX, data is lower case hex, length the decimal size of the data portion:
- `TX:16:504d336100800701…` - Client to device (16 bytes)
- `RX:12:504d336200000701…` - Device to client (12 bytes)
Each line must be a complete frame (data transfer, packet, …).
A daemon should correctly handle SIGINT/SIGTERM to clean up and terminate.
The exact line-based protocol is:
- Monitor output:
DIRECTION:LENGTH:HEXDATAwhere DIRECTION isTXorRX, LENGTH is decimal, and HEXDATA is lower-case hex. - Command input:
CMD:UID:COMMAND:ARGSwhere UID is any client-provided string echoed in the response. - Response output:
OK:UID:MESSAGEon success,ERR:UID:MESSAGEon failure.
For more functionality we also want a communication channel to the daemon, which adds to the architecture:
stdin → Command Thread → Command Processor → stdout (responses)
To make things easier a base implementation for complex daemons is provided in lempo-bridge-base. This provides:
- a monitor thread to dump RX/TX data to stdout
- a command reader thread to read commands from stdin
- a command processor thread to parse the commands
- a response writer thread to write command responses to stdout
- a ring buffer for captured packets, size set at compile time (default: 1000 packets)
- queues for incoming commands and outgoing responses
- test command: PING, responds with PONG
To use that base implementation inherit from LempoBridgebase and implement:
int run_Capture()for your protocols capture loop- optionally,
bool process_custom_command(req, resp)to handle bridge specific commands
Basic implementation:
class LempoUartBridge : public LempoBridgeBase {
protected:
// Implement capture logic
int run_capture() override {
// Your capture loop here
while (running) {
// ... capture data ...
push_monitor_data(buffer, length, is_tx);
}
return 0;
}
// Implement custom commands
bool process_custom_command(const CommandRequest& req,
CommandResponse& resp) override {
if (req.command == "MYCMD") {
// Handle command
resp.success = true;
resp.message = "Command executed";
return true;
}
return false; // Unknown command
}
public:
LempoUartBridge(const std::string& arg1) : arg1_(arg1) {}
};Add to cmake for building:
add_executable(lempo-uart-bridge lempo-uart-bridge.cpp)
target_link_libraries(lempo-uart-bridge PRIVATE lempo-bridge-base)The checksum analyzer tests the following algorithms and positions:
| Algorithm | Description |
|---|---|
| XOR | XOR all bytes together |
| Sum8 | 8-bit sum (modulo 256) |
| Sum16-LE | 16-bit little-endian sum |
| LRC | Longitudinal Redundancy Check (two’s complement) |
| 2s-Comp-8 | 8-bit two’s complement of sum |
| 2s-Comp-16 | 16-bit two’s complement of sum |
| Algorithm | Polynomial | Init |
|---|---|---|
| CRC-8 | 0x07 | 0x00 |
| CRC-8/0x31 | 0x31 | 0x00 |
| CRC-8/0x07/FF | 0x07 | 0xFF |
| Algorithm | Polynomial | Init |
|---|---|---|
| CRC-16-CCITT | 0x1021 | 0xFFFF |
| CRC-16-CCITT/0000 | 0x1021 | 0x0000 |
| CRC-16-MODBUS | 0xA001 | 0xFFFF |
| CRC-16-IBM | 0x8005 | 0x0000 |
For each input, the analyzer tests end-of-packet, middle-position, and dual-checksum patterns.
Each bridge supports the common commands PING, STOP, QUIT, and SHUTDOWN (handled by the base class), plus bridge-specific commands:
| Command | Arguments | Description |
|---|---|---|
| SETBAUD | baudrate | Change TTY baud rate (e.g. 115200) |
| QUIT | - | Stop the bridge |
| SHUTDOWN | - | Stop the bridge |
| Command | Arguments | Description |
|---|---|---|
| SETBAUD | baudrate | Change baud rate on both devices |
| QUIT | - | Stop the bridge |
| SHUTDOWN | - | Stop the bridge |
| Command | Arguments | Description |
|---|---|---|
| START | - | Begin packet capture |
| STOP | - | Stop packet capture |
| SETDEV | device | Set capture device (e.g. eth0) |
| SETFILTER | expr | Set BPF filter expression |
| QUIT | - | Stop the bridge |
| SHUTDOWN | - | Stop the bridge |