SysEx Specifications and Checksums

SysEx Specifications and Mathematical Checksum Proofs

System Exclusive (SysEx) messages (0xF0 ... 0xF7) enable manufacturers to transmit proprietary voice parameters, waveform tables, and microcode configurations over MIDI. Because MIDI status bytes occupy 0x80 through 0xFF, all SysEx payload bytes must strictly keep bit 7 cleared (0x00 to 0x7F).

---

1. Roland 7-Bit Two's Complement Checksum Theory

Note

Address and Payload Boundaries: In Roland SysEx protocol, the checksum calculation window strictly begins after the command byte (index 5) and includes all address and data payload bytes. The start status byte (0xF0), manufacturer ID (0x41), device ID, model ID, command byte, checksum byte itself, and end status byte (0xF7) are strictly excluded from the sum.

Roland Corporation (D-50, JV-1080, JV-2080, MKS-70 V4, XP-50, JD-Xi) implements a 7-bit two's complement additive identity checksum across address and data blocks.

Architectural Processing Pipeline

flowchart LR
    subgraph PacketStream ["Roland SysEx Frame"]
        H1["F0 (Start)"]
        H2["41 (Roland ID)"]
        H3["Device ID"]
        H4["Model ID"]
        H5["Command ID"]
        A["Address (3-4 bytes)"]
        D["Data Payload"]
        C["Checksum Byte"]
        E["F7 (End)"]
    end

    A --> SumNode["Sum Address & Data Bytes"]
    D --> SumNode
    SumNode --> Mod["Modulo 128 (sum % 128)"]
    Mod --> Diff["Subtract from 128 (128 - rem)"]
    Diff --> Mask["Bitwise AND 0x7F (& 0x7F)"]
    Mask --> ChecksumOutput["Resulting Checksum Byte"]
    ChecksumOutput -. "Compare / Validate" .-> C

Loading

Formal Mathematical Specification

Let $\mathbf{A} = (A_0, A_1, \dots, A_{M-1})$ represent the sequence of $M$ address bytes, and $\mathbf{D} = (D_0, D_1, \dots, D_{N-1})$ represent the sequence of $N$ data payload bytes, where:

$$\forall x \in \mathbf{A} \cup \mathbf{D}, \quad x \in \mathbb{Z} \cap [0, 127]$$

The total accumulated packet sum $\Sigma$ is given by:

$$\Sigma = \sum_{m=0}^{M-1} A_m + \sum_{n=0}^{N-1} D_n$$

The modulo-128 residue $R$ is defined by:

$$R = \Sigma \pmod{128} = \Sigma - 128 \left\lfloor \frac{\Sigma}{128} \right\rfloor$$

The Roland 7-bit checksum $C_{\text{Roland}}$ is the two's complement of $R$ in base-128:

$$C_{\text{Roland}} = (128 - R) \pmod{128} \equiv -\Sigma \pmod{128}$$

Verification Algebraic Identity

The hardware receiver verifies packet integrity by summing the address bytes, payload bytes, and the received checksum byte $C_{\text{rx}}$:

$$V_{\text{packet}} = \left( \sum_{m=0}^{M-1} A_m + \sum_{n=0}^{N-1} D_n + C_{\text{rx}} \right) \pmod{128}$$

If and only if $C_{\text{rx}} = C_{\text{Roland}}$, we have:

$$V_{\text{packet}} = (\Sigma + (-\Sigma \pmod{128})) \pmod{128} \equiv 0$$

Any non-zero result ($V_{\text{packet}} \neq 0$) indicates byte corruption, dropped bits, or frame misalignment, causing the instrument to reject the write request.

Production Python Implementation

def calculate_roland_checksum(payload: bytes) -> int:
    """
    Computes Roland 7-bit two's complement checksum.
    Satisfies: (sum(payload) + checksum) % 128 == 0
    """
    remainder = sum(payload) % 128
    return (128 - remainder) & 0x7F

def verify_roland_frame(frame: bytes) -> bool:
    if len(frame) < 8 or frame[0] != 0xF0 or frame[1] != 0x41 or frame[-1] != 0xF7:
        return False
    
    # Address + Data payload spans index 5 up to checksum (index -2)
    payload_and_checksum = frame[5:-1]
    return (sum(payload_and_checksum) % 128) == 0

---

2. Yamaha DX7 Bitfield Packing Algebra

Tip

Bulk Bank vs Single Voice: When auditioning single patches in real time, Yamaha DX7 uses an unpacked 163-byte frame (155 parameter bytes). When transferring full 32-voice libraries, Yamaha packs each voice into 128 bytes, producing the canonical 4,096-byte payload (4,104 bytes with SysEx framing).

The Yamaha DX7 (Mk1) stores 32 voices in a monolithic 4,096-byte bank dump. In internal memory, an uncompressed voice contains 155 parameters requiring 1,024 bits of information.

flowchart TD
    subgraph RawVoice ["155 Unpacked Voice Parameters (0 to 99 values)"]
        P1["Operator 1..6 EG Rates & Levels"]
        P2["Operator Frequency Coarse/Fine & Detune"]
        P3["Operator Scaling, Velocity, Sensitivity"]
        P4["Pitch EG, Algorithm (1..32), Feedback (0..7)"]
        P5["LFO Wave, Speed, Delay, PMD, AMD"]
        P6["Voice Name (10 ASCII Characters)"]
    end

    RawVoice --> PackEngine["Yamaha 7-Bit Bit-Packer Engine"]

    subgraph PackedVoice ["128-Byte Packed Memory Block"]
        B1["Bitfield Merging (e.g. 2-bit Detune + 5-bit Coarse)"]
        B2["Oscillator Mode (1-bit) + Frequency Fine (6-bit)"]
        B3["10-Byte ASCII Patch Name Block"]
    end

    PackEngine --> PackedVoice
    PackedVoice --> BulkMerge["Concatenate 32 Voices (32 x 128 = 4,096 bytes)"]
    BulkMerge --> DX7Header["Prepend Header: F0 43 00 09 20 00"]
    DX7Header --> DX7Chk["Append 7-Bit Two's Complement Sum Checksum + F7"]
    DX7Chk --> DX7Syx["Final 4,104 Byte .SYX Bank File"]

Loading

Theoretical Information Density Proof

Given 155 parameters where parameter $i$ has maximum value $V_{\text{max}, i}$, the minimum required entropy $H_{\text{total}}$ is:

$$H_{\text{total}} = \sum_{i=1}^{155} \lceil \log_2(V_{\text{max}, i} + 1) \rceil = 1{,}024 \text{ bits}$$

Expressed in 8-bit octets:

$$N_{\text{bytes}} = \frac{1{,}024 \text{ bits}}{8 \text{ bits/byte}} = 128 \text{ bytes}$$

Because $128 \times 32 \text{ voices} = 4{,}096 \text{ bytes}$, Yamaha achieves optimal $100%$ storage utilization in physical 4 KB SRAM chips.

Bitfield Merge Transformation

For composite parameters sharing a single byte (e.g., Detune $\in [0, 14]$ (4 bits) and Oscillator Frequency Coarse $\in [0, 31]$ (5 bits)):

$$B_{\text{packed}} = (P_{\text{detune}} \ll 5) \mid (P_{\text{coarse}} \land 0\text{x}1F)$$

During unpacking:

$$P_{\text{coarse}} = B_{\text{packed}} \land 0\text{x}1F$$

$$P_{\text{detune}} = (B_{\text{packed}} \gg 5) \land 0\text{x}0F$$

Yamaha 7-Bit Bulk Checksum

$$\Sigma_{\text{DX7}} = \sum_{k=0}^{4095} \text{Byte}_k$$

$$C_{\text{DX7}} = (128 - (\Sigma_{\text{DX7}} \land 0\text{x}7F)) \land 0\text{x}7F \equiv -\Sigma_{\text{DX7}} \pmod{128}$$

---

3. Korg 7-to-8 Bit Nibblizing Transformation

Synthesizers such as the Korg M1, MS2000, and microKORG map 8-bit binary payloads across 7-bit MIDI data channels using an octet-expansion mapping.

flowchart LR
    subgraph EightBit ["Original 8-bit Data (7 bytes)"]
        D0["Byte 0 [b7..b0]"]
        D1["Byte 1 [b7..b0]"]
        D2["Byte 2 [b7..b0]"]
        D3["Byte 3 [b7..b0]"]
        D4["Byte 4 [b7..b0]"]
        D5["Byte 5 [b7..b0]"]
        D6["Byte 6 [b7..b0]"]
    end

    EightBit --> Enc["Korg 7-to-8 Encoder"]

    subgraph SevenBit ["Transmitted MIDI Data (8 bytes)"]
        H["Header Byte: [0, D6[7], D5[7], D4[7], D3[7], D2[7], D1[7], D0[7]]"]
        M0["Byte 0 [0, b6..b0]"]
        M1["Byte 1 [0, b6..b0]"]
        M2["Byte 2 [0, b6..b0]"]
        M3["Byte 3 [0, b6..b0]"]
        M4["Byte 4 [0, b6..b0]"]
        M5["Byte 5 [0, b6..b0]"]
        M6["Byte 6 [0, b6..b0]"]
    end

    Enc --> SevenBit

Loading

Mathematical Mapping Functions

Given 7 bytes of unconstrained 8-bit data: $\mathbf{D} = (D_0, D_1, \dots, D_6) \in [0, 255]^7$.

The header byte $H$ extracts the most significant bit (MSB) from each byte:

$$H = \sum_{k=0}^{6} \left( \left\lfloor \frac{D_k}{128} \right\rfloor \right) \cdot 2^k = \sum_{k=0}^{6} ((D_k \gg 7) \land 1) \cdot 2^k$$

The corresponding 7 transmitted data bytes $M_k$ retain their lower 7 bits:

$$M_k = D_k \pmod{128} = D_k \land 0\text{x}7F \quad \forall k \in {0, \dots, 6}$$

Reconstruction Function (Receiver Unpacking)

The receiver reconstructs the original 8-bit byte sequence $D_k$ via:

$$D_k = M_k \mid \left( ((H \gg k) \land 1) \ll 7 \right) \quad \forall k \in {0, \dots, 6}$$

This bijective mapping expands data transmission volume by exactly $\frac{8}{7} \approx 14.28%$, while ensuring every transmitted byte satisfies $M \le 127$.

---

4. Casio CZ Series 4-Bit Nibble Serialization

The Casio CZ-101 and CZ-1000 employ a nibblized encoding for their 135-byte sound memory images. Rather than using bit-shifting masks, each internal 8-bit memory byte is split into two consecutive 7-bit MIDI bytes, isolating the lower 4 bits (LSN) and upper 4 bits (MSN):

flowchart LR
    Byte["Internal 8-bit Parameter: B = [b7..b4, b3..b0]"] --> Split{"Nibble Splitter"}
    Split --> LSN["Byte 1: [0 0 0 0, b3 b2 b1 b0] (Low Nibble)"]
    Split --> MSN["Byte 2: [0 0 0 0, b7 b6 b5 b4] (High Nibble)"]

Loading

The reconstructed internal byte $B_i$ from transmitted sequence $(N_{2i}, N_{2i+1})$ is:

$$B_i = (N_{2i} \land 0\text{x}0F) \mid ((N_{2i+1} \land 0\text{x}0F) \ll 4)$$

This transformation doubles the byte count over the wire ($135 \times 2 = 270 \text{ data bytes}$) plus standard Casio manufacturer header (F0 44 00 00 70 ... F7), ensuring complete immunity against 8th-bit MIDI framing errors.

---

5. MIDI Sample Dump Standard (SDS) Handshake Protocol

For vintage samplers (Akai S900/S1000, Ensoniq Mirage, Sequential Prophet 2000), the non-real-time Universal System Exclusive protocol defines the MIDI Sample Dump Standard (SDS).

SDS uses bidirectional handshaking with 120-byte data packets and 7-to-8 bit audio framing:

sequenceDiagram
    autonumber
    participant Host as bipluk Client
    participant Sampler as Hardware Sampler

    Host->>Sampler: F0 7E <DevID> 01 (Dump Header: Sample #, 12-bit/16-bit, Loop points) F7
    Sampler->>Host: F0 7E <DevID> 7F <Packet#> (ACK: Acknowledge Header) F7
    loop For Every 120-byte Packet
        Host->>Sampler: F0 7E <DevID> 02 <Packet#> <120 Data Bytes> <Checksum> F7
        alt Packet Verified
            Sampler->>Host: F0 7E <DevID> 7F <Packet#> (ACK) F7
        else Checksum Mismatch
            Sampler->>Host: F0 7E <DevID> 7E <Packet#> (NAK: Request Retransmit) F7
        end
    end

Loading

SDS 7-Bit Packet Checksum Formulation

The SDS packet checksum is a 7-bit Exclusive-OR (XOR) fold computed across the packet index and all 120 payload bytes:

$$\text{Checksum}_{\text{SDS}} = \left( \text{Packet Number} \oplus \bigoplus_{i=1}^{120} \text{Payload}_i \right) \land 0\text{x}7F$$

For physical wiring and interface diagnostics, see the Hardware MIDI Troubleshooting Guide. To explore acoustic parameters, read FM Synthesis and Algorithm Mathematics.