Binary Format

Binary Format Specification

Overview

FLEXON's binary format is designed for maximum efficiency while maintaining readability and extensibility. This document details the binary format specification.

File Structure

┌─────────────┬──────────────┬─────────────┐
│   Header    │ Content Size │   Content   │
│   (8 bytes) │   (4 bytes) │  (variable) │
└─────────────┴──────────────┴─────────────┘

Header Format

┌───────────┬───────────┬───────────┬───────────┐
│  Magic    │ Version   │   Flags   │ Reserved  │
│ (4 bytes) │ (1 byte) │ (1 byte)  │ (2 bytes) │
└───────────┴───────────┴───────────┴───────────┘

• Magic: 0x464C584E ("FLXN" in ASCII)
• Version: Current format version (e.g., 0x01)
• Flags: Bit flags for file properties
  • Bit 0: Compression enabled
  • Bit 1: Schema validation required
  • Bit 2: Contains custom types
  • Bits 3-7: Reserved

Type Encoding

Basic Types

Type	Code	Format	Example
Null	0x00	No value	0x00
Bool	0x01	1 byte (0=false, 1=true)	0x01 0x01
Int32	0x02	4 bytes, little-endian	0x02 0x2A 0x00 0x00 0x00
Int64	0x03	8 bytes, little-endian	0x03 0x2A...
Float	0x04	4 bytes, IEEE 754	0x04 0x00 0x00 0x80 0x3F
Double	0x05	8 bytes, IEEE 754	0x05 0x00...
String	0x06	Length + UTF-8 bytes	0x06 0x05 "Hello"
Binary	0x07	Length + raw bytes	0x07 0x04 0xDE 0xAD 0xBE 0xEF

Complex Types

Array

┌────────┬──────────┬─────────┬─────────┬─────┬─────────┐
│ Type   │  Length  │ Item[0] │ Item[1] │ ... │ Item[n] │
│ (0x10) │ (4 bytes)│ (var)   │ (var)   │     │ (var)   │
└────────┴──────────┴─────────┴─────────┴─────┴─────────┘

Object

┌────────┬──────────┬──────────┬────────┬─────┬──────────┬────────┐
│ Type   │   Size   │  Key[0]  │ Val[0] │ ... │  Key[n]  │ Val[n] │
│ (0x11) │ (4 bytes)│ (string) │ (var)  │     │ (string) │ (var)  │
└────────┴──────────┴──────────┴────────┴─────┴──────────┴────────┘

Extended Types

DateTime

┌────────┬────────────┬──────────┐
│ Type   │   Ticks    │  Offset  │
│ (0x20) │ (8 bytes) │ (2 bytes)│
└────────┴────────────┴──────────┘

UUID

┌────────┬────────────────┐
│ Type   │     Bytes     │
│ (0x21) │   (16 bytes)  │
└────────┴────────────────┘

Compression

When compression is enabled (Flag bit 0):

┌────────────┬────────────┬──────────────┐
│ Header     │ Size       │ GZIP Content │
│ (8 bytes)  │ (4 bytes) │  (variable)  │
└────────────┴────────────┴──────────────┘

Schema Validation

When schema validation is required (Flag bit 1):

┌────────────┬────────────┬──────────┬──────────┐
│ Header     │ Schema Hash│ Size     │ Content  │
│ (8 bytes)  │ (32 bytes)│ (4 bytes)│ (var)    │
└────────────┴────────────┴──────────┴──────────┘

Custom Types

When custom types are present (Flag bit 2):

┌────────────┬───────────┬──────────┬──────────┐
│ Header     │ Type Info │ Size     │ Content  │
│ (8 bytes)  │ (var)     │ (4 bytes)│ (var)    │
└────────────┴───────────┴──────────┴──────────┘

Type Info Format:

┌───────────┬────────────┬──────────┐
│ Type Count│ Type Code  │ Type Name│
│ (2 bytes) │ (1 byte)  │ (string) │
└───────────┴────────────┴──────────┘

Examples

Simple Value

# Integer value 42
0x46 0x4C 0x58 0x4E  # Magic "FLXN"
0x01                 # Version 1
0x00                 # No flags
0x00 0x00            # Reserved
0x05 0x00 0x00 0x00  # Content size (5 bytes)
0x02                 # Int32 type
0x2A 0x00 0x00 0x00  # Value 42

String with Compression

# String "Hello, World!" with compression
0x46 0x4C 0x58 0x4E  # Magic "FLXN"
0x01                 # Version 1
0x01                 # Compression enabled
0x00 0x00            # Reserved
0x0F 0x00 0x00 0x00  # Compressed size
[GZIP compressed content]

Object with Schema

# Object with schema validation
0x46 0x4C 0x58 0x4E  # Magic "FLXN"
0x01                 # Version 1
0x02                 # Schema validation required
0x00 0x00            # Reserved
[32-byte schema hash]
[4-byte content size]
[Object content]

Implementation Guidelines

Reading Data

public class FlexonReader
{
    public Header ReadHeader()
    {
        var magic = ReadInt32();
        if (magic != 0x464C584E)
            throw new FlexonException("Invalid magic number");
            
        var version = ReadByte();
        var flags = ReadByte();
        var reserved = ReadInt16();
        
        return new Header(version, flags);
    }
    
    public object ReadValue()
    {
        var typeCode = ReadByte();
        return typeCode switch
        {
            0x00 => null,
            0x01 => ReadBoolean(),
            0x02 => ReadInt32(),
            0x06 => ReadString(),
            0x10 => ReadArray(),
            0x11 => ReadObject(),
            _ => throw new FlexonException($"Unknown type code: {typeCode}")
        };
    }
}

Writing Data

public class FlexonWriter
{
    public void WriteHeader(bool compressed, bool hasSchema)
    {
        Write(0x464C584E);  // Magic
        Write((byte)1);     // Version
        Write((byte)((compressed ? 1 : 0) | (hasSchema ? 2 : 0)));
        Write((short)0);    // Reserved
    }
    
    public void WriteValue(object value)
    {
        switch (value)
        {
            case null:
                Write((byte)0x00);
                break;
            case int i:
                Write((byte)0x02);
                Write(i);
                break;
            case string s:
                Write((byte)0x06);
                WriteString(s);
                break;
            // ... other types
        }
    }
}

Performance Considerations

Memory Layout

• Align data on natural boundaries
• Use compact representations
• Pool buffers for efficiency

Optimization Tips

1. Use direct memory access when possible
2. Implement buffer pooling
3. Optimize string handling
4. Use SIMD operations where available

Compatibility

Version Changes

• Version 1: Initial format
• Version 2: Added custom types
• Version 3: Enhanced compression

Migration

1. Read version from header
2. Apply appropriate reader
3. Convert if necessary

Security

Validation

• Check magic number
• Validate size fields
• Verify checksums
• Validate type codes

Protection

• Set size limits
• Validate input
• Handle overflows
• Secure sensitive data