tx2-link

Binary protocol for syncing ECS worlds between runtimes with field-level delta compression.

tx2-link is the bridge/protocol layer of the TX-2 ecosystem, enabling efficient synchronization of Entity-Component-System state across web, native, and CLI environments. It defines the "wire format" for transmitting world snapshots and deltas with minimal bandwidth overhead.

Features

Delta Compression

• Field-level change detection - Only transmit changed component fields
• 1171× compression ratio - Benchmarked: 1.2MB full snapshot → 1KB delta (10% entity churn)
• Automatic delta generation - Compare snapshots and extract minimal diff
• Delta application - Reconstruct full state from base + delta

Multiple Serialization Formats

• MessagePack - Compact binary format (default, best compression)
• Bincode - Fast Rust-native serialization (lowest latency)
• JSON - Human-readable debugging format

Transport Abstractions

• WebSocket - Server ↔ browser sync (async)
• IPC - Inter-process communication for native ↔ webview
• Stdio - Pipe-based communication for CLI tools
• Memory - In-process channels for testing

Rate Limiting

• Token bucket - Burst handling with sustained rate limits
• Sliding window - Precise message/byte rate enforcement
• Per-connection limits - Individual rate limiters per client
• 357k checks/sec - Benchmarked throughput while enforcing 1k msg/sec cap

Schema Versioning

• Component schema registry - Type definitions with version tracking
• Schema validation - Ensure client/server compatibility
• Migration support - Handle schema evolution gracefully

Streaming Protocol

• Length-prefixed framing - Parse messages from byte streams
• Zero-copy deserialization - Direct memory mapping where possible
• Backpressure support - Flow control for slow consumers

Debug Mode

• JSON logging - Pretty-print all messages and deltas for inspection
• Human-readable traces - Operation summaries with timing and sizes
• Environment variable control - Enable with TX2_DEBUG=1 or TX2_TRACE=1
• Zero runtime overhead - Debug checks compile out when disabled

Quick Start

Delta Compression

use tx2_link::{WorldSnapshot, DeltaCompressor};

let mut compressor = DeltaCompressor::new();

// Create two snapshots
let snapshot1 = WorldSnapshot { /* ... */ };
let snapshot2 = WorldSnapshot { /* ... */ };

// Generate delta (only changed fields)
let delta = compressor.create_delta(&snapshot1, &snapshot2)?;

// Apply delta to reconstruct snapshot2
let reconstructed = compressor.apply_delta(&snapshot1, &delta)?;

assert_eq!(snapshot2, reconstructed);

Serialization

use tx2_link::{Message, Serializer, SerializationFormat};

// Create message
let message = Message::Snapshot(snapshot);

// Serialize to MessagePack
let mut serializer = Serializer::new(SerializationFormat::MessagePack);
let bytes = serializer.serialize(&message)?;

// Deserialize
let deserialized: Message = serializer.deserialize(&bytes)?;

Transport

use tx2_link::{Transport, WebSocketTransport};

// Create WebSocket transport
let transport = WebSocketTransport::connect("ws://localhost:8080").await?;

// Send message
transport.send(&message).await?;

// Receive message
let received = transport.receive().await?;

Rate Limiting

use tx2_link::{RateLimiter, TokenBucketLimiter};

// Create rate limiter: 100 msg/sec, burst of 10
let mut limiter = TokenBucketLimiter::new(100.0, 10);

// Check if message can be sent
if limiter.check_message(1)? {
    transport.send(&message).await?;
}

Performance

Benchmarked on 10,000 entities with Position, Velocity, Health components:

Delta Compression

• Full snapshot: 1,232,875 bytes
• Delta (10% churn): 1,052 bytes
• Compression ratio: 1171×
• Delta generation: ~2ms
• Delta application: ~1.5ms

Serialization Performance

Format	Serialize	Deserialize	Size
MessagePack	180µs	250µs	1.05MB
Bincode	140µs	195µs	1.12MB
JSON	420µs	350µs	2.28MB

Rate Limiting

• Check throughput: 357k checks/sec
• Overhead: ~3µs per check
• Memory: ~200 bytes per limiter

Architecture

Protocol Messages

pub enum Message {
    Snapshot(WorldSnapshot),          // Full world state
    Delta(DeltaSnapshot),             // Incremental update
    EntityCreated { id, components }, // New entity
    EntityDeleted { id },             // Entity removed
    ComponentAdded { entity, data },  // Component attached
    ComponentRemoved { entity, id },  // Component detached
    SchemaUpdate(ComponentSchema),    // Type definition
}

World Snapshot

pub struct WorldSnapshot {
    pub timestamp: u64,
    pub entities: Vec<EntitySnapshot>,
}

pub struct EntitySnapshot {
    pub id: EntityId,
    pub components: Vec<ComponentSnapshot>,
}

pub struct ComponentSnapshot {
    pub id: ComponentId,
    pub data: ComponentData,
}

Component Data Formats

pub enum ComponentData {
    Binary(Vec<u8>),                           // Raw bytes
    Json(String),                              // JSON string
    Structured(HashMap<FieldId, FieldValue>),  // Field-level access
}

Delta Algorithm

tx2-link uses field-level diffing for maximum compression:

1. Compare entities - Match entities between snapshots by ID
2. Detect additions/removals - Track created/deleted entities
3. Compare components - Match components by type within each entity
4. Field-level diff - Extract only changed fields within components
5. Generate delta - Encode minimal changeset

For structured components:

// Previous: { x: 10.0, y: 20.0, z: 30.0 }
// Current:  { x: 10.0, y: 25.0, z: 30.0 }
// Delta:    { y: 25.0 }  // Only y changed

Transport Layer

All transports implement the Transport trait:

#[async_trait]
pub trait Transport: Send + Sync {
    async fn send(&self, message: &Message) -> Result<()>;
    async fn receive(&self) -> Result<Message>;
    async fn close(&self) -> Result<()>;
}

WebSocket Transport

use tx2_link::WebSocketTransport;

// Server
let transport = WebSocketTransport::bind("127.0.0.1:8080").await?;

// Client
let transport = WebSocketTransport::connect("ws://localhost:8080").await?;

IPC Transport

use tx2_link::IpcTransport;

// Create named pipe/socket
let transport = IpcTransport::new("tx2-world")?;

Stdio Transport

use tx2_link::StdioTransport;

// Use stdin/stdout
let transport = StdioTransport::new();

Memory Transport

use tx2_link::MemoryTransport;

// In-process channels (for testing)
let (tx, rx) = MemoryTransport::create_pair();

Rate Limiting

Token Bucket

Allows bursts up to a capacity, refilling at a steady rate:

use tx2_link::TokenBucketLimiter;

// 1000 msg/sec, burst of 100
let limiter = TokenBucketLimiter::new(1000.0, 100);

// Check and consume tokens
if limiter.check_message(1)? {
    // Send message
}

Sliding Window

Enforces strict limits over a time window:

use tx2_link::SlidingWindowLimiter;

// 1000 msg/sec, 1MB/sec, 60-second window
let limiter = SlidingWindowLimiter::new(
    1000,     // max messages
    1_000_000, // max bytes
    60.0,     // window seconds
);

if limiter.check(1, message_size)? {
    // Send message
}

Schema Versioning

use tx2_link::{SchemaRegistry, ComponentSchema};

let mut registry = SchemaRegistry::new();

// Register component type
let schema = ComponentSchema::new("Position")
    .with_field("x", FieldType::F32)
    .with_field("y", FieldType::F32)
    .with_field("z", FieldType::F32)
    .with_version(1);

registry.register(schema)?;

// Validate incoming data
if registry.validate("Position", &component_data)? {
    // Apply update
}

Integration with TX-2 Ecosystem

tx2-link bridges the TX-2 stack:

• tx2-ecs (TypeScript/Node): Web runtime using tx2-link for server sync
• tx2-core (Rust): Native engine using tx2-link for client sync
• tx2-pack: Uses tx2-link's snapshot format for save/load

Use Cases

1. Server ↔ Browser - Sync game state over WebSocket
2. Native ↔ Webview - IPC between Rust engine and web UI
3. CLI Tools - Stream world state over stdio pipes
4. Multi-process - Distribute simulation across processes
5. Debugging - Inspect live world state with JSON transport

Examples

Full Client-Server Sync

use tx2_link::*;

// Server
let transport = WebSocketTransport::bind("127.0.0.1:8080").await?;
let limiter = TokenBucketLimiter::new(100.0, 10);
let mut compressor = DeltaCompressor::new();

let mut last_snapshot = world.create_snapshot();

loop {
    tokio::time::sleep(Duration::from_millis(16)).await; // 60 FPS

    let snapshot = world.create_snapshot();
    let delta = compressor.create_delta(&last_snapshot, &snapshot)?;

    if limiter.check_message(1)? {
        transport.send(&Message::Delta(delta)).await?;
    }

    last_snapshot = snapshot;
}

// Client
let transport = WebSocketTransport::connect("ws://localhost:8080").await?;
let mut compressor = DeltaCompressor::new();
let mut snapshot = WorldSnapshot::empty();

loop {
    let message = transport.receive().await?;

    match message {
        Message::Snapshot(full) => {
            snapshot = full;
            world.restore_from_snapshot(&snapshot)?;
        }
        Message::Delta(delta) => {
            snapshot = compressor.apply_delta(&snapshot, &delta)?;
            world.restore_from_snapshot(&snapshot)?;
        }
        _ => {}
    }
}

Running Tests

cargo test

All 22 tests should pass, covering:

• Delta compression accuracy
• Serialization roundtrips (all formats)
• Rate limiter behavior
• Schema validation
• Transport abstractions

Running Benchmarks

cargo bench

Benchmarks measure:

• Delta compression performance
• Serialization/deserialization speed
• Rate limiter throughput
• Field-level diff overhead

Debug Mode

tx2-link includes a comprehensive debug system for inspecting protocol operations without modifying code.

Environment Variables

Enable debug features using environment variables:

• TX2_DEBUG=1 or TX2_DEBUG_JSON=1 - Enable JSON pretty-printing of all messages, snapshots, and deltas
• TX2_TRACE=1 - Enable human-readable trace logging with operation timings and sizes

Both can be combined: TX2_DEBUG=1 TX2_TRACE=1

Debug Mode Features

When TX2_DEBUG=1 is set:

• All serialized/deserialized messages are logged as pretty-printed JSON
• World snapshots are logged with entity counts
• Deltas are logged showing all changes in readable JSON format

When TX2_TRACE=1 is set:

• Delta summaries showing entities added/removed/modified
• Serialization performance (format, size, duration)
• Delta compression ratios and timing
• Rate limiter decisions (allowed/blocked, current rate)
• Transport operations (bytes sent/received)

Usage

# Run with JSON debug logging
TX2_DEBUG=1 cargo run

# Run with human-readable traces
TX2_TRACE=1 cargo run

# Run with both
TX2_DEBUG=1 TX2_TRACE=1 cargo run

Example Output

With TX2_TRACE=1:

[TX2-LINK] Delta Summary:
  Timestamp: 2.0 (base: 1.0)
  Total changes: 5
  + 1 entities added
  ~ 2 components modified

[TX2-LINK] Delta compression: 1232875 bytes → 1052 bytes (1171.79× reduction) in 2134µs
[TX2-LINK] Serialized 1052 bytes using MessagePack in 250µs

With TX2_DEBUG=1:

[TX2-LINK] Serialized Message:
{
  "header": {
    "msg_type": "Delta",
    "sequence": 42,
    "timestamp": 1234567890
  },
  "payload": {
    "changes": [
      {
        "EntityAdded": { "entity_id": 123 }
      },
      {
        "ComponentAdded": {
          "entity_id": 123,
          "component_id": "Position",
          "data": { "x": 10.0, "y": 20.0 }
        }
      }
    ]
  }
}

Programmatic Access

You can also enable debug mode programmatically:

use tx2_link::init_debug_mode;

fn main() {
    // Reads TX2_DEBUG and TX2_TRACE environment variables
    init_debug_mode();

    // Your code here - debug logging happens automatically
}

Why Debug Mode?

Binary protocols like MessagePack and Bincode are efficient but opaque. Without debug mode, inspecting what's being sent over the wire requires:

• Packet capture tools
• Manual deserialization
• Custom logging code

With debug mode, you get instant visibility into:

• What data is changing between snapshots
• How effective delta compression is
• Serialization format efficiency
• Rate limiting behavior
• Network traffic patterns

This makes tx2-link extremely "vibe coding friendly" - you can see exactly what's happening without writing debugging code.

Development Status

• Core protocol messages
• Delta compression with field-level diffing
• Multi-format serialization (MessagePack, Bincode, JSON)
• Transport abstractions (WebSocket, IPC, stdio, memory)
• Rate limiting (token bucket, sliding window)
• Schema versioning and validation
• Streaming serializer/deserializer
• Comprehensive benchmarks
• Debug mode with JSON logging and traces
• Compression (zstd/lz4) for large snapshots
• Encryption support
• Reconnection handling with state recovery
• Binary diff algorithms for large binary components

Dependencies

• serde - Serialization framework
• rmp-serde - MessagePack format
• bincode - Bincode format
• serde_json - JSON format
• tokio - Async runtime
• bytes - Efficient byte buffers
• ahash - Fast hashing
• thiserror - Error handling

License

MIT

Contributing

Contributions are welcome! This is part of the broader TX-2 project for building isomorphic applications with a unified world model.

Learn More

• TX-2 Framework Outline
• tx2-core Native Engine
• tx2-pack Snapshot Format
• tx2-ecs TypeScript Runtime