Skip to content

potionodevil/Jitter

BranchesTags

Repository files navigation

Jitter — Adaptive Neural Runtime System

A sparse modular neural runtime that replaces monolithic LLM inference with Just-in-Time Weight Injection: only the neural clusters needed for the current context are mapped into memory, executed, and evicted.

Rust Edition License Tauri

Table of Contents

What is Jitter?

Traditional LLM inference loads the entire model (potentially several gigabytes) into memory before any computation can begin. For a 1.5B parameter model, this means ~3 GB of RAM occupied regardless of whether the user is asking a quick question or a complex coding task.

Jitter flips this model on its head:

  • The model is decomposed into functional units called Neural Clusters (e.g. reasoning block, syntax block, language block, expert block, shared components).
  • At runtime, only the clusters relevant to the current request are mapped into memory via memmap2.
  • Unused clusters never touch RAM; their weights remain on disk.
  • A persistent Experience Database learns which cluster paths work best for which types of requests, so future similar queries route instantly without analysis.

The result: dramatically lower memory footprint, faster cold-start, and a system that gets smarter the more you use it.

Core Concepts

Neural Cluster

A logical unit of weights that can be independently loaded, executed, and evicted. Each cluster contains metadata about its functional role (Reasoning, Language, Syntax, Shared, etc.).

Block Segmentation

The factory segments dense models into functional chunks of N layers (configurable, default 4). For MoE models, each expert is a cluster.

JIT Injection

When a request arrives, the WeightManager looks up which clusters are needed and mmaps them on demand. Cached clusters are returned without touching disk.

Experience Database

Every successful (or failed) inference is recorded in a sled-backed database. The router consults this database to find the best cluster path for similar future queries.

LRU Eviction

A thread-safe DashMap + LRU cache keeps frequently-used clusters resident. When memory pressure rises, least-recently-used clusters are evicted (their mmap handles dropped, OS frees the pages).

Embedding-Based Routing

A small embedding model (e.g. bge-micro, 384-D) converts the user's text into a vector. The router uses this vector to query the experience database for similar past queries and their optimal cluster paths.

Architecture Overview

+---------------------------------------------------------------------+
|                       Jitter Runtime                                |
+---------------------------------------------------------------------+
|  User Request                                                       |
|      |                                                              |
|      v                                                              |
|  +-----------------+  +-----------------+  +-----------------+    |
|  | Embedder        |->| RoutingController|->| WeightFactory   |    |
|  | (bge-micro)     |  | (sled + LSH)    |  | creates clusters|    |
|  +-----------------+  +-----------------+  +--------+--------+    |
|                                                       |             |
|                                                       v             |
|                                            +-----------------+     |
|                                            | WeightManager   |     |
|                                            | DashMap + LRU   |     |
|                                            +--------+--------+     |
|                                                     |              |
|                                                     v              |
|                                            +-----------------+     |
|                                            | memmap2 (zero-  |     |
|                                            | copy mapping)   |     |
|                                            +--------+--------+     |
|                                                     |              |
|                                                     v              |
|                                            +-----------------+     |
|                                            | candle Engine   |     |
|                                            | (forward pass)  |     |
|                                            +--------+--------+     |
|                                                     |              |
|                                                     v              |
|  +-----------------+  +-----------------+  +-----------------+    |
|  | ExperienceDB    |<-| Feedback Loop   |<-| InferenceResult |    |
|  | (persist path)  |  | (.report_*)     |  | (latency, etc.) |    |
|  +-----------------+  +-----------------+  +-----------------+    |
|                                                                     |
|  +--------------------------------------------------------------+  |
|  | Event Bus (tokio::sync::broadcast) ---> Tauri UI              |  |
|  +--------------------------------------------------------------+  |
+---------------------------------------------------------------------+

Features

Current (Sprint 1+2)

  • Factory-based cluster segmentation for dense and MoE models
  • .safetensors parser with shared/dense/expert cluster detection
  • sled-backed experience database with cosine-similarity routing
  • Embedding-based routing via bge-micro (local path or HuggingFace Hub fallback)
  • LRU cache with DashMap for thread-safe, lock-free parallel reads
  • memmap2 JIT injection with zero-copy candle tensor views
  • tokio broadcast event bus for monitoring
  • FeedbackReporter trait with report_success / report_error
  • Synthetic test helper to generate dummy .safetensors without 5 GB downloads
  • 4 integration tests + routing benchmark
  • CLI demo with live event tracing
  • Tauri 2.0 UI (Svelte 4 + TypeScript) — src-tauri/ + ui/
  • Real candle forward pass with plug-in architecture (LlamaArch for Llama/Qwen/Mistral)
  • SimHash-LSH routing for sub-millisecond lookups at scale
  • Cross-platform VRAM tracking (NVML behind nvml feature, plus Null fallback)
  • tracing + JSON observability pipeline
  • Curated model picker with automatic HuggingFace download (see below)
  • Rich cli_demo with --json, --batch, --stats, --model, --interactive
  • real_model_test example that downloads Qwen-2.5-1.5B automatically

Planned (next sprints)

  • KV-cache management across cluster boundaries
  • Full Qwen-2.5 forward pass with RoPE + GQA + SwiGLU
  • Streaming inference
  • Distributed experience sync

Project Status

Component Status Notes
Library core Done Compiles, all tests pass
Factory + indexer Done Heuristic + manual roles
ExperienceDB Done sled-backed with EMA scoring
Routing (cosine) Done Brute-force top-K
MmapCache + LRU Done DashMap + lru crate
WeightManager Done Thread-safe inject/release
Candle bridge Done Zero-copy tensor views
Embedder (bge) Done Local + HF fallback
Event bus Done tokio broadcast
CLI demo Done End-to-end working
Tauri UI Planned Next sprint
Real forward Planned Next sprint
LSH routing Planned Next sprint
VRAM tracking Planned Next sprint

Installation

Prerequisites

  • Rust 1.75+ (rustup install stable)
  • Tokio runtime (bundled)
  • Node.js 20+ and npm/pnpm (only for Tauri UI development)
  • WebView2 on Windows (preinstalled on Windows 10+)
  • Tauri CLI (only for UI): cargo install tauri-cli --version "^2.0"

Clone

git clone https://github.com/yourusername/jitter.git
cd jitter

Build the library

cargo build --release

Run tests

cargo test

Run the CLI demo

cargo run --example cli_demo

Quick Start

1. Use the library in your own Rust project

Add to your Cargo.toml:

[dependencies]
jitter = { path = "../jitter" }
tokio = { version = "1.40", features = ["full"] }

Minimal example:

use jitter::prelude::*;
use std::sync::Arc;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    // Configure
    let config = JitterConfig::default()
        .with_model_path("./model.safetensors")
        .with_db_path("./jitter_db");

    // Build the cluster index
    let strategy = BlockStrategy::default();
    let mut factory = WeightFactory::new(&config.model_path, strategy);
    factory.build_index()?;
    let index = Arc::new(factory.into_index());

    // Open the experience database
    let experience_db = Arc::new(SledExperienceDB::open(&config.db_path)?);

    // Create the weight manager
    let event_bus = EventBus::new(config.event_bus_capacity);
    let manager = Arc::new(WeightManager::new(
        config.model_path.clone(),
        index,
        config.max_cache_bytes,
        event_bus.clone(),
    ));

    // Set up the loader and pipeline
    let loader = Arc::new(ContextAwareLoader::new(manager));
    let router = Arc::new(RoutingController::new(
        experience_db.clone(),
        Arc::clone(&loader.manager().index()),
        config.routing_confidence_threshold,
        config.routing_top_k,
    ));

    // Use a dummy embedder (replace with BgeMicroEmbedder for production)
    let embedder = Arc::new(DummyEmbedder::new(config.embedding_dim));

    let pipeline = InferencePipeline::new(
        embedder,
        router,
        loader,
        experience_db as Arc<dyn FeedbackReporter>,
        event_bus,
    );

    // Run a query
    let result = pipeline.run("Explain Rust ownership").await?;
    println!("Loaded {} clusters in {} micros",
             result.loaded_clusters.len(),
             result.latency_us);

    Ok(())
}

2. Use a real embedding model

use jitter::embedder::BgeMicroEmbedder;

let embedder = Arc::new(
    BgeMicroEmbedder::load(
        Some("./models/bge-small-en-v1.5".into()),  // local
        "BAAI/bge-small-en-v1.5",                    // HF fallback
    ).await?
);

3. Manual cluster strategies

use jitter::factory::{BlockStrategy, RoleAssignment, ClusterKind};
use std::ops::Range;
use std::collections::HashMap;

let strategy = BlockStrategy {
    dense_chunk_size: 6,
    role_assignment: RoleAssignment::ByNamePattern(HashMap::from([
        ("code".to_string(), ClusterKind::DenseFunctional {
            layer_range: Range { start: 0, end: 12 },
            role: FunctionalRole::Syntax,
        }),
    ])),
};

Real-Model Test (auto-download)

The real_model_test example downloads a curated model from HuggingFace on first run, builds the cluster index, and runs a small set of inference requests.

# List available curated models
cargo run --example real_model_test -- --list

# Download and run the default (Qwen 2.5 1.5B Instruct, ~3 GB)
cargo run --example real_model_test --release

# Pick a specific model
cargo run --example real_model_test -- --model qwen2.5-0.5b-instruct

# Override the cache directory
cargo run --example real_model_test -- --cache-dir D:\models

# Provide a HuggingFace token for gated repos
cargo run --example real_model_test -- --model llama-3.2-1b-instruct --hf-token hf_xxx

# Add your own queries
cargo run --example real_model_test -- "Explain Rust lifetimes" "Write quicksort in Python"

Models are cached at ~/.cache/jitter/models/<slug>/. Subsequent runs skip the download.

Curated models

Slug Family Size Notes
qwen2.5-1.5b-instruct Qwen2 ~3 GB Default. Compact and capable.
qwen2.5-0.5b-instruct Qwen2 ~1 GB Low-RAM systems.
qwen2.5-3b-instruct Qwen2 ~6 GB Balanced accuracy/footprint.
llama-3.2-1b-instruct Llama ~2.5 GB Gated - HF token required.
phi-3-mini-4k-instruct Phi ~7.6 GB Excellent reasoning, large.
tinyllama-1.1b-chat Llama ~2.2 GB Compact Llama chat.

CLI Demo

The cli_demo example is a complete end-to-end demonstration with synthetic data:

cargo run --example cli_demo

Flags:

  • --json - emit events as JSON to stdout
  • --batch <file> - read queries from a file (one per line)
  • --model <path> - use an existing .safetensors model
  • --chunk-size <N> - override the dense chunk size
  • --stats - print RAM/VRAM stats after the run
  • --interactive - start a REPL after the demo

Output (abridged):

============================================================
  Jitter - Adaptive Neural Runtime (CLI Demo)
============================================================

[1/5] Generating synthetic model at "...\demo_model.safetensors"
[2/5] Building cluster index...
      Registered 3 clusters
        - shared (Shared, 516352 bytes)
        - dense_chunk_0 (DenseFunctional { layer_range: 0..4, role: FeatureExtraction }, 1048576 bytes)
        - dense_chunk_1 (DenseFunctional { layer_range: 4..8, role: Reasoning }, 1048576 bytes)
[3/5] Opening experience database...
[4/5] Initializing routing controller and weight manager...
[5/5] Running sample inference requests...

> Request: Explain Rust ownership
      [EVENT] RoutingDecision { ... source: Heuristic, confidence: 0.5 }
      [EVENT] ClusterInjected { source: Mmap, latency_us: 240, size_bytes: 516352 }
  Loaded 2 clusters in 1186 micros

> Request: Write a Python function
      [EVENT] RoutingDecision { ... source: Learned, confidence: 0.86 }
      [EVENT] ClusterInjected { source: Cache, latency_us: 0, ... }
  Loaded 2 clusters in 507 micros

Notice how the first request triggers Mmap loads (cold), while subsequent requests hit the Cache (warm).

Tauri UI (Dark-Mode Desktop App)

The Tauri 2.0 desktop UI lives under src-tauri/ (Rust backend) and ui/ (Svelte 4 + TypeScript + Vite frontend). It uses a dark theme by default and provides a Flight Recorder tab for live JSON tracing.

The build produces a real native Windows desktop application (.exe) — the Svelte UI is embedded as a static asset and rendered through WebView2. No web server is required at runtime.

One-time setup

# 1. Install Node dependencies for the Svelte UI
cd E:\Blackforest digital\Jitter
npm --prefix ui install

# 2. Install the Tauri CLI (once per Rust toolchain)
cargo install tauri-cli --version "^2.0" --locked

Run the UI in development mode

Use the wrapper script (handles the TAURI_FRONTEND_PATH env var automatically):

E:\Blackforest digital\Jitter\dev-desktop.bat

What happens on first run:

  • Vite dev server starts on http://localhost:1420
  • Rust backend compiles (5-10 minutes the first time; ~10s after)
  • A WebView2 window opens with the Jitter UI
  • Hot-reload is active for Svelte components

The app starts with a synthetic 6-layer model so the UI is interactive immediately — no download required.

Build a native Windows desktop installer

E:\Blackforest digital\Jitter\build-desktop.bat

This produces three artifacts under src-tauri\target\release\:

File Size Description
jitter-ui.exe ~7 MB Standalone desktop binary. Double-click to launch.
bundle\msi\Jitter_0.1.0_x64_en-US.msi ~3.4 MB Windows MSI installer
bundle\nsis\Jitter_0.1.0_x64-setup.exe ~2.5 MB NSIS setup wizard

All three are self-contained native applications that run without a web server, Node.js, or Rust toolchain installed on the target machine. The UI assets are baked into the binary at build time.

Switching to a real model (1.5B Qwen-2.5)

In the running app:

  1. Open the Model Picker tab.
  2. Click Download on qwen2.5-1.5b-instruct (~3 GB). The progress bar shows real-time byte counts.
  3. When the download finishes, click Activate. The WeightManager is hot-swapped via a RwLock<SharedWeightManager>. The active model label appears in the top bar.

The new manager is built from the real .safetensors file. The cluster index, route, and pipeline are all rebuilt atomically. The synthetic model in ~/.cache/jitter/models/synthetic/ remains untouched.

Why the wrapper scripts

Tauri 2.0's beforeDevCommand / beforeBuildCommand shells out to cmd /S /C. Because the project path contains a space (E:\Blackforest digital\Jitter\…), the resolved current_dir is unreliable. The wrapper scripts set TAURI_FRONTEND_PATH to the absolute path of the ui/ directory, which makes Tauri's dirs.frontend resolution work on every machine.

Tabs in the UI

Tab What it shows
Cluster Dashboard Live cards with heat-badges (gray = on disk, yellow = cached, green = locked). Evict / Lock / Unlock buttons per cluster. Evict uses Arc::strong_count to defer freeing if a caller still holds a reference.
Memory Monitor RAM, cache (mmap), and VRAM (when available) with live progress bars.
Routing Log Table of the last 200 routing decisions (Learned / Heuristic / Cold-Start, confidence).
Inference Console Textbox + "Run" button. Calls the loaded model's pipeline and shows latency.
Experience Table All learned cluster paths with success score, hits, average latency.
Model Picker 6 curated models with Download + Activate.
Flight Recorder Live JSON stream of every runtime event. Filter, scroll, or pause to inspect what your JIT routing is doing in real time.

What the backend does

  • src-tauri/src/state.rs keeps the active WeightManager behind Arc<RwLock<SharedWeightManager>> so the user can hot-swap between synthetic and real models.
  • src-tauri/src/commands.rs exposes 16 Tauri commands. Highlights:
    • list_clusters, get_memory_stats, run_inference
    • force_evict (real Arc::strong_count-based eviction with deferred drop)
    • lock_cluster, unlock_cluster
    • download_curated_model, load_real_model (hot-swap)
  • src-tauri/src/ipc.rs bridges the tokio::sync::broadcast event bus to the Tauri webview.

Documentation

Module overview

Module Purpose
cluster NeuralCluster, ClusterKind, ClusterMetadata, TensorDescriptor
factory WeightFactory parses .safetensors, builds cluster index, BlockStrategy
experience SledExperienceDB, ExperienceEntry, FeedbackReporter trait
routing RoutingController, QuerySignature, RoutingDecision
memory MmapCache, WeightManager, MappedTensor (zero-copy candle views)
candle_bridge Adapter from mmap'd tensors to candle (forward pass integration point)
embedder Embedder trait, BgeMicroEmbedder (384-D, local + HF fallback), DummyEmbedder
events EventBus, RuntimeEvent (broadcast for monitoring)
runtime ContextAwareLoader, InferencePipeline, end-to-end workflow
config JitterConfig with builder pattern
error JitterError, Result<T>

Data structures

ClusterMetadata

pub struct ClusterMetadata {
    pub id: ClusterId,                       // UUID
    pub name: String,                        // e.g. "dense_chunk_2"
    pub kind: ClusterKind,                   // Dense / Expert / Shared
    pub tensors: Vec<TensorDescriptor>,      // offset + len into safetensors
    pub total_size_bytes: u64,
    pub activation_signature: Option<Vec<f32>>,  // learned
}

ClusterKind

pub enum ClusterKind {
    DenseFunctional { layer_range: Range<usize>, role: FunctionalRole },
    Expert { layer_index: usize, expert_index: usize },
    Shared,
}

pub enum FunctionalRole {
    FeatureExtraction, Reasoning, Syntax, Language, Projection, Shared, Expert, Mixed,
}

ExperienceEntry

pub struct ExperienceEntry {
    pub signature: Vec<f32>,             // 384-D embedding
    pub cluster_path: Vec<ClusterId>,    // learned optimal path
    pub success_score: f32,              // EMA 0.0..1.0
    pub avg_latency_us: u64,
    pub hit_count: u64,
    pub last_used_ms: u64,
}

RuntimeEvent

pub enum RuntimeEvent {
    ClusterRegistered { ... },
    ClusterInjected { ... },
    ClusterEvicted { ... },
    RoutingDecision { ... },
    FeedbackRecorded { ... },
    MemoryPressure { ... },
    InferenceStep { ... },
}

Testing

Run all tests

cargo test

Run only library tests

cargo test --lib

Run only integration tests

cargo test --test integration_tests

Run tests with output

cargo test -- --nocapture

Current test coverage

  • test_factory_builds_index — verifies the indexer correctly identifies shared + dense clusters
  • test_weight_manager_inject_and_release — checks mmap inject/release roundtrip
  • test_experience_db_feedback — verifies feedback is recorded and retrievable
  • test_routing_controller — checks cold-start routing returns a valid decision

Benchmarking

cargo bench

This runs the routing_bench benchmark, which measures the latency of the routing controller across many iterations.

Adding new benchmarks: drop a new file in benches/ and add an entry to Cargo.toml [[bench]].

Project Structure

jitter/
|-- Cargo.toml
|-- Cargo.lock
|-- README.md
|-- .gitignore
|
|-- src/                          # Library source
|   |-- lib.rs                    # Library entry
|   |-- main.rs                   # Default binary (placeholder)
|   |-- config.rs                 # JitterConfig
|   |-- error.rs                  # JitterError, Result<T>
|   |
|   |-- cluster/                  # Data model
|   |   |-- mod.rs
|   |   |-- kinds.rs              # ClusterKind, FunctionalRole
|   |   |-- metadata.rs           # ClusterMetadata, TensorDescriptor
|   |   '-- cluster.rs            # NeuralCluster
|   |
|   |-- factory/                  # WeightFactory
|   |   |-- mod.rs
|   |   |-- factory.rs            # .safetensors parser + index builder
|   |   |-- strategy.rs           # BlockStrategy, RoleAssignment
|   |   '-- synthetic.rs          # generate_safetensors() test helper
|   |
|   |-- experience/               # Persistent learning
|   |   |-- mod.rs
|   |   |-- db.rs                 # SledExperienceDB
|   |   |-- entry.rs              # ExperienceEntry, apply_feedback()
|   |   '-- feedback.rs           # FeedbackReporter trait, FeedbackSignal
|   |
|   |-- routing/                  # Query -> ClusterPath
|   |   |-- mod.rs
|   |   |-- controller.rs         # RoutingController
|   |   |-- decision.rs           # RoutingDecision, RoutingSource
|   |   '-- signature.rs          # QuerySignature
|   |
|   |-- memory/                   # JIT injection
|   |   |-- mod.rs
|   |   |-- cache.rs              # MmapCache (DashMap + LRU)
|   |   |-- manager.rs            # WeightManager
|   |   '-- mmap_view.rs          # MappedTensor, mmap_cluster()
|   |
|   |-- candle_bridge/            # candle integration
|   |   |-- mod.rs
|   |   '-- adapter.rs            # CandleClusterView, CandleClusterAdapter
|   |
|   |-- embedder/                 # Text -> Vector
|   |   |-- mod.rs
|   |   |-- trait.rs              # Embedder trait
|   |   '-- bge_micro.rs          # BgeMicroEmbedder
|   |
|   |-- events/                   # Monitoring
|   |   |-- mod.rs
|   |   '-- bus.rs                # EventBus, RuntimeEvent
|   |
|   '-- runtime/                  # End-to-end
|       |-- mod.rs
|       |-- context.rs            # Request, InferenceResult
|       |-- loader.rs             # ContextAwareLoader
|       '-- pipeline.rs           # InferencePipeline
|
|-- tests/
|   '-- integration_tests.rs      # End-to-end tests
|
|-- examples/
|   '-- cli_demo.rs               # Full demo with synthetic model
|
|-- benches/
|   '-- routing_bench.rs          # Routing controller benchmark
|
|-- src-tauri/                    # Tauri 2.0 backend (Sprint 2)
|   '-- ...
|
'-- ui/                           # Svelte 5 frontend (Sprint 2)
    '-- ...

Configuration

Default JitterConfig

JitterConfig {
    model_path: "model.safetensors",
    db_path: "./jitter_db",
    embedder_path: None,                         // HF fallback if None
    embedder_repo: "BAAI/bge-small-en-v1.5",
    max_cache_bytes: 4 * 1024 * 1024 * 1024,     // 4 GB
    min_resident_clusters: 1,
    embedding_dim: 384,
    dense_chunk_size: 4,
    routing_top_k: 3,
    routing_confidence_threshold: 0.75,
    event_bus_capacity: 256,
}

Builder pattern

let config = JitterConfig::default()
    .with_model_path("./qwen2.5-1.5b.safetensors")
    .with_db_path("./runtime_db")
    .with_embedder_path("./models/bge-small-en-v1.5");

Architecture: Factory Pattern

Jitter is built strictly around the factory pattern:

WeightFactory

  • Reads .safetensors once at startup
  • Registers functional blocks as NeuralClusters
  • Returns a WeightFactoryIndex used by the rest of the system

ExperienceDB

  • Persists query -> cluster_path mappings
  • Updates success scores via exponential moving average
  • Implements the FeedbackReporter trait for type-safe access

RoutingController

  • The "learning" part
  • Consults ExperienceDB for similar past queries
  • Falls back to heuristics or cold-start
  • Decides which clusters to inject

This separation means each component can be tested in isolation, swapped independently, and reasoned about clearly.

How Learning Works

Step 1: Cold start

  • No experiences in sled.
  • The router uses a heuristic based on tensor names (e.g. reasoning layers + shared).
  • The user gets a result, and the system records the path with success_score = 0.5.

Step 2: Feedback

  • The user (or a downstream system) calls feedback.report_success(signature, path, latency_us).
  • The DB updates the entry's success_score (EMA: 0.7 * old + 0.3 * signal).
  • Average latency is updated similarly.

Step 3: Learned routing

  • On the next similar query, the router computes cosine similarity between the new embedding and stored ones.
  • If the best match exceeds confidence_threshold, the stored cluster_path is reused without any analysis.
  • This is the "fast path" — no model parsing, no heuristic evaluation.

Step 4: Continuous improvement

  • Every interaction refines the experience database.
  • The system gets faster and more accurate the more it's used.

Memory Management

Two-tier strategy

  1. Disk (.safetensors): The full model, always present. Weights are mmap'd, not loaded.
  2. RAM (cache): Only the currently-needed clusters. Bounded by max_cache_bytes.

Eviction policy

  • LRU eviction with shared-cluster protection.
  • When ref_count > 0 (cluster in active use), it cannot be evicted.
  • When memory pressure rises, the least-recently-used cluster with ref_count == 0 is evicted.
  • Eviction drops the Arc<Mmap>, which releases the OS-level mapping.

Thread safety

  • DashMap for lock-free parallel reads.
  • parking_lot::Mutex for the LRU list (fast, contention is minimal).
  • AtomicUsize for byte counts.
  • RefCount is atomic.

Roadmap

Sprint Focus Status
1 Library core, factory, experience DB, routing, mmap cache, candle bridge, embedder, CLI demo Done
2 Tauri 2.0 UI (Svelte + TS), real candle forward (Llama/Qwen/Phi plug-in) Planned
3 SimHash-LSH routing, cross-platform VRAM tracking, rich CLI modes Planned
4 KV-cache management across cluster boundaries, streaming inference Future
5 Distributed experience sync, multi-model registry, plugin marketplace Future

Contributing

Jitter is in active early development. Contributions are welcome but expect breaking changes until v0.2.0.

Development setup

git clone https://github.com/yourusername/jitter.git
cd jitter
cargo test                       # verify everything works
cargo run --example cli_demo     # smoke test
cargo bench                      # optional benchmarks

Coding conventions

  • Edition 2021
  • unsafe is allowed but must be documented with safety comments
  • All public APIs must be Send + Sync
  • Use parking_lot for new mutexes (not std::sync)
  • Use thiserror for error types, anyhow only in examples/
  • No unwrap() in library code outside tests
  • Format with cargo fmt and lint with cargo clippy

Pull request process

  1. Fork the repository
  2. Create a feature branch (git checkout -b feature/simhash-routing)
  3. Add tests for new functionality
  4. Ensure cargo test and cargo clippy pass
  5. Update the README and module docs as needed
  6. Open a PR with a clear description

License

This project is dual-licensed under either of:

at your option.

Acknowledgements

Jitter stands on the shoulders of giants:

Built with Rust. The model is on disk. Only the bits you need are in memory.

About

No description, website, or topics provided.

Resources

Readme
Activity

Stars

1 star

Watchers

0 watching

Forks

0 forks
Report repository

Releases

No releases published

Packages

 
 
 

Contributors

Languages