Entroly 0.18.0: Building a 5,300-line Rust Engine for AI Context Optimization

[juyterman1000]

Entroly is an open-source context optimization engine for AI coding tools. The core is a single Rust crate (entroly-core, ~5,300 lines) exposed to Python via PyO3/maturin. v0.18.0 shipped this week with gzip-compressed index persistence, 415 Rust unit tests, and several architectural lessons worth sharing.

This post covers the Rust-specific implementation details: what worked, what surprised us, and the patterns that emerged from building a high-performance data plane that Python calls into via FFI.

This article was written with AI assistance.

---

Architecture: Why Rust for the Hot Path

Entroly's Python layer handles orchestration (MCP server, HTTP proxy, CLI). The Rust core owns everything latency-sensitive:

• Fragment scoring and selection (PRISM reinforcement learning)
• SimHash-based near-duplicate detection
• Entropy-gated semantic caching (EGSC)
• Submodular knapsack optimization with (1-1/e) approximation guarantee
• Index persistence (gzip serialization/deserialization)

The boundary is a single #[pyclass] struct with ~40 #[pymethods]. PyO3's GIL management is straightforward once you accept that every method returning a PyObject needs Python::with_gil.

#[pyclass]
pub struct ContextEngine {
    fragments: HashMap<String, ContextFragment>,
    fragment_slot_ids: Vec<String>,
    // PRISM RL weights — learned online via REINFORCE
    w_recency: f64,
    w_frequency: f64,
    w_semantic: f64,
    w_entropy: f64,
    // ... 30+ fields for caching, resonance, causal graph
}

Pattern 1: REINFORCE with EMA Baseline (PRISM)

The core optimization loop uses policy-gradient RL to learn which scoring weights produce useful context selections. The key insight: we don't need a neural network. Four scalar weights with REINFORCE + an EMA baseline work remarkably well.

pub fn record_success(&mut self, fragment_ids: Vec<String>) {
    self.n_feedback_events += 1;
    let raw_reward = if self.enable_channel_coding {
        let suff = self.last_optimization.as_ref()
            .map(|s| s.sufficiency)
            .unwrap_or(0.7);
        channel::modulated_reward(true, suff)
    } else {
        1.0
    };
    // A = R - baseline  (REINFORCE advantage with EMA baseline)
    let advantage = raw_reward - self.reward_baseline_ema;
    // baseline <- 0.9*baseline + 0.1*R  (~10-step memory horizon)
    self.reward_baseline_ema = 0.9 * self.reward_baseline_ema + 0.1 * raw_reward;
    self.apply_prism_rl_update(&fragment_ids, advantage);
}

The apply_prism_rl_update computes per-weight policy gradients and applies them with a temperature-scaled learning rate. A 5D variant adds a resonance dimension for pairwise fragment interaction learning.

What we learned: The EMA baseline is critical. Without it, gradient variance makes the weights oscillate wildly. With it, convergence happens in ~50 feedback events.

Pattern 2: SimHash for Near-Duplicate Detection

AI coding tools frequently re-ingest the same file with minor edits. We use SimHash (Charikar's locality-sensitive hash) to detect near-duplicates in O(1):

pub fn simhash(text: &str) -> u64 {
    let mut v = [0i32; 64];
    for token in text.split_whitespace() {
        let h = hash_token(token);
        for i in 0..64 {
            if (h >> i) & 1 == 1 { v[i] += 1; } else { v[i] -= 1; }
        }
    }
    let mut hash = 0u64;
    for i in 0..64 {
        if v[i] > 0 { hash |= 1 << i; }
    }
    hash
}

Two fragments are near-duplicates if their Hamming distance is <= 3 bits (out of 64). This catches copy-paste variations, reformatted code, and minor edits — without the overhead of embedding models.

What we learned: The 3-bit threshold was tuned empirically. At 2 bits, we missed too many real duplicates. At 4 bits, we started collapsing genuinely different files that happened to share common boilerplate.

Pattern 3: Gzip Persistence with Backward-Compat Magic-Byte Detection

v0.18.0 fixed a latent bug: persist_index was writing plain JSON to a file named index.json.gz. The fix uses flate2::write::GzEncoder for actual compression, and load_index detects the format by inspecting the first two bytes:

pub fn persist_index(&self, path: &str) -> PyResult<()> {
    use flate2::Compression;
    use flate2::write::GzEncoder;
    let tmp = format!("{}.tmp", path);
    let f = std::fs::File::create(&tmp)?;
    // Owner-only permissions before writing data
    #[cfg(unix)]
    {
        use std::os::unix::fs::PermissionsExt;
        f.set_permissions(std::fs::Permissions::from_mode(0o600))?;
    }
    let mut gz = GzEncoder::new(f, Compression::default());
    gz.write_all(&json)?;
    gz.finish()?;
    std::fs::rename(&tmp, path)?; // atomic swap
    Ok(())
}

pub fn load_index(&mut self, path: &str) -> PyResult<usize> {
    let raw = std::fs::read(path)?;
    // Magic bytes: 0x1f 0x8b = gzip, 0x7b = '{' = legacy plain JSON
    let decoded: Vec<u8> = if raw.len() >= 2 && raw[0] == 0x1f && raw[1] == 0x8b {
        let mut gz = GzDecoder::new(&raw[..]);
        let mut out = Vec::with_capacity(raw.len() * 4);
        gz.read_to_end(&mut out)?;
        out
    } else {
        raw  // legacy format — plain JSON in a .gz file
    };
    let snapshot: IndexSnapshot = serde_json::from_slice(&decoded)?;
    // ... restore state
}

What we learned: The #[cfg(unix)] guard for set_permissions(0o600) is essential — the index file contains ingested source code and must not be world-readable. On Windows, this is a no-op. The atomic temp-file-then-rename pattern prevents corruption if the process is killed mid-write.

Pattern 4: Entropy-Gated Semantic Caching (EGSC)

The cache uses Thompson Sampling for admission control — a technique borrowed from multi-armed bandits. Each cache entry has a Beta distribution quality model. Admission decisions sample from this distribution rather than using a fixed threshold:

pub fn should_admit(&mut self, query: &str, tokens: u32) -> bool {
    let sample = self.thompson_sample(); // Beta(a, b)
    let freq_score = self.frequency_sketch.estimate(query);
    let cost_benefit = tokens as f64 * self.cost_per_token;
    sample * freq_score * cost_benefit > self.entropy_threshold
}

The frequency sketch is a Count-Min Sketch (4 rows x 256 columns) that estimates query popularity without storing the queries themselves. Cache invalidation uses dual signals: content hash (MD5 of the fragment body) and a monotonic generation counter.

Pattern 5: Submodular Knapsack with Greedy (1-1/e) Guarantee

Context selection is a submodular maximization problem under a cardinality (token budget) constraint. We use the standard greedy algorithm with lazy evaluations:

// Greedy submodular selection: pick the fragment with highest
// marginal gain per token cost, repeat until budget exhausted.
while remaining_budget > 0 && !candidates.is_empty() {
    candidates.sort_unstable_by(|a, b|
        (b.marginal_gain / b.token_cost as f64)
            .partial_cmp(&(a.marginal_gain / a.token_cost as f64))
            .unwrap_or(std::cmp::Ordering::Equal)
    );
    let best = candidates.remove(0);
    if best.token_cost <= remaining_budget {
        selected.push(best);
        remaining_budget -= best.token_cost;
    }
}

The marginal gain function combines four PRISM-weighted signals (recency, frequency, semantic relevance, entropy) plus optional resonance and causal bonuses.

Testing: 415 Rust Tests

The test suite covers:

• Core engine: Fragment CRUD, scoring, optimization, checkpoint roundtrips
• EGSC cache: Baseline gauntlet, distribution shift torture, mutation stress, DAG-aware eviction, cost-aware utility, multi-workload hit rate, throughput benchmarks, ablation study
• SimHash dedup: Exact match, near-duplicate detection, false positive rate
• Channel coding: Reward modulation, sufficiency scoring
• Causal graph: Trace recording, temporal eviction under capacity
• Utilization scoring: Trigram + identifier overlap measurement

All 415 tests pass in 1.6 seconds on a single core.

Numbers

On Entroly's own codebase (433 files, 1M tokens):

Metric	Result
Indexing speed	0.66s
Token savings (32K budget)	97.0%
Token savings (8K budget)	99.2%
Optimization latency	22ms
Rust test suite	415/415 pass
Python test suite	965/965 pass

Links

• GitHub: github.com/juyterman1000/entroly
• PyPI: pip install entroly
• License: Apache-2.0