RISC Thought Engine: softmax fix + highheelbgz encoding proof

crates/thinking-engine/Dockerfile

@@ -0,0 +1,36 @@
+FROM rust:1.82-bookworm AS builder
+WORKDIR /app
+
+# Dependencies first (cached layer)
+COPY Cargo.toml Cargo.lock ./
+COPY crates/thinking-engine/Cargo.toml crates/thinking-engine/
+COPY crates/bgz-tensor/Cargo.toml crates/bgz-tensor/
+COPY crates/highheelbgz/Cargo.toml crates/highheelbgz/
+RUN mkdir -p crates/thinking-engine/src && echo "fn main() {}" > crates/thinking-engine/src/lib.rs \
+    && mkdir -p crates/bgz-tensor/src && echo "" > crates/bgz-tensor/src/lib.rs \
+    && mkdir -p crates/highheelbgz/src && echo "" > crates/highheelbgz/src/lib.rs \
+    && cargo build --release --manifest-path crates/thinking-engine/Cargo.toml 2>/dev/null || true
+
+# Source code
+COPY crates/ crates/
+
+# Download codebooks from GitHub Release
+ADD https://github.com/AdaWorldAPI/lance-graph/releases/download/v0.2.0-7lane-codebooks/qwen3-vl-embedding-7lane.tar.gz /tmp/
+ADD https://github.com/AdaWorldAPI/lance-graph/releases/download/v0.2.0-7lane-codebooks/jina-v5-7lane.tar.gz /tmp/
+ADD https://github.com/AdaWorldAPI/lance-graph/releases/download/v0.2.0-7lane-codebooks/jina-reranker-v3-BF16-7lane.tar.gz /tmp/
+RUN cd /app/crates/thinking-engine/data && \
+    tar xzf /tmp/qwen3-vl-embedding-7lane.tar.gz && \
+    tar xzf /tmp/jina-v5-7lane.tar.gz && \
+    tar xzf /tmp/jina-reranker-v3-BF16-7lane.tar.gz && \
+    rm /tmp/*.tar.gz
+
+# Build thinking engine
+RUN cargo build --release --manifest-path crates/thinking-engine/Cargo.toml
+
+# Runtime image
+FROM debian:bookworm-slim
+RUN apt-get update && apt-get install -y ca-certificates && rm -rf /var/lib/apt/lists/*
+COPY --from=builder /app/crates/thinking-engine/target/release/examples/playground /usr/local/bin/thinking-engine
+COPY --from=builder /app/crates/thinking-engine/data/*-7lane /app/data/
+EXPOSE 8080
+ENTRYPOINT ["thinking-engine"]

crates/thinking-engine/QUICKSTART.md

@@ -0,0 +1,138 @@
+# Thinking Engine — Quickstart (LM Studio-style)
+
+> Load a model. Type text. See thoughts. No GPU needed.
+
+## 1. Install
+
+```bash
+# Clone
+git clone https://github.com/AdaWorldAPI/lance-graph.git
+cd lance-graph
+
+# Download codebooks (770 KB per model, from GitHub Release)
+cd crates/thinking-engine/data
+curl -LO https://github.com/AdaWorldAPI/lance-graph/releases/download/v0.2.0-7lane-codebooks/qwen3-vl-embedding-7lane.tar.gz
+curl -LO https://github.com/AdaWorldAPI/lance-graph/releases/download/v0.2.0-7lane-codebooks/jina-v5-7lane.tar.gz
+curl -LO https://github.com/AdaWorldAPI/lance-graph/releases/download/v0.2.0-7lane-codebooks/jina-reranker-v3-BF16-7lane.tar.gz
+tar xzf qwen3-vl-embedding-7lane.tar.gz
+tar xzf jina-v5-7lane.tar.gz
+tar xzf jina-reranker-v3-BF16-7lane.tar.gz
+cd ../../..
+```
+
+## 2. Run (interactive, like LM Studio)
+
+```bash
+cargo run --release --manifest-path crates/thinking-engine/Cargo.toml --example playground
+```
+
+Type any text → see which codebook atoms light up, how energy flows through the distance table, which peaks emerge after 10 cycles of signed softmax thinking.
+
+## 3. Run benchmarks
+
+```bash
+# Does thinking beat plain cosine? (honest answer)
+cargo run --release --example benchmark_thinking \
+  --manifest-path crates/thinking-engine/Cargo.toml
+
+# 7-lane encoding (which formats preserve ground truth?)
+cargo run --release --features calibration --example seven_lane_encoder \
+  --manifest-path crates/thinking-engine/Cargo.toml -- qwen3-vl-embedding
+
+# Forward pass: real 2048D embeddings from Qwen3-VL
+cargo run --release --features calibration --example qwen3_vl_forward \
+  --manifest-path crates/thinking-engine/Cargo.toml
+```
+
+## 4. Docker
+
+```bash
+cd crates/thinking-engine
+docker build -t thinking-engine .
+docker run -it thinking-engine
+```
+
+## 5. Use as library
+
+```rust
+use thinking_engine::f32_engine::F32ThinkingEngine;
+
+// Load precomputed f32 cosine table (256 KB)
+let table: Vec<f32> = load_f32_table("data/qwen3-vl-embedding-7lane/cosine_matrix_256x256.f32");
+let mut engine = F32ThinkingEngine::new(table);
+
+// Perturb with codebook indices from tokenized input
+engine.perturb(&[42, 100, 150]);
+
+// Think: 10 cycles of signed softmax (T=0.1)
+let result = engine.think(10);
+
+// Get top-5 peaks
+let peaks = engine.top_k(5);
+for (atom, energy) in &peaks {
+    println!("Atom {} energy {:.4}", atom, energy);
+}
+```
+
+## 6. Online learning (table improves with use)
+
+```rust
+use thinking_engine::contrastive_learner::ContrastiveLearner;
+
+let table = load_f32_table("data/qwen3-vl-embedding-7lane/cosine_matrix_256x256.f32");
+let mut learner = ContrastiveLearner::new(table, 0.01);
+
+// Each forward pass teaches the table
+let real_cosine = forward_pass_cosine("text A", "text B");
+let centroid_a = codebook_lookup("text A");
+let centroid_b = codebook_lookup("text B");
+let error = learner.update_pair(centroid_a, centroid_b, real_cosine);
+println!("Table error: {:.4}", error);
+
+// After learning, build engine from improved table
+let engine = F32ThinkingEngine::new(learner.table().to_vec());
+```
+
+## Architecture
+
+```
+Text → Tokenize (Qwen3 BPE) → Codebook lookup (256 centroids)
+  → F32ThinkingEngine.perturb() → .think(10)
+  → Signed MatVec + Softmax(T=0.1) per cycle
+  → Top-K peaks = thought output
+
+The distance table IS the brain.
+Each MatVec cycle spreads energy through cosine similarity.
+Softmax concentrates on best matches (not ReLU which destroys information).
+10 cycles: 70-77% agreement with ground truth embedding cosine.
+```
+
+## Key Results
+
+| Metric | Value |
+|--------|-------|
+| Table format | f32 (Pearson r=0.9999 vs ground truth) |
+| Normalization | Softmax T=0.1 (not ReLU) |
+| Top-5 agreement | 70% (Qwen3-VL), 77% (Reranker) |
+| Entropy reduction | -21% to -31% (focuses, doesn't diffuse) |
+| Peak diversity | 100% (no attractor collapse) |
+| Speed | ~600μs/query (256 centroids, CPU) |
+| Table size | 256 KB (f32) or 64 KB (i8 signed) |
+| Models | 3 pretrained codebooks in GitHub Release |
+
+## Models Available
+
+| Model | Params | Dims | Cosine Range | Best For |
+|-------|--------|------|--------------|----------|
+| Qwen3-VL-Embedding-2B | 2B | 2048D | [-0.85, 0.54] | Multimodal (text+vision) |
+| Jina v5 | 0.6B | 1024D | [-0.19, 0.68] | Text embedding |
+| Jina Reranker v3 | 0.6B | 1024D | [-0.89, 0.83] | Cross-encoder (50% inhibition) |
+
+## What Failed (honest)
+
+- u8 CDF tables: destroy value geometry (Pearson 0.80)
+- γ+φ golden ratio: identical to CDF (zero added value)
+- ReLU normalization: attractor collapse (power iteration)
+- Multi-lens superposition: Cronbach α < 0.37 (models don't agree)
+- Gestalt awareness: zero improvement
+- Inhibition leak: zero improvement

crates/thinking-engine/examples/dual_signed_experiment.rs

@@ -28,7 +28,7 @@ fn run_lens_experiment(name: &str, table: &[u8], cos_range: &str) {
     println!("{}", stats.sign_stats());
     drop(stats);
 
-    let mut dual = DualEngine::from_unsigned_table(table.to_vec());
+    let mut dual = DualEngine::u8_vs_bf16(table.to_vec());
 
     // Test 1: Clustered input
     println!("\n--- Clustered (centroids 50, 52, 54) ---");

crates/thinking-engine/src/f32_engine.rs

@@ -99,12 +99,14 @@ impl F32ThinkingEngine {
         self.cycles = 0;
     }
 
-    /// ONE signed MatVec cycle:
-    ///   next[j] = sum_i distance_table[i][j] * energy[i]   (FULL signed, no floor, no threshold)
-    ///   normalize: total = sum|next[j]|, next /= total
-    ///   ReLU: next[j] = max(0, next[j])                    (can't have negative probability)
-    ///   re-normalize after ReLU
+    /// ONE signed MatVec cycle with softmax normalization:
+    ///   next[j] = sum_i distance_table[i][j] * energy[i]   (FULL signed)
+    ///   softmax: next[j] = exp(next[j] / T) / Σ exp(next[k] / T)
     ///   returns L1 delta for convergence check
+    ///
+    /// Softmax replaces ReLU — ReLU destroys inhibition information and causes
+    /// attractor collapse. Softmax preserves relative ordering while ensuring
+    /// positive probabilities. Low temperature (T=0.1) concentrates on best matches.
     fn cycle(&mut self) -> f32 {
         let k = self.size;
         let mut next = vec![0.0f32; k];
@@ -121,26 +123,16 @@ impl F32ThinkingEngine {
             }
         }
 
-        // Normalize by sum of absolute values
-        let abs_total: f32 = next.iter().map(|x| x.abs()).sum();
-        if abs_total > 1e-10 {
-            let inv = 1.0 / abs_total;
-            for e in &mut next {
-                *e *= inv;
-            }
-        }
-
-        // ReLU: can't have negative probability
+        // Softmax with temperature (default T=0.1 for sharp focus)
+        let inv_t = 10.0f32; // 1/T where T=0.1
+        let max_e = next.iter().cloned().fold(f32::NEG_INFINITY, f32::max);
+        let mut exp_sum = 0.0f32;
         for e in &mut next {
-            if *e < 0.0 {
-                *e = 0.0;
-            }
+            *e = ((*e - max_e) * inv_t).exp();
+            exp_sum += *e;
         }
-
-        // Re-normalize after ReLU
-        let total: f32 = next.iter().sum();
-        if total > 1e-10 {
-            let inv = 1.0 / total;
+        if exp_sum > 1e-10 {
+            let inv = 1.0 / exp_sum;
             for e in &mut next {
                 *e *= inv;
             }
@@ -154,8 +146,7 @@ impl F32ThinkingEngine {
         delta
     }
 
-    /// ONE cycle with temperature scaling.
-    /// Scales MatVec output by 1/T before normalization.
+    /// ONE cycle with explicit temperature parameter.
     fn cycle_with_temp(&mut self, temperature: f32) -> f32 {
         let k = self.size;
         let t = temperature.max(0.01);
@@ -174,31 +165,15 @@ impl F32ThinkingEngine {
             }
         }
 
-        // Scale by 1/T before normalization
+        // Softmax with explicit temperature
+        let max_e = next.iter().cloned().fold(f32::NEG_INFINITY, f32::max);
+        let mut exp_sum = 0.0f32;
         for e in &mut next {
-            *e *= inv_t;
+            *e = ((*e - max_e) * inv_t).exp();
+            exp_sum += *e;
         }
-
-        // Normalize by sum of absolute values
-        let abs_total: f32 = next.iter().map(|x| x.abs()).sum();
-        if abs_total > 1e-10 {
-            let inv = 1.0 / abs_total;
-            for e in &mut next {
-                *e *= inv;
-            }
-        }
-
-        // ReLU
-        for e in &mut next {
-            if *e < 0.0 {
-                *e = 0.0;
-            }
-        }
-
-        // Re-normalize after ReLU
-        let total: f32 = next.iter().sum();
-        if total > 1e-10 {
-            let inv = 1.0 / total;
+        if exp_sum > 1e-10 {
+            let inv = 1.0 / exp_sum;
             for e in &mut next {
                 *e *= inv;
             }