Version v1.0.1

🚀 ThinTensor Release v1.0.1

📦 Optimized Architectures & Speedups

• Gemma-4-E2B: Achieved 43.60 tok/s (vs 1.62 tok/s HuggingFace baseline, 26.91x speedup).
• OLMoE-1B-7B: Achieved 33.70 tok/s (vs 9.22 tok/s HuggingFace baseline, 3.65x speedup).
• Both models achieve 0.999+ cosine similarity correctness.

💡 Documentation Note: Qwen-0.8B Sample Behavior

If you run the sample Qwen3.5-0.8B model, note that:

1. Document Completion: It is a raw base model (not chat/instruct), so it performs raw next-token document completion. It naturally prepends commas or continuations (e.g., Hello -> , I am working with...).
2. Greedy decoding repetitions: Under greedy decoding (required for correctness/parity checking), tiny models (0.8B) lack the stochastic sampling needed to break out of repetition loops.
3. For interactive conversational chat, download and convert fine-tuned instruct models (e.g. Qwen/Qwen2.5-3B-Instruct).

Version v1.0.0

ThinTensor Release v1.0.0

ThinTensor v0.2.0

ThinTensor v0.2.0 expands the native single-stream runtime and makes unsupported
Transformers causal LMs runnable through an explicit compatibility route.

Highlights:

• Native Gemma2 semantics: scaled embeddings, unit-offset RMSNorm, GeGLU,
  four-norm decoder blocks, alternating local/global attention, and attention
  and final-logit softcaps.
• Verified Gemma-2-2B-IT max path: 57.29 tok/s versus Transformers 53.58 tok/s,
  0.999555 minimum post-switch cosine, exact top-1 and ordered top-5.
• Verified Qwen2.5-3B path: 93.08 tok/s versus 48.18 tok/s at 500 tokens.
• Llama candidate path: 180.28 tok/s versus 132.36 tok/s on TinyLlama-1.1B;
  retained as candidate because a short BF16 cutoff tie misses strict top-5.
• Full causal BF16 KV retention in every published benchmark.
• Device-budgeted guarded FP8 vocabulary heads instead of a fixed
  vocabulary-size cutoff.
• Phi multi-EOS generation fix, including generation_config.json EOS IDs.
• Authenticated downloads through the official hf CLI without putting tokens
  in process arguments.
• Per-profile speed, peak-memory, cosine, top-k, and retention matrix in
  BENCHMARKS.md.
• Bundled thintensor-core archive binary in the Linux CPython 3.14 wheel.

The wheel SHA-256 is:

28f54aefff2b7e61d7c098cbc8e48011a6f9732a4714b45917266c1bb6ac0d5b

Performance is model-, context-, device-, driver-, clock-, and power-dependent.
max-performance remains opt-in and should be validated per model.