Skip to content

MicroLLM v0.3.0 — Calibrated Multilingual System-1 Decision Engine

Choose a tag to compare

@jmrenouard jmrenouard released this 09 Nov 21:17
· 55 commits to main since this release

MicroLLM v0.3.0 — Calibrated Multilingual System-1 Decision Engine

We are thrilled to announce the production release of MicroLLM v0.3.0 (micro-jev), delivering a complete, high-performance, calibrated Multilingual System-1 Decision Engine across English (EN), French (FR), Spanish (ES), and Portuguese (PT).

MicroLLM v0.3.0 eliminates the heavy latency, memory, and operational costs of generative autoregressive LLMs for deterministic classification, workflow routing, and AI safety guardrails, executing certified probabilistic decisions locally on standard CPU hardware with sub-millisecond speed and mathematically bounded confidence.


🚀 Key Highlights & Verified Production Metrics

  • Dual-Gear Multilingual Architecture:
    • Strategy 1 (Fast Lightweight Engine / Gear 1): Pure Python stdlib execution (unicodedata, re, functools) with zero third-party runtime dependencies. Delivers sub-millisecond latency (p50 < 0.2 ms, SLA < 1.0 ms) with localized concept clusters, fast Unicode unaccenting (NFKD), rule-based light suffix stemming, fast language detection (detect_language_fast), and localized stopword/negation/connector tables across EN, FR, ES, and PT.
    • Strategy 2 (Neural Multilingual Models / Gear 2): Dual ONNX INT8 cross-encoder models—multilingual-MiniLMv2-L6-mnli-xnli (~48 MB INT8, < 8 ms) and mDeBERTa-v3-base-mnli-xnli (~134 MB INT8, < 20 ms). Dynamic label2id entailment index resolution (RCA-075) and localized source-language hypothesis templates eliminate cross-lingual transfer drift.
  • Unified Strategy Cascade & Serving:
    • Flexible strategy selection via multilingual_strategy ("fast", "neural", "auto") and neural_model ("minilm", "mdeberta"), fully accessible across FastAPI (POST /v1/systemone), Python SDK (MicroJevClient), CLI (microllm serve), and Model Context Protocol (microllm mcp).
  • Interactive Multilingual Decision Studio:
    • Full multilingual strategy selection and ticket evaluation presets (billing, shipping, technical support) in the Gradio web application (src/microllm/gradio_app.py), synchronized with the Hugging Face Space jmrenouard/microllm-decision-studio.
  • Comprehensive E2E Opaque-Box & Adversarial Test Validation:
    • 913 executable unit tests passed (100% pass rate), including 194 comprehensive opaque-box and adversarial tests (tests/e2e_multilingual/) verifying end-user contracts across all 4 languages.
    • Zero static typing errors under mypy src/microllm --ignore-missing-imports --check-untyped-defs.
    • 100% LaTeX math formatting compliance verified by pytest tests/test_docs_math_syntax.py.
  • Reproducible Multilingual Benchmarks:
    • Synthetic probe tickets across billing, shipping, and tech support in EN, FR, ES, and PT with isolated benchmark telemetry recorded in benchmarks/multilingual_benchmarks_results.json and documented in BENCHMARKS.md §11.
    • Hugging Face Hub dataset publishing utilities under jmrenouard/microllm-multilingual-benchmarks with --dry-run and live token resolution.
  • In-Depth Failure Modes & Architectural Critiques:
    • Comparative critique of lightweight vs neural strategies and detailed worst-case failure mode analysis documented in docs/FAILURE_MODES_AND_WORST_CASES.md and docs/MULTILINGUAL_STRATEGIES_SUMMARY.md.

🛠 Features & Capabilities

1. Strategy 1: Lightweight Multilingual Engine

  • Localized Lexicons: Package microllm.lexicons containing curated lexicons for en.py, fr.py, es.py, and pt.py.
  • Multilingual Normalizer: Fast unaccenting (strip_accents), rule-based stemming (light_stem), normalized tokenization (tokenize_multilingual_normalized), and O(1) heuristic language detection (detect_language_fast).
  • Concept Clusters: Expanded CONCEPT_CLUSTERS spanning 10 domain categories activating the first 16 slots of the 256-D embedding space.
  • Negation & Connectors: Multi-word negation phrases, double negations, and localized sentence connectors preventing false-positive sentiment and intent flips.

2. Strategy 2: Neural Multilingual Models

  • Dynamic Entailment Resolution: Resolves entailment label indices dynamically via ONNX config.json (label2id), resolving hardcoded index assumptions.
  • Native Hypothesis Templates: Localized templates in format_nli_hypothesis across EN ("It is {opt}: {desc}."), FR ("Il s'agit de {opt}: {desc}."), ES ("Se trata de {opt}: {desc}."), and PT ("Trata-se de {opt}: {desc}.").
  • Dual Profiles: Support for multilingual-minilm-int8 (ultra-fast INT8) and multilingual-mdeberta-int8 (high-capacity multilingual NLI).

3. Dual-Gear Cascade

  • Automatically commits Gear 1 when confidence exceeds the calibration threshold $\tau$.
  • Transparently escalates ambiguous tickets to Gear 2 neural cross-encoders with adaptive probability fusion:

$$ p_{\text{fused}} = (1 - \alpha) p_0 + \alpha p_1 $$

4. Interactive Studio & Cloud Sync

  • Interactive Gradio Decision Studio supporting live switching between Fast, MiniLM, mDeBERTa, and Auto cascade modes.
  • Seamless synchronization workflow to Hugging Face Spaces (jmrenouard/microllm-decision-studio).

📦 Getting Started

1. Installation

Install via pip:

pip install micro-jev

Or install in development mode from source:

git clone https://github.com/jmrenouard/MicroLLM.git
cd MicroLLM
pip install -e ".[dev]"

2. Python SDK Example

from microllm import MicroJevClient

client = MicroJevClient()

# Evaluate multilingual support state with Fast Strategy (Sub-millisecond)
response = client.choice(
    state="Client mécontent: la facture #9823 comporte un prélèvement en double.",
    question_id="ticket_category",
    question_text="Quelle est la catégorie de cette demande?",
    options=[
        ("billing", "Problème de facturation ou paiement"),
        ("shipping", "Expédition ou livraison de colis"),
        ("tech_support", "Assistance technique ou panne"),
    ],
    multilingual_strategy="fast",
)

print(f"Decision: {response.best_option}")
print(f"Confidence: {response.confidence:.4f}")
print(f"Abstain: {response.x_abstain}")

3. REST API & CLI Serving

Launch the local decision engine server:

microllm serve --host 0.0.0.0 --port 8000 --strategy auto --neural-model minilm

Send a request via cURL:

curl -X POST http://localhost:8000/v1/systemone \
  -H "Content-Type: application/json" \
  -d '{
    "model": "jev-latest",
    "state": "Mi pedido no ha llegado y el número de seguimiento no funciona.",
    "multilingual_strategy": "auto",
    "neural_model": "minilm",
    "questions": [
      {
        "id": "departamento",
        "type": "choice",
        "question": "¿A qué departamento corresponde esta consulta?",
        "options": [
          {"name": "facturacion", "description": "Consultas sobre facturas o pagos"},
          {"name": "envios", "description": "Seguimiento y entrega de paquetes"},
          {"name": "soporte", "description": "Problemas técnicos con el servicio"}
        ]
      }
    ]
  }'

📋 Full Milestone Summary (M1–M10)

Milestone Scope Key Deliverables PR
M1 Strategy 1: Lightweight Multilingual Engine src/microllm/lexicons/ (en, fr, es, pt), normalizer, concept clusters, negation PR #154
M2 Strategy 2: Neural Multilingual Models ONNX entailment fix, localized hypothesis templates, dual model profiles (minilm, mdeberta) PR #156
M3 Strategy Selection Options Runtime configuration in create_app, SystemOneRequest, MicroJevClient, and CLI PR #158
M4 Multilingual Benchmark Translation Translated probe tickets in microllm_multilingual_corpus.jsonl, 4-language runner PR #160
M5 Hugging Face Publishing Utilities Dataset upload (microllm-multilingual-benchmarks), model profiles, KeePass token resolution PR #162
M6 Isolated Benchmark Execution Automated benchmark runner, multilingual_benchmarks_results.json, BENCHMARKS.md §11 PR #164
M7 Gradio Application & HF Space Sync Strategy selection & multilingual ticket testing in gradio_app.py, Space sync automation PR #166
M8 Documentation & Critiques Comparative critique, worst-case analysis in docs/FAILURE_MODES_AND_WORST_CASES.md PR #168
M9 Full E2E Pass & Hardening 194 opaque-box and adversarial tests passed (100%), latency SLA hardening, typing fixes PR #170
M10 Production Release Publication v0.3.0 version bump across 8 files, release notes, changelog, git tag, and GitHub Release PR #228

What's Changed

  • feat(studio): upgrade Decision Studio usability across all 3 pillars with Gradio engine & mode switchers (#117) by @jmrenouard in #120
  • ci: use microllm-jev as PyPI fallback package name (#121) by @jmrenouard in #122
  • fix: import create_app in _HeadlessGradioComponent.launch (#123) by @jmrenouard in #124
  • feat(studio): add consumer-friendly per-type tabs alongside advanced studio & custom parameters (#125) by @jmrenouard in #126
  • feat: Add MCP tool safety annotations and M8ven Trust Index badge (#127) by @jmrenouard in #128
  • fix: set PyPI distribution package name to micro-jev in pyproject.toml (#130) by @jmrenouard in #131
  • docs: Update M8ven badge in README.md with verified publisher token (#132) by @jmrenouard in #133
  • docs: improve Markdown readability and visual spacing across GitHub and Hugging Face repositories (#129) by @jmrenouard in #134
  • feat: implement automated CI/CD README.md synchronization between GitHub and Hugging Face Hub (#135) by @jmrenouard in #136
  • fix(docs): isolate KaTeX display math delimiters in ALGORITHMS.md (#137) by @jmrenouard in #138
  • docs: publish bilingual MicroLLM launch articles on jmrenouard.fr (#139) by @jmrenouard in #140
  • chore(rules): enforce mandatory post-merge issue closure with explicit summary comment (#141) by @jmrenouard in #142
  • feat(multilingual): implement Strategy 1 lightweight engine for EN, FR, ES, PT (#143) by @jmrenouard in #144
  • fix(multilingual): resolve deep clause salience and negative concord in Strategy 1 (#145) by @jmrenouard in #146
  • fix(gradio): workspace mode switcher and enterprise preset interactivity (#147) by @jmrenouard in #148
  • fix(multilingual): extend salient clause extraction cap to resolve 7-clause ticket truncation (#149) by @jmrenouard in #150
  • feat: Strategy 2 Neural Multilingual Models & ONNX Entailment Fix (#151) by @jmrenouard in #152
  • fix: binary NLI entailment mapping resolution & session metadata loop scoping (#153) by @jmrenouard in #154
  • feat: multilingual strategy selection options (API, CLI & auto-cascade) (#155) by @jmrenouard in #156
  • fix(server): resolve cascade NameError in batch/mcp and create_app env defaulting (#157) by @jmrenouard in #158
  • feat: translate multilingual benchmark datasets and implement evaluation runner (#159) by @jmrenouard in #160
  • feat: publish multilingual benchmark datasets and neural models to Hugging Face Hub (#161) by @jmrenouard in #162
  • feat(benchmarks): execute multilingual performance suite and document benchmarks §11 (#163) by @jmrenouard in #164
  • feat(studio): add multilingual strategy selection and sync gradio space (#165) by @jmrenouard in #166
  • docs(multilingual): author comparative critiques and executive summary (#167) by @jmrenouard in #168
  • fix(tests): calibrate Strategy 1 local latency assertion with 1.0ms SLA (#169) by @jmrenouard in #170
  • refactor: resolve all 15 ponytail-review security, reliability, performance, and technical debt findings (#171) by @jmrenouard in #172
  • docs: add static-weight 300M classifier vs System-1 engine note and comparative matrix (#173) by @jmrenouard in #174
  • feat: register Tier-3 deberta-large-int8 profile, pareto benchmark, and agentic workflows (IDEA-048) (#175) by @jmrenouard in #176
  • fix(boundary): harden pareto benchmark empty sets, mcp compact primitives, and guardrail shell-pipes (#177) by @jmrenouard in #178
  • fix(docs): repair corrupted LaTeX math formulas and establish universal markdown math delimiter rules (#179) by @jmrenouard in #180
  • fix: repair latex math formulas, ban operatorname and enforce column-0 display math (#181) by @jmrenouard in #182
  • fix(docs): enforce display math blank line isolation and harden regex syntax assertions (#184) by @jmrenouard in #185
  • feat: multi-engine LLM benchmarks, BERT Large empirical comparison, and specialized linguistic models (#183) by @jmrenouard in #187
  • fix(docs): resolve unescaped currency math delimiter collisions and harden inline LaTeX verification (#186) by @jmrenouard in #188
  • fix(tests): exclude virtual environments and site-packages from repository markdown discovery (#189) by @jmrenouard in #190
  • fix(registry): register missing bert-large aliases and harden sequence scaling benchmark (#191) by @jmrenouard in #193
  • fix(docs): repair multi-label bare summation operators and harden math index assertions (#192) by @jmrenouard in #194
  • refactor: BENCHMARKS.md, benchmark automation and comparative evaluation (#195) by @jmrenouard in #196
  • docs: record PR #196 in ACTION_LOGBOOK (#197) by @jmrenouard in #198
  • fix(tests): stabilize Pareto unit test against CPU timing jitter (#199) by @jmrenouard in #200
  • fix(ci): guard Windows ctypes and Unix resource memory collection by platform in pareto_benchmark (#201) by @jmrenouard in #202
  • chore: merge trailing commit from issue-96 branch and prune stale merged branches (#203) by @jmrenouard in #204
  • fix(ci): make conda path resolution resilient against missing \ (#205) by @jmrenouard in #206
  • ci(conda): harden conda workflow with dispatch trigger, curl retries, and tests (#207) by @jmrenouard in #208
  • feat(core): remediate 12 ponytail audit findings (#209) by @jmrenouard in #210
  • fix(core): resolve CI and Unit Tests workflow failures in RLCD concurrency and VoC latency (#211) by @jmrenouard in #212
  • fix: fully vectorize RLCD Stage 2 and eliminate event loop probe latency (#213) by @jmrenouard in #214
  • fix(core): calibrate RLCD adaptive step sleep and pre-build middleware stack (#215) by @jmrenouard in #216
  • fix(core): vectorize semantic match logit and harden RLCD rollout boundaries (#217) by @jmrenouard in #218
  • fix(tests): add slack floor to micro-benchmark timing assertion in adversarial m3 stress (#219) by @jmrenouard in #220
  • fix(core): optimize uncached semantic matching and eliminate test sequence SLA jitter (#221) by @jmrenouard in #222
  • fix: optimize concept cluster prefix filtering and fast-path choice scoring (#223) by @jmrenouard in #224
  • fix: narrow options type in tournament fast path to satisfy mypy (#225) by @jmrenouard in #226
  • release: publish MicroLLM v0.3.0 production release (#227) by @jmrenouard in #228

Full Changelog: v0.1.0...v0.3.0

Full Changelog: v0.1.0...v0.3.0