Skip to content

Releases

v2.3.0 — Regime-Invariant 8D Code-Graph Sieve, Conformal ToolGate & Gemma 4 ADK Bundles

Choose a tag to compare

@napattan napattan released this 13 Nov 04:58

What's New in XLaya v2.3.0

Released alongside our Google Gemma 4 Developer Agent Competition Paper Track Writeup (Submitted!).

XLaya v2.3.0 Architecture Flowchart

Core Architecture Upgrades (--profile nano • Zero PyTorch / Zero NumPy)

  • 2-Tier Source-First Lexical Pre-Ranker & Compact AST Skeleton Headers: Partitions .py/.pyi repositories into Tier-1 core package files ahead of Tier-2 docs_src/, tests/, examples/, and scripts/ (0/129 core gold truncation on Kaggle 129), attaching compact enclosing class/import headers (100% split-method coverage).
  • 4-PC .npz Whitening & Isolated Call-Graph Channels (F-01..F-08 Regime Shield): Eliminates narrow-cone embedding anisotropy via All-but-the-Mean + Top-4 PC whitening (raw cosine median 0.8518 → -0.0158 across 127/127 Kaggle archives) and isolates 237,386 matched NetworkX node_link_data edges without inflating AST degree (0 degree cliff).
  • Smooth Channel Availability (M_avail, γ_max = 1.45): Preserves calibrated reliability across Regime R1 (with .json/.npz sidecars) and pure-Python stdlib Regime R2 (|ΔS| = 0.0041, |ΔTop-1 ECE| = 0.0073, "torch" not in sys.modules, 10.414 MB ΔRSS, 3.99 ms warm query / 255.33 ms/repo uncontended cold scan).
  • 2-Stage Causal Monotonic Tool-Schema Gate (triage_tool_call): Split-conformal abstention & tool routing in policies/sieve_genomes.json (5,679 B LF / 6,020 B CRLF), evolved strictly on 125 repo-disjoint @base_commit tasks (21 unseen repos, 1.28M CUDA evals in 74.87 s) with Slot-1 Platt calibration (T = 1.67, b = -0.795, conformal_q_hat = 0.5885).
  • Declarative Gemma 4 ADK Export (benchmarks/colab_kaggle_l4x4_runner.py --build-submissions): Generates compliant Container-A archives for both tracks (submission_31b_deep.zip 83,695 B and submission_e4b_sniper.zip 83,384 B, sys.modules["numpy"] = None & torch = None parity 0.0, 0 /workspace writes).

Zero-Shot Blind Verification Scorecard

Benchmark Metric v2.2.0 / Baseline XLaya v2.3.0
Kaggle 129 (R1 Sidecars) E4B R@3 / 31B R@8 / Top-1 ECE 31.01% / 35.66% / 0.0705 65.12% / 78.29% / 0.0568 (80.60% fastapi)
Kaggle 129 (R2 Pure Stdlib) E4B R@3 / 31B R@8 / Top-1 ECE — 62.79% / 78.29% / 0.0641 (98.49% Stage-A tok compression)
SWE-bench_Verified (n = 94) E4B R@3 / 31B R@8 / Top-1 ECE 24.47% / 26.60% / 0.2910 64.89% / 79.79% / 0.0155 (365.4 ms sympy)
nvidia/When2Call Val (n = 1,616) BinAcc (ToolRecall / Abstain) 62.75% (0% / 100%) 81.37% (81.40% / 81.36%, ECE = 0.0279)
nvidia/When2Call Test MCQ (n = 3,652) BinAcc (ToolRecall / Abstain) 64.54% (ECE = 0.3462) 71.69% (69.11% / 73.10%, ECE = 0.0777, 1.11 ms/row)
nvidia/When2Call LLM-Judge (n = 300) BinAcc (CannotAns / ReqInfo) 66.67% (ECE = 0.3187) 68.67% (79.00% / 59.00%, ECE = 0.1210)
Cross-Domain (xerj#1108 / PLE-29) Blind Acc / Wrong-Conf(p≥0.80) 58.62% nano (PLE) 100.0% (48/48) & 0.00% / 86.21% (25/29)

Packaging & Verification Checks

  • python -B skills/xlaya/scripts/laya_triage.py --self-test → 12/12 checks passed (version: "2.3.0")
  • python -B -m pytest -q tests/test_v23_sieve_and_adk.py → 5/5 passed
  • xlaya_mcp_server.py --self-check → {"ok": true, "server": "xlaya-mcp", "version": "2.3.0"}

v2.1.0 — Pure-Local Ensemble Jury, Multi-View Conformal Debiasing, 4-Facet Thorough Scan & Persistent TTT Policy

Choose a tag to compare

@napattan napattan released this 12 Nov 10:25

XLaya v2.1.0 — 100% Pure-Local High-Accuracy Edition

  • Stage 1 Dual-Encoder Ensemble Jury (--laya-model typed|ensemble): Blends 0.45 * laya-english + 0.55 * laya-typed-decisions with automatic disagreement >= 0.15 Stage 2 escalation and continuous model_agreement = round(1.0 - disagreement, 4).
  • Stage 2 Multi-View Position-Debiased Conformal Consensus (--vega-views 1..3): Evaluates up to 3 cyclic/reversed option permutations, un-permutes posteriors, and intersects/unions conformal prediction_set to eliminate option-order bias.
  • Persistent Zero-Backprop TTTHead & Split-Conformal Policy (xlaya_fit / --fit-csv / --policy-path): Fits and persists calibrated TTTHead + temperature T + per-class q_hat policies (~/.cache/xlaya/policies/<name>.json, 90.0% = 18/20 holdout accuracy in 0.79 s).
  • Decomposed 4-Facet Thorough Rubric & Document-Level Aggregation (--thorough): Scores chunks across relevance (0.35), technical_specificity (0.25), actionability (0.20), and ssot_consistency (0.20), plus document-level file_summaries.
  • Explicit Out-of-Distribution (outbound) Guard & Local Multimodal (image/audio/video): Surfaces normalized outbound + ood_score on every Stage 2 answer and supports 100% local image, audio, and video decision routing.
  • 6-Tool Resident JSON-RPC 2.0 stdio MCP Server (xlaya_mcp_server.py): Adds xlaya_fit alongside upgraded xlaya_triage, xlaya_predict, xlaya_status, xlaya_preload, and xlaya_unload.

v2.0.0 — 3-Stage System 1 to System 1.5 Semantic Decision & Conformal Triage Engine

Choose a tag to compare

@napattan napattan released this 11 Nov 08:28

XLaya v2.0.0 (iscvlab/xlaya)

  • 3-Stage System 1 -> System 1.5 Cascade (output_tokens: 0): Stage 0 pure-Python 4-backtick fence-safe multi-format chunker + BM25/code/Thai-3gram sieve (6,841.3 chunks/s, -99.35% Pass-B token compression) -> Stage 1 laya 0.4.31+ (ModernBERT-large & mmBERT + Head-Language Guard + 3-Question Joint NLI Contradiction Triad + predict_tournament on >20 labels) -> Stage 2 vegaml 0.16.1+ vega-U conformal verifier (up to 73,728 tokens).
  • Resident JSON-RPC 2.0 stdio MCP Server (xlaya_mcp_server.py): 29.99 ms warm single-state inference, 662.49 ms warm 32-chunk Triad (112.5x faster than v1.0 cold CLI), and LAYA_IDLE_TTL_SEC=1800 idle VRAM reclamation.
  • Empirical 4-Axis RTX 5060 Ti (16 GB VRAM) Benchmarks: 74.51 s -> 9.41 s (7.92x) cold CLI boot; 859.1 q/s (5.9x-6.5x) mmBERT & 837.9 q/s (8.9x) mixed EN/HI batch routing (0 / 432 flips); 60.0% -> 90.0% (18/20) 40-row fitted vega-U 0.8B routing accuracy in 7.58 s; 8 / 8 (100%) Joint NLI Triad precision; and 24.49% -> 0.00% (0/30 standalone, 0/36 ensemble) wrong-and-confident fires on xerj-org/xerj#1108.