Skillware-Bench v0.3.3

Executive Summary

We are pleased to announce Skillware-Bench v0.3.3, bringing key empirical publication exhibits directly into the primary root documentation, standardizing repository citation infrastructure with the permanent Zenodo Concept DOI, refining ecosystem badge aesthetics to match our pastel design system, and deepening the foundational Skillware architecture specification.

v0.3.3 enhances repository visual transparency for visitors and researchers by embedding the critical empirical proof points: run-to-run determinism, cross-model pass rate uplift trajectories, granular per-call token ledger decompositions, and tool call survival flows.

---

What's New in v0.3.3

1. Key Publication Figures in Root Documentation (README.md)

Four high-impact 3x-Retina publication figures have been integrated directly into README.md under Key Empirical Findings with detailed context and empirical captions:

• Figure 14 — Run-to-Run Determinism & Variance Contraction:
  • Highlights variance contraction across 40 seeded replays under identical prompts for an 8B compact model.
  • Contrasts unassisted Vanilla execution scattering across an erratic score distribution (mean 61.0, $\sigma = 4.1$) against Skillware's 5.1× tighter dispersion (mean 88.2, $\sigma = 0.8$), proving enterprise release gating predictability.
• Figure 18 — Equip-Any-Model Pass Rate Uplift Trajectories:
  • Visualizes trajectory lifts across 14 representative models, demonstrating how compact sovereign models achieve capability parity with unassisted frontier LLMs (e.g. Meta Llama 4 Maverick jumping from 37.5% to 93.8% across 16 tasks).
• Figure 20 — Token Ledger Breakdown per Tool Call (Office Mail):
  • Decomposes token expenditure per tool call during binary attachment extraction, demonstrating how Skillware preserves core reasoning (1,340 tokens) while excising documentation down to typed schemas (−88%) and eliminating error repair loops entirely (−100%), reducing per-call token tax by 70.6% (1,620 vs 5,520 tokens).
• Figure 33 — 368-Layer Benchmark Call Survival Flow:
  • Details tool invocation survival normalized across 1,000 emitted calls per arm. Illustrates how Skillware's preflight typing delivers 94.1% clean first-pass calls (vs 71.2% in Vanilla) with only 7 terminal aborts (vs 57 in Vanilla).

2. Standardized Zenodo Concept DOI Badge & Citation Infrastructure

• Permanent Concept DOI: Minted and permanently linked the Zenodo Concept DOI 10.5281/zenodo.23725543, ensuring all future benchmark releases resolve to a persistent, citable scientific record.
• Shields.io Design Alignment: Standardized the Zenodo badge to match the repository's flat-square design system with #007ec6 blue styling (DOI | 10.5281/zenodo.23725543).
• Metadata Synchronization: Synchronized the permanent Concept DOI across README.md, CITATION.cff, and pyproject.toml ([project.urls].DOI).
• Complete BibTeX Citation: Added ready-to-copy BibTeX block in README.md citing v0.3.3.

3. Streamlined Ecosystem Navigation & Badges

• AURA Versioned Badge: Updated attestation badge to AURA | 0.3.6 in pastel peach (#ffdac1), positioned in second place immediately following Skillware | 0.6.0 (#D4E4F1).
• Cleaned Header Navigation: Streamlined the top navigation links row in README.md by removing external project links, keeping the table of contents focused on benchmark sections.

4. Grounded Skillware Framework Architecture

• 5-Layer Skill Anatomy: Updated the Skillware section in README.md to thoroughly explain Skillware's core philosophy (decoupling capability from intelligence) and five integrated bundle layers:
  1. Contract (manifest.yaml): Strict constitutional safety boundaries, dry-run guarantees, and typed I/O schemas.
  2. Effect (skill.py): Deterministic Python handlers executing verified actions directly.
  3. Directive (instructions.md): Structured cognitive guidance and system instructions adapted to host LLMs.
  4. Assurance (test_skill.py): Offline verification suites ensuring effects honor contracts before registry admission.
  5. Interface (card.json): Universal tool adapters standardized across Gemini, Claude, OpenAI, Ollama, and OpenAI-compatible runtimes.

---

Upgrade & Verification

Upgrade via pip:

pip install --upgrade skillware-bench==0.3.3

Verify package version and CLI environment:

skillware-bench --version
skillware-bench check-env

Cryptographically verify any model execution report without API keys:

skillware-bench verify benchmarks/reports/google_gemma_4_31b_it.json