Skip to content

Releases: ismaelsoilet/jev-harness

Release v0.2.0

Choose a tag to compare

@ismaelsoilet ismaelsoilet released this 23 Sep 16:44

Release v0.2.0 — System 1.5 Architecture, Provider Resilience & Tri-Runtime Parity

Data da pesquisa e calibração: 23 de setembro de 2026
Suíte de verificação: 569 testes verdes (393 Python + 89 TypeScript + 87 Rust) · 100% paridade tri-runtime


🌟 O que há de novo na v0.2.0

Esta versão consolida o jev-harness como a camada de decisão System 1.5 para o ciclo de qualidade de agentes autônomos de código, elevando robustez, rastreabilidade e calibração empírica.

🦀 Paridade Rust Live & Quebra Semver Controlada (0.x)

  • Correção no parser live de Score: suporte completo a floats (f64), mapas de legenda e probabilidades em respostas do provedor.
  • Assinatura pública ajustada: JevClient::retry_delay_ms(Option<f64>) (suporte a frações em Retry-After) e retorno de erro explícito em parse_api_response.

🔁 Resiliência de Provedor & Degradação Fail-Open

  • Política de retry com backoff exponencial suave e leitura rigorosa de headers Retry-After (429, 5xx e timeouts).
  • Política padrão fail-open com marcação clara da causa de degradação (degraded_reason: auth_401, http_429, http_500, timeout, connection, invalid_response).
  • Suporte a --fail-closed para pipelines rígidos e flag --retries N.
  • Respostas malformadas HTTP 200 tratadas como falhas de primeira classe sem traces não tratados.

🛡️ Limites de Payload, Redação de Segredos & Detecção de Injeção

  • Validação prévia de payload (128.000 code points para estado, 256.000 total) nos 3 runtimes.
  • Redação automática de credenciais e tokens sensíveis em todos os campos de saída e requisições de rede.
  • Detector determinístico de injeção em logs de teste não confiáveis, forçando escalonamento de julgamento (nunca skip_llm=true).

👻 Shadow Mode & Auditoria Completa

  • Flag --shadow (ou "shadow": true em .jev.json): avalia e reporta [SHADOW] would exit N sem bloquear pipelines.
  • Ferramenta de diagnóstico jev-harness doctor (inspeção de credenciais mascaradas, permissões de diretórios, hooks e limites).
  • Trilha de auditoria append-only com jev-harness receipts (permissões restritas 0600, TTL e hashes de integridade).
  • Separação entre métricas medidas em tempo real e estimativas heurísticas no jev-harness metrics.
  • Cache de decisão local baseado em hash de entrada com relatório de hit-rate (--no-cache).

📊 Calibração Empírica & Paridade 100% Travada

  • Corpus com 160 casos rotulados (tests/corpus) e comando replay integrado para cálculo de acurácia, macro-F1 e ECE.
  • Fixture de paridade tri-runtime (tests/fixtures/corpus_parity.json) garantindo decisões idênticas em Python, TypeScript e Rust.
  • Exemplos prontos de GitHub Action (examples/github-action/) para triagem de CI em runners Linux, macOS e Windows.

📦 Comandos de Instalação nos 4 Canais

1. Python (PyPI)

pip install --upgrade jev-harness

2. TypeScript / JavaScript (npm)

npm install -g @ismaelsoilet/jev-harness
# ou executar via npx
npx @ismaelsoilet/jev-harness --help

3. Rust (Crates.io)

cargo install jev-harness
# ou como dependência em Cargo.toml:
# jev-harness = "0.2.0"

4. GitHub Releases & Standalone Binaries

Baixe os binários e códigos fontes diretamente da release oficial no GitHub:
https://github.com/ismaelsoilet/jev-harness/releases/tag/v0.2.0

Full Changelog: v0.1.14...v0.2.0

Release v0.1.14

Choose a tag to compare

@github-actions github-actions released this 23 Sep 02:23

Full Changelog: v0.1.13...v0.1.14

Release v0.1.13

Choose a tag to compare

@github-actions github-actions released this 23 Sep 02:09

Full Changelog: v0.1.12...v0.1.13

v0.1.12 — Green runs never block, working pre-commit hook & provider access guide

Choose a tag to compare

@github-actions github-actions released this 23 Sep 01:38

⚡ v0.1.12 — Green runs never block, working pre-commit hook & provider access guide

Frontier model registry and provider access re-verified by web research on September 23, 2026 (TypeSafe AI console, OpenCode Zen, Command Code, OpenRouter alpha, Vercel AI Gateway).

✅ Green runs never block or escalate

  • A strict, deterministic success detector runs before any API call. Passing summaries from pytest, vitest, jest, cargo, go, mocha, rspec and unittest return category: "no_failure", skip_llm=true and exit 0 — with zero network calls.
  • Fixes false failures in pre-commit/husky recipes for JavaScript and Rust projects (e.g. vitest run | jev-harness test-gate previously exited 1 on a fully green suite).
  • Real failures always veto the shortcut: non-zero failed/failing/error counts (including comma, space, underscore and non-ASCII digit formats), failures:/failures= assignments, N test failed, FAIL/FAILED markers, go's --- FAIL:, tracebacks, panics, Error:, failure glyphs and dependency/transient signals.
  • no_failure is not counted in the ROI telemetry, so savings estimates stay honest.

🪝 Pre-commit integration that actually works

  • .pre-commit-hooks.yaml now uses a console entry point (jev-test-gate) that runs from any consumer directory; the previous entry: bash scripts/... was resolved in the consumer's working directory and failed with exit 127.
  • scripts/jev_test_gate_hook.sh remains available for shell and husky users.
  • jev-harness init --git (Python, TypeScript and Rust) detects your runner (npm / pytest / cargo / unittest), writes a failure-only hook using python3, marks it executable, recognises pre-v0.1.12 hooks and never overwrites a foreign hook (it writes pre-commit.jev instead).

🔐 Hardening

  • ~/.config/jev is created 0700; session.json and the lock file are written 0600, so telemetry error snippets are no longer world-readable.
  • TypeScript CLI reads stdin through an async iterator: fs.readFileSync(0) threw EAGAIN on streaming pipes (the documented npm test | ... test-gate recipe) for logs above ~64 KB.
  • TypeScript CLI prints the deterministic no-failure verdict (Rust and Python already did).

📖 Documentation

  • Provider access and API-key acquisition for every backend: TypeSafe (console.typesafe.ai), OpenCode Zen (opencode.ai/auth, jev-1.13-free), Command Code (signup + cmd login), OpenRouter (alpha, requires access), Vercel AI Gateway — plus a verification date and a mandatory 30-day re-verification rule for agents.
  • Explicit privacy matrix: offline mode makes zero network calls; live mode transmits the typed questions and the raw log (head 2,000 + tail 4,000 characters). Secret redaction applies to error messages, not to the log payload.
  • Corrected pre-commit, husky and CI recipes, documented no_failure behaviour and detector scope in README (EN/PT) and both integration guides.

🧪 Verification

207 tests, 100% passing — 113 Python, 49 Rust, 45 TypeScript. Zero cargo clippy warnings, actionlint clean, quad-manifest parity verified. Three independent adversarial review rounds found and we fixed a masking regression introduced by the first repair attempt.

📦 Install

pip install jev-harness==0.1.12                # Python (PyPI)
npm install @ismaelsoilet/jev-harness@0.1.12   # TypeScript / Node (npm)
cargo add jev-harness@0.1.12                   # Rust (crates.io)
git clone https://github.com/ismaelsoilet/jev-harness   # source

🇧🇷 Resumo

Execuções verdes nunca bloqueiam (detector determinístico com no_failure, zero chamadas de API), hook de pre-commit funcional com console entry point, init --git que detecta o runner, ficheiros de estado 0600/0700, e documentação completa de acesso a provedores com data de verificação e regra de re-checagem a cada 30 dias. 207 testes, 100% de aprovação.

Full Changelog: v0.1.11...v0.1.12

v0.1.11 — Tri-runtime heuristic parity, .jev.json precedence & release gate hardening

Choose a tag to compare

@github-actions github-actions released this 22 Sep 23:25

⚡ v0.1.11 — Tri-runtime heuristic parity, .jev.json precedence & release gate hardening

Frontier model registry re-verified by web research on September 22, 2026 (GPT-6 Astra, Claude Fable 5.1, DeepSeek V4.1-Flash/Pro, Qwen 3.8 Max, Gemini 3.8 Flash Thinking, Kimi-k3, MiMo-v2.6-pro).

🐛 Fixed

  • Tri-runtime classification parity (TypeScript) — an explicit assertion now scores identically in Python, TypeScript and Rust. The rules/04 precedence case (FAIL plus a cross-line Expected:/Received: pair containing a module name) is deep_logic / skip_llm=false on every runtime again.
  • Cross-line assertionsExpected:/Received: pairs spanning lines are detected through a bounded 300-character window with mandatory colons, so ordinary prose (expected response not received) can never suppress a transient classification.
  • Bare exceptions no longer mask root causesRuntimeError: ... Caused by: ModuleNotFoundError is env_missing, timeouts stay flaky_transient, and busy ports (already in use, Port 8080 is already in use, Failed to start server: ...) are retried deterministically.
  • FAIL/FAILED word boundariesfailing tests: and failsafe mode are no longer mistaken for assertion lines. Rust mirrors the Python/TypeScript lookahead with two anchored regexes, because the regex crate has no lookahead support.

⚙️ Configuration and resilience

  • .jev.json honored end-to-end in Python, TypeScript and Rust: model, skip_llm_threshold and abort_threshold, with clamping to [0, 1], corrupted-file tolerance, and the jev-latest scaffold placeholder treated as "use the provider default".
  • No more auth crashes in CI — HTTP 401/403 from any provider degrades to offline simulation with a stderr warning and is_mock=true; other errors still surface.
  • Honest ROI metrics — savings counters are labeled as heuristic estimates, the assumption model is printed, and --json exposes estimates_are_heuristic.
  • OpenRouter documented as alpha — it requires approved access; the endpoint and the typesafe/jev-1.13 model are not publicly listed.

🚦 Release engineering

  • release.yml now requires the full CI matrix (Linux/macOS/Windows × Python 3.9-3.13, Node 18-22, Rust) through a reusable-workflow gate before publishing to PyPI, npm or crates.io.
  • verify-sync now fails when user-agent or MCP server version literals drift from the manifest version.
  • Zero cargo clippy warnings; actionlint clean.

🧪 Verification

197 tests, 100% passing — 107 Python, 47 Rust, 43 TypeScript. Independently re-verified through adversarial review; two review findings were reproduced, fixed and re-checked.

📦 Install

pip install jev-harness==0.1.11                # Python (PyPI)
npm install @ismaelsoilet/jev-harness@0.1.11   # TypeScript / Node (npm)
cargo add jev-harness@0.1.11                   # Rust (crates.io)
git clone https://github.com/ismaelsoilet/jev-harness   # source

🇧🇷 Resumo

Paridade tri-runtime nas heurísticas, .jev.json honrado de ponta a ponta, fallback gracioso em falhas de autenticação (401/403), métricas de ROI rotuladas como estimativa e gate de CI completo antes de publicar. 197 testes, 100% de aprovação.

Full Changelog: v0.1.10...v0.1.11

Release v0.1.10

Choose a tag to compare

@ismaelsoilet ismaelsoilet released this 22 Sep 22:09

🌟 Jev Harness v0.1.10: Native Rust MCP Server, Concurrency Lock & Tri-Runtime Contract Parity

🚀 Highlights

  • 🛡️ Native Rust MCP Server (packages/rust/src/mcp.rs): High-performance JSON-RPC 2.0 stdio MCP server for the Rust crate (jev mcp / jev-harness mcp), providing full feature parity with Python and TypeScript across all 6 semantic decision gates.
  • Atomic Concurrency File Locking (fcntl.flock): Robust transactional file locking in session.py guaranteeing zero metric corruption and 0.0% counter data loss under heavy concurrent agent executions.
  • 🔄 OpenCode Zen Live Auth Fallback: Automatic graceful fallback to offline heuristic simulation on HTTP 401/403 when community dummy/zen keys are used, eliminating unhandled crashes in production.
  • 🛠️ CLI Subcommands Unification (init & metrics): Full cross-runtime availability of init (repo scaffolding & agent adapter generation) and metrics (session ROI and token telemetry) across Python, TypeScript, and Rust.
  • 📐 Rigid JSON Schema & Contract Parity: Ensured dual availability of action_recommendation + recommendation and reasoning_summary + summary across all CLI --json outputs and MCP tool invocations.
  • 🧪 149-Test Battery: 100% test pass rate across 149 tests (86 Python, 34 Rust, 29 TypeScript) with sub-100µs latency in Rust.

📦 Quad-Sync Registries

  1. GitHub: https://github.com/ismaelsoilet/jev-harness/releases/tag/v0.1.10
  2. PyPI: https://pypi.org/project/jev-harness/
  3. npm: https://www.npmjs.com/package/@ismaelsoilet/jev-harness
  4. Crates.io: https://crates.io/crates/jev-harness

Full Changelog: v0.1.9...v0.1.10

v0.1.9 — Command Code Free Provider & Continuation Nudge Gate (Jev Nudge)

Choose a tag to compare

@github-actions github-actions released this 22 Sep 20:58

Release v0.1.9: Native Command Code Free Provider (provider=commandcode, https://api.commandcode.ai/provider/v1/systemone, model typesafe/jev) and 6th Semantic Decision Gate (should_nudge_continuation / CLI nudge-gate / MCP tool jev_should_nudge_continuation) with gated workflow phases (research, ask, plan, execute, verify, complete) and calibrated Noul probabilities across Python (84 tests), TypeScript (29 tests), and Rust (32 tests) — 145/145 tests passing.

Release v0.1.8

Choose a tag to compare

@ismaelsoilet ismaelsoilet released this 22 Sep 19:54

🌟 Key Highlights in v0.1.8

  • 🏛️ Astra-Ares v0.2.1 Protocol Parity:

    • openrouter native endpoint: https://openrouter.ai/api/alpha/decisions (typesafe/jev-1.13, provider: { only: ["typesafe"], allow_fallbacks: false }).
    • vercel native endpoint: https://ai-gateway.vercel.sh/v1/evaluate (typesafe-ai/jev, providerOptions: { gateway: { only: ["typesafe-ai"] } }, supporting VERCEL_API_KEY, AI_GATEWAY_API_KEY, and VERCEL_AI_GATEWAY_API_KEY).
    • typesafe direct endpoint: https://api.typesafe.ai/v1/systemone (jev-latest).
    • opencode zen endpoint: https://opencode.ai/zen/v1/systemone (jev-1.13-free).
  • 8-Level Reasoning Scale & Provider Dialects:

    • Full support for none, minimal, low, medium, high, xhigh, max, ultra via supported_efforts / supportedEfforts.
    • Dialect compiling with clean CoT disable mapping (none / minimal) for DeepSeek (extra_body.thinking.type: disabled), Qwen (enable_thinking: false), Anthropic (thinking.type: disabled), Kimi (extra_body.thinking: false), and MiMo (thinking.type: disabled).
  • 🔄 Multi-Generation Effort Leasing:

    • Parallel evaluation of lease (1, 2, 5, 10 steps) respecting max_lease_steps with safe clamping (max_lease_steps >= 1).
    • Returns lease_steps = 5 for mechanical steps, lease_steps = 1 for error states/tracebacks, and lease_steps = 2 for standard tasks.
  • 🔒 Zero-Trust Secret Redaction:

    • Automatic masking ([REDACTED]) of Bearer ..., sk-..., vck_..., and active API keys in all HTTP error messages and diagnostics across Python, TypeScript, and Rust.
  • 🌐 Polyglot & Multilingual Semantic Parity:

    • Exact semantic categorization across 9 programming languages: Python, TypeScript/Node, Rust, Go, Java, C#, Ruby, C++ -> env_missing (skip_llm = true) vs deep_logic (skip_llm = false).
    • Multilingual evaluation parity across English, Portuguese (PT-BR), and Spanish (ES).
  • 🛡️ Red-Team Defense Hardening:

    • Collision resistance in verify_step_completion and should_abort_trajectory.
    • Prompt injection resistance in modulate_reasoning_effort.
  • 🔌 CLI & MCP Tooling Evolution:

    • CLI binaries across Python, TypeScript, and Rust accept --supported-efforts and --max-lease-steps.
    • Native MCP servers expose updated tool schemas and return lease_steps in JSON responses.
  • 🧪 Expanded 138-Test Battery:

    • 100% test pass rate across 138 tests (79 Python, 31 Rust, 28 TypeScript) with Rust latency p99 < 90µs.

Full Changelog: v0.1.7...v0.1.8

Release v0.1.7

Choose a tag to compare

@ismaelsoilet ismaelsoilet released this 22 Sep 06:50

⚡ Jev Harness v0.1.7 Release Notes

Zero-dependency System One decision harness, token optimizer, and semantic guardrails for AI coding agents.

🌟 Highlights & Key Enhancements

  • 📊 Empirical Tri-Runtime Latency Benchmarks (< 500µs Guarantee): Added comprehensive empirical latency benchmark tables ($p50$, $p95$, $p99 &lt; 500\mu\text{s}$) across Python, TypeScript, and Rust, verified with 1,000-iteration automated test suites.
  • Rust Engine Optimization: Cached static regexes with std::sync::LazyLock and optimized zero-allocation fallback client references in gates, dropping Rust triage $p99$ to 37.5µs and pure System One simulation to 1.3µs.
  • 🛠️ TypeScript CLI & Native MCP Full Parity: Hardened Unix semantic exit code handling (code 2 on missing command or invalid arguments) and added dual camelCase + snake_case JSON field support across CLI commands and native MCP tools (skip_llm, should_abort, provider_params, is_reasoning_supported).
  • 🚀 In-Memory Heuristic Acceleration: Added record_session: bool = False flag to decouple memory-only gate calls from filesystem I/O, achieving sub-150µs $p99$ latency in Python while preserving full telemetry on CLI executions.
  • 🧪 Complete 123-Test Battery: 100% test pass rate across 123 tests (73 Python, 26 Rust, 24 TypeScript) with zero compiler warnings.
  • 🔧 Release & Wiki Tooling: Enhanced scripts/release.sh to automatically synchronize TS MCP server version, AGENTS constitutions, and modular rules. Added scripts/push_wiki.sh and scripts/push_wiki.ps1.

📦 Official Registries & Artifacts

Full Changelog: v0.1.6...v0.1.7

v0.1.6: AGENTS Constitution, Modular Rules & 2026 Frontier Governance

Choose a tag to compare

@github-actions github-actions released this 22 Sep 04:51

🏛️ Jev Harness v0.1.6 — AGENTS Constitution, Modular Rules & 2026 Frontier Governance

📜 Master Architecture & AGENTS Constitution

  • AGENTS.md Root Entrypoint: Established the master constitution and architectural blueprint that all autonomous AI coding agents (OpenCode, Cursor, Claude Code, Antigravity IDE, Windsurf, Zed, Pi) must read before executing tasks.
  • Modular Engineering Rules (.agents/rules/):
    • 01_project_blueprint.md: Full architectural map, directory structure, Daniel Kahneman System 1 vs. System 2 cognitive duality, and tri-runtime parity.
    • 02_software_engineering_principles.md: Karpathy LLM coding principles, Fable method, Unix pipe philosophy, and anti-Frankenstein design (zero reverse-proxy instability).
    • 03_model_governance_and_frontier_registry.md: Active web research mandate, explicit research provenance dates, KV cache hysteresis, and direct model safeguards.
    • 04_testing_and_truthfulness.md: Zero-trust posture, 109-test distribution, and strict prohibition of paper-tiger assertions.
    • 05_release_and_quad_sync_protocol.md: 6-step release checklist, automated pre-push verification, and post-release monitoring across all 4 registries.
    • 06_code_style_and_conventions.md: Strict conventions for Python (zero runtime deps, stdlib only), TypeScript (pure ESM, zero deps), and Rust (Tokio 2021).

🌐 Strict 2026 Frontier Model Governance

  • Mandatory Web Research with Verified Date Stamps: All frontier model references are verified via active web research with formal provenance recording (Pesquisa realizada em 22 de setembro de 2026).
  • Anthropic Flagship Normalization:
    • Officially released active flagships: Claude Opus 5 (released July 24, 2026) and Claude Fable 5.1 (released September 1, 2026).
    • Unreleased version projections (e.g. 5.2 / 5.5) strictly prohibited from being documented as available.
  • 2026 Multi-Dialect Reasoning Compiler: Native payload generation for gpt-6-astra, o3-mini, deepseek-v4.1-flash, deepseek-v4-pro, qwen-3.8-max (2.4T MoE), gemini-3.8-flash-thinking, kimi-k3, and Xiaomi mimo-v2.6-pro.

🔒 Mandatory Pre-Push Parity & Registry Protection

  • Automated Pre-Push Hook (.git/hooks/pre-push): Strictly intercepts and blocks any git push if versions differ across pyproject.toml, package.json, Cargo.toml, and __init__.py.
  • CI Parity Verification (ci.yml): Continuous integration checks manifest parity on all PRs and pushes via ./scripts/release.sh --verify-sync.
  • Prohibition of Orphan Bumps: Eliminates version discrepancies between code manifests, git tags, and release notes.

📦 Quad-Registry Synchronized Availability

  • GitHub: Tagged v0.1.6 on main branch
  • PyPI (Python): pip install jev-harness==0.1.6
  • npm (TypeScript): npm install @ismaelsoilet/jev-harness@0.1.6 (Sigstore OIDC provenance)
  • Crates.io (Rust): cargo add jev-harness@0.1.6

🧪 Quality & Evidence

  • 109 Tests Passing (100% Pass Rate):
    • Python: 68 tests (python3 -m unittest discover tests -v)
    • Rust: 23 tests (cargo test --manifest-path packages/rust/Cargo.toml)
    • TypeScript: 18 tests (npm test --prefix packages/ts)