Agent Preflight v0.3.0 - Blind paired evaluation

Agent Preflight v0.3.0 upgrades the skill pack from self-reported routing checks to a reproducible paired, reviewer-blinded evaluation workflow.

Highlights

• Added paired control-versus-preflight trials with identical requests and opaque IDs.
• Added anonymous A/B review queues, separate blind keys, and SHA-256 artifact binding.
• Added automatic unblinding with paired score deltas, bootstrap 95% confidence intervals, two-sided sign tests, win rates, hard-failure comparison, and complete-pair token/latency/tool-call metrics.
• Expanded the behavior corpus from 39 to 60 adversarial cases: 32 English and 28 Chinese.
• Improved prompt-injection handling, contradiction detection, scoped approvals, local re-entry, and high-impact gate questioning.
• Reduced visible preflight ceremony for trivial and inspectable work.

Validation

• 6 skills pass the official skill-format validator.
• 60-case corpus validation passes.
• 26 unit and integration tests pass on Python 3.8 and 3.12.
• Full-corpus preparation produces 120 isolated trials across 60 pairs.
• PowerShell and Git Bash installation paths were verified.

Evidence note

The evaluation machinery is validated, but universal product-effect claims still require fresh, representative runs with at least 30 complete pairs and independent blinded review.

See the evaluation guide and changelog.

Agent Preflight v0.2.0

Agent Preflight v0.2.0 makes preflight behavior measurable and installation recoverable.

[0.2.0] - 2026-07-16

Added

• A versioned pack manifest shared by validation, installation, health checks, and eval tooling.
• A machine-readable forward-evaluation response protocol and deterministic 12-point scorer.
• Transactional cross-platform installation with dry-run, JSON output, backup, rollback, and post-copy verification.
• An installation doctor that reports missing, changed, and unexpected files.
• Execution contracts with autonomy lanes, checkpoints, layered verification, and local re-entry triggers.
• Uncertainty classification and contradiction scanning before user questions.
• Cross-platform unit tests and GitHub Actions CI.
• Python 3.8+ support for the dependency-free project utilities.
• Structured GitHub behavior reports and pull-request evidence checklists.

Changed

• Install wrappers now delegate to one tested Python implementation.
• Pack validation now checks manifest synchronization, response schema enums, Python syntax, orphaned references, and README version status.
• Forward-eval exports now require a strict machine-readable trace.

Full Changelog: v0.1.0...v0.2.0

Agent Preflight v0.1.0

Highlights

• Six composable skills for routing, intent clarification, capability gaps, tool discovery, tool review, and context budgeting
• Adaptive DIRECT / INSPECT / CLARIFY / DISCOVER / GATE modes
• Fast, balanced, deep, and critical preflight profiles
• Local-first capability discovery with explicit approval before installation, authentication, external data transfer, payment, deployment, deletion, or migration
• Context reduction and recoverable compression policies that preserve exact evidence
• 39 bilingual behavior cases covering trivial, complex, high-risk, tooling, and large-context tasks
• Windows and Unix installers with safe overwrite protection and backups

Install

git clone https://github.com/wde123sadw/agent-preflight.git
cd agent-preflight
./scripts/install.ps1

Validate

python scripts/validate_pack.py
python scripts/run_evals.py --check

This is the initial development release. Feedback and real-world agent traces are welcome.