Specification Generation: Internal Evidence

Three public document inputs and 27 assigned calls. Collection stopped: 14 complete responses, 5 incomplete or unavailable, 8 not dispatched. Reference decisions are assistant drafts. Not independent validation, not a qualified ranking. Full offline replay verified on Linux and macOS; Windows replay remains unsupported.

PALO BlueprintBench 0.3.0: Specification Fidelity

Known evaluator flaw: EV-01

This historical release contains evaluator 0.2.0. A substantive incorrect assertion can be hidden in an excluded non-claim span. Do not use its acceptance outputs for qualification.

Use internal corrective release v0.3.0-internal.1 for evaluator 0.3.0 and migration guidance. The corrective release is internal software evidence, not independent semantic validation. The original tag and ZIP below remain unchanged for audit reproduction. An explicit warning is attached separately.

Original historical release notes

PALO BlueprintBench opens a public research programme for specification fidelity: whether obligations, permissions, ownership and exceptions survive the translation from business documents to a development specification.

This release corrects the evaluator's critical-error counting and task acceptance, introduces an executable evidence schema with raw-output binding, and publishes the retained development harness, requests and responses. The explorer includes filters, comparison charts and complete configuration disclosures in CSV exports.

Validation: 42 evaluator tests, a packaged synthetic demonstration, generated-artifact parity and 26/26 offline structural-count reproductions pass in GitHub Actions on Python 3.10 and 3.13. Verification run.

Qualified held-out results: 0. No PALO Blueprint Index score is assigned. Historical diagnostics use differing configurations and are not a ranking or semantic-accuracy result. Local integrity locks do not establish external preregistration. This release does not qualify or release a new Workbench app.

The first planned independent study focuses on procurement. Participate, read the verification and boundaries, or use the prepared announcement.

Components: explorer 0.3.0; evaluator/evidence contract 0.2.0; vocabulary/taxonomy 0.1.1; ontology 0.1.0. The attached manifest records source and package SHA-256 hashes. PALO remains free.

BlueprintBench internal release 0.3.0

Internal engineering prerelease for BlueprintBench, publication revision 2026-11-11.3, evaluator/evidence contract 0.3.0.

The results dashboard now reports descriptive observations without numerical model ranks. Filters, charts, comparison and CSV/JSON exports remain. Conditional label agreement, coverage and fixed-denominator matches are distinct. Frozen model evidence revision 2026-11-11.2 is unchanged; its hash is recorded in release.json.

Evaluator 0.3.0 closes the EV-01 non-claim exclusion flaw and includes executable contract, configuration, timing and annotation checks. The ZIP embeds release identity and migration guidance. Historical evaluator 0.2 remains available for reproducibility and must not be used for qualification; see the attached warning. Historical packages were not silently replaced.

Validation: 76 repository evaluator tests; 73 standalone extracted-package tests (three adapter tests require the separate evidence archive); 65 tool tests including 14 next-cycle tests; 26/26 offline historical replay; generated-artifact and multi-platform CI checks. Browser checks cover descriptive sorting, exports, missing denominators, charts and separate reviewer diagnostics.

Six development scenarios, 48 draft probes, six workflow tasks and blinded annotation templates prepare the next measurement cycle. They are exposed assistant drafts, not independent gold or statistically sufficient evidence. No new inference spending occurred.

Trust boundary: no independent human review, external custody, independent semantic validation or PALO Blueprint Index is asserted. Automated cross-review does not replace human adjudication. The annotated release tag is not cryptographically signed; file hashes establish byte identity only. This release does not qualify or release a new Workbench application.

Internal results · Reviewer diagnostics · Next-cycle plan · Method

BlueprintBench v0.2.0-pilot.1: evidence explorer and PBI proposal

BlueprintBench is a public research pilot for evaluating document-to-blueprint workflows. This release establishes the reproducible research repository and proposes the PALO Blueprint Index (PBI). No PBI score or independently validated leaderboard is released.

Included:

• Evaluator, protocol, candidate source registry and 17 passing tests.
• Diagnostic explorer for 10 model configurations with charts, filters, comparisons and CSV export.
• Explicit failure, cost and missing-evidence accounting, with a dated reasoning-metadata correction.
• Governance, contribution, versioning and qualification policies.
• Real Workbench/LM Studio screenshots and source for a 15-slide walkthrough.

Validation: CI passes on Python 3.10 and 3.13; Schema.org reports zero errors/warnings; Google validates one breadcrumb item. These are technical checks, not benchmark qualification.

The three large development repositories remain development evidence. There are no independently qualified held-out entries and no newly qualified Workbench app release in this release.

Live explorer: https://paloframework.org/PALO_BlueprintBench.html

Process Interpretation: Internal Evidence

Internal engineering feasibility on six synthetic procurement scenarios, 48 provisional propositions, six workflow tasks, seven metamorphic tasks and advisory reviews of P002/P013/P036.

All 60 assigned requests completed. Seventeen of eighteen workflow drafts passed structural checks; the rejected draft and all disagreements remain in the evidence. Provider-reported charge: USD 2.5539187. Conservative cumulative commitment: USD 21.581286351 within the authorised USD 40.

These observations do not establish independent semantic accuracy. The release retains provisional-label ambiguities and the threshold experiment's eight-to-six proposition confound. Original labels, denominators and the evaluator v0.3.0-internal.1 package remain unchanged. This is a postcollection evidence release, not an independently witnessed preregistration.

Live interactive results · Detailed results and reproduction

Publication verification includes exact HTTPS hashes for all 11 deployed files, merged-tree checks, live browser checks and screenshots. The evidence archive contains the original requests, responses, receipts, source packets and unapproved blueprint exports.

CI for the released source passed on Linux, macOS and Windows: source verification. The platform build and deployment also passed.

Presentation update 2026-11-11.5

The live overview, dashboard and six-scenario page now share source-generated Dataset metadata, an explicit link to this immutable evidence release, and the planned qualification protocol. Protocol and semantic rubric at the publication commit. All independent acceptance gates remain open.

This is a presentation and method update, not a new scored campaign or independent validation. The four assets attached to this release, their source commit, the frozen observation revision 2026-11-11.4 and evaluator package retain their original bytes. The attached publication manifest and verification archive describe the original release; the live manifests additionally identify the current presentation revision. No new inference calls or API charges were incurred for this update.

Verification: 84 publication/evidence tests, deterministic generators, four-page desktop/mobile checks, and exact HTTPS byte verification of nine updated site files. The merged source trees match the tested pull-request heads: benchmark CI, platform CI.