Skip to content

Qualify 8.1.3 with hardened release evals - #58

Merged
vriesd merged 23 commits into
mainfrom
codex/assurance-hardening-release-8.1.3
Aug 31, 2026
Merged

Qualify 8.1.3 with hardened release evals#58
vriesd merged 23 commits into
mainfrom
codex/assurance-hardening-release-8.1.3

Conversation

@vriesd

@vriesd vriesd commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Prepares 8.1.3 and hardens release qualification with repository-owned sampling, durable failure evidence, bounded environment reserves, prospective fail-fast stopping, and independent bundle regrading.
  • Binds explicitly named acceptance cases from the original OpenCode request to immutable plan evidence and the originating pending-plan host session.
  • Clarifies literal JUnit assertion names and keeps plan-only command timing out of immutable product scope.
  • Commits the fresh exact-artifact canary and content-addressed qualification bundle required by the tag workflow.

The public command and tool surface remains unchanged. Session v5 gains one optional backward-compatible request-evidence anchor.

Release qualification

  • Measured candidate: c6711c3
  • Evidence head: 29a7802
  • Campaign: 2026-08-30T20-37-00-610Z.v2
  • Matrix: 76/76 scored attempts passed across xai/grok-4.6 and openai/gpt-5.6-sol
  • Thresholds: every 100% stratum passed 3/3; both 90% strata passed 10/10 per provider
  • Reserves: one predeclared Grok failing-gate reserve passed after one retained host tool wedge; the original failure remains in the 77-attempt ledger
  • False completions: 0
  • Unsubmitted reviews: 0
  • Packed artifact: sha256:5dd2857fc10993ad649a37a5415f99f0eee00448d977987b0c2e0de3d2ec20e7

The exact-artifact Grok canary passed all six derived checks with one manager/reviewer lineage, named broad JUnit evidence, completed closure, and runtime identity bound to 8.1.3 and the measured plugin entry. Its record is sha256:72bb80a760a7c85532e0cc17277500538f574128340d5507662dfce1df3f44f2.

Qualification bundle qb1-8b6e7e3f935c26d42d2fa235dcd2b412c2b19d2e92f6980b2a9322e982e76492 independently regrades to VERIFIED. The committed evidence has zero raw session handles, encrypted reasoning metadata, project-linkage fields, credentials, real host paths, or symlinks.

Verification

  • bun run check: 659 passed, 1 intentional non-live skip
  • bun run replay: 13/13 cassettes reproduced
  • bun run smoke:live: 14/14 passed on OpenCode 1.18.6
  • strict exact-artifact canary: VERIFIED
  • strict qualification regrade: VERIFIED
  • rebuilt tarball is byte-identical to the campaign artifact
  • contribution preflight passed in commit and push modes

The architectural arena selected durable request anchoring and shared online/final release arithmetic. Multi-model interrogation found two valid issues, both fixed before qualification: active-plan collisions now fail closed, and hard product evidence remains distinct from pass-rate exhaustion.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ccc10c1ef6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread evals/harness.ts
@vriesd
vriesd merged commit c28a55c into main Aug 31, 2026
6 checks passed
@vriesd
vriesd deleted the codex/assurance-hardening-release-8.1.3 branch August 31, 2026 06:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants