Skip to content

Visual Contract v0.1.0

Choose a tag to compare

View all tags
@aomizuki0307 aomizuki0307 released this 19 Jul 09:26
· 3 commits to main since this release
v0.1.0
33e2666

Visual Contract v0.1.0

Visual Contract turns a production brief into a typed visual specification and audits an
image against it. GPT-5.6 produces structured observations; deterministic Python code
owns the pass, fail, or uncertain verdict.

Highlights

  • Ten typed assertion families with Pydantic v2 validation and safe YAML editing
  • One-call GPT-5.6 Contract Compiler and one-call Vision Observer
  • Local size, aspect-ratio, and color-presence checks
  • JSON, JUnit XML, autoescaped HTML, and evidence-overlay reports
  • Server-rendered Brief -> Contract -> Audit -> Report Web workflow
  • Bounded sessions, CSRF protection, validated and re-encoded uploads, and safe downloads
  • Exact-input record/replay workflow for a complete demo without an API key
  • Published main, prompt-injection, live-stability, and post-freeze holdout results

Reproduce the demo

On Windows PowerShell:

git clone https://github.com/aomizuki0307/visual-contract.git
cd visual-contract
.\run_demo.ps1

The launcher removes any inherited OpenAI API key and runs the exact bundled replay. A
fixture miss is an explicit error and never falls back to a live request.

Published evaluation

  • Main objective assertions: 43/43 correct
  • Semantic assertions: 2/2, reported separately
  • Prompt injection: status-identical to its failing baseline
  • Live stability: zero wrong-verdict flips; two confidence-boundary uncertain results
  • Commit-derived post-freeze holdout: 12/12 objective and 1/1 semantic
  • Critical misses: zero in every published run

Core behavior is preserved at annotated tag build-week-feature-freeze (09c2fe9). The
holdout parameters were derived mechanically from that full commit SHA before the holdout
images existed. Raw normalized results and the methodology are committed in evals/.

Requirements

Python 3.11 is required. The Web replay demo is local-only and needs no OpenAI API key.
See the README for Linux/macOS setup, live-mode configuration, security boundaries, and
the full evaluation disclosure.

Assets 2
Loading