v1.4.0

1.4.0 (2026-10-03)

Features

• automate Siri route checks with an XCTest-only runner (8d1b0d8)
• brand: rename Foundation Evals to Intents across the app and release experience (27741da)
• compare stable Intent Lab requirements across app rebuilds (a760d1f)
• evals: run resumable batches and control evaluations through MCP (5008e16)
• guide connected app checks through the current Intent Lab interface (c28dd28)
• intent-lab: add reusable App Intents test integrations (35c647c)
• intent-lab: add scenario evaluation and project release reporting (9e2c318)
• intent-lab: guide app checks and show observed route outcomes (b4c0a7f)
• intent-lab: keep developer checks comparable across app fixes and routes (b69d7ce)
• intent-lab: retain regression collections and scoped reruns (b4c0a7f)
• intent-lab: verify saved evidence offline against trusted requirements (b69d7ce)
• review saved evaluation outputs, track failure patterns, and create verified regression cases (a4e593d)
• site: add an interactive showcase with recorded Intents app examples (2477e15)
• ui: redesign the workspace with native toolbar navigation and a unified visual style (5d2b98e)

Bug Fixes

• brand: restore the established Intents social preview layout (b60b097)
• evals: bind judge credentials and require coherent approved release evidence (5008e16)
• intent-lab: reject incomplete scenario release evidence (9e2c318)
• intent-lab: select Siri choices with symbol-prefixed app attribution (9e2c318)
• keep CLI credentials explicit for custom endpoints and reject redirects (301ac1a)
• preserve suite decisions and history across repository and recovery failures (301ac1a)
• reject DTD-bearing project previews before XML entity expansion (301ac1a)
• site: keep copy feedback with the current response (2477e15)
• site: stop active transitions when motion is paused (2477e15)
• ui: explain evaluation controls, comparison coverage, and workflow timing (3947e78)
• verify executed actions in intent lab results (890ab6b)
• verify native Mac bundle resources and product fingerprints consistently (301ac1a)

v1.3.0

1.3.0 (2026-09-19)

Features

• cases: add starter packs and structured case import (161d1fb)
• comparison: identify device and OS versions when comparing saved runs (a25beb4)
• evals: update evaluation workflow implementation (1071c70)
• evidence: preserve run provenance and explicit human approvals (161d1fb)
• experiments: compare controlled instruction variants (161d1fb)
• integration: evaluate registered Swift app features on paired Apple devices (a25beb4)
• integration: run repository and application checks through MCP and CLI (161d1fb)
• judging: configure independent judges and reassess saved responses (161d1fb)
• judging: harden compatible judge connections with attempt traces and extended timeouts (1071c70)
• releases: fail closed on stale or incomplete project evidence (161d1fb)
• runs: show scoring attribution and model identity in run history (1071c70)
• workspace: inspect suite health and recent evaluation runs in a project dashboard (a25beb4)
• workspace: organize evaluations into projects and saved suites (161d1fb)

Bug Fixes

• assessments: keep incomplete reassessments separate from original passing scores (6afcfd3)
• ci: keep main push checks from being cancelled (ab5be0f)
• editor: fallback on-device context size and harden case pickers (1071c70)
• editor: preserve prompt edits when keeping a stale rewrite (8dc3628)
• editor: simplify suite navigation and keep prompts with expected answers (a25beb4)
• evaluation: stop cancelled or unavailable judge work without duplicate requests (a106300)
• judge: reject failed completions and preserve authentic request evidence (c1e1d25)
• judge: tolerate malformed optional streaming usage without retrying valid verdicts (c1e1d25)
• judging: restore compatible JSON requests and actionable endpoint errors (a106300)
• mcp: preserve hidden suite policy and tokenizer state across automation updates (ab5be0f)
• models: reload Core AI resources when model files change (a106300)
• release: reject ambiguous or semantically inconsistent release overrides (f10abc3)
• release: require curated notes for squash-merged features and fixes (f10abc3)
• suite: keep git definitions portable and commit repository links atomically (ab5be0f)
• tests: align migration fixture timestamps with canonical JSON precision (750f56a)
• tests: emit correctly framed SSE events in the judge fixture (c1e1d25)
• tests: synchronize quick-action completion without polling deadlines (8dc3628)
• tools: bound workflow evidence for rejected shared reference-tool admission (bbbbcda)
• workspace: load replacement workspaces before atomically archiving the current selection (ab5be0f)
• workspace: preserve valid catalogs and fail closed across recovery mutations (ab5be0f)
• workspace: prevent legacy recovery from restoring revoked approval authority (750f56a)
• workspace: reject unsafe attachments and preserve unreadable evaluation state (a106300)

v1.2.0

1.2.0 (2026-09-08)

Features

• traces: add a native workflow waterfall and inspector (#19) (bf350d2)

v1.1.1

1.1.1 (2026-09-06)

Bug Fixes

• app: add signed updates and private launch statistics (#16) (2353a78)
• updates: enable automatic Sparkle updates (#17) (b0a5b70)

v1.1.0

1.1.0 (2026-09-06)

Features

• release: add manually controlled signed GitHub releases (#8) (843e3e1)

Bug Fixes

• docs: make the README artwork a full-width banner (#5) (cfff5cc)
• evals: address concurrency warnings and refresh lifecycle checks (#11) (4a86219)
• release: automate release PRs on main merges (#12) (a4c856b)
• release: publish releases only after verified DMG upload (#14) (014d9e5)
• strengthen CI and add repository branding (#3) (73f2d8c)
• ui: reshape the evaluation workbench around summary cards (#6) (d5469ea)

Foundation Evals 1.0.0

Foundation Evals is a native macOS workbench for testing Apple's Foundation Models.

Create repeatable evaluation suites, inspect responses and execution traces, score answers against explicit requirements, and compare saved runs. The built-in MCP server lets coding agents use the same evaluation workflow.

Install

Download Foundation-Evals-1.0.0-macOS-arm64.dmg, open it, and drag Foundation Evals onto Applications.

Requires Apple Silicon, macOS 27 or later, and Apple Intelligence enabled with its on-device model downloaded.

The app and installer are Developer ID signed. The installer is notarized by Apple and includes its notarization ticket. SHA256SUMS.txt contains the installer checksum.

This release is distributed through the private repository; downloads require repository access.

Built from commit 3ea6f1b9d6ccd0ff6c939f2dce074cbf3de818ad (version 1.0.0, build 2).