v1.18.0 — evidence strength, pinned guideline citations, SKIP class, guideline 4.3

[1.18.0] - 2026-08-31

Two gaps closed, both about not overstating what the tool knows. Prior art credit: the
validator-vs-reviewer confidence split and the never-quote-from-memory rule are adapted from
dabodamjan/app-store-rejection-checker
(MIT), which applies them to an LLM auditor; this is the deterministic-scanner form of both.

Added

• Evidence strength and confidence on every finding
  (scripts/evidence.sh). Severity says how bad a
  finding is; it never said how firmly it was established. Apple's upload validators run against
  the built product while this scanner reads a repository, so a missing purpose string found by
  grepping .swift and a missing key in a checked-in Info.plist were reported identically even
  though the first can be dead code, #if DEBUG, or a file excluded from the shipping target. Each
  of the 53 rules now carries an evidence class (metadata / manifest / resource /
  build-setting / source, the weakest artifact its conclusion depends on) and a confidence
  level (validator-blocking / review-risk / judgment-call, who actually acts on it).
• needs_build_verification, derived not stored: true exactly when a validator-blocking claim
  rests on source or build-setting evidence. It cannot drift out of sync with the catalogues
  because it is computed from them. Phase 5 surfaces the count as a "what this run could not
  establish" note; it never changes the GREEN/YELLOW/RED verdict.
• set_evidence / set_confidence per-branch refinement so a degraded read cannot inherit the
  rule's label. §2's empty-purpose-string FAIL is manifest rather than the rule's source floor;
  §2's "Info.plist not found", §1's "declared but no code usage grepped" and "could not auto-detect
  iOS source dir", §42's "could not read PNG dimensions", §10's "no paywall view found" and §53's
  "no metadata dir" drop to judgment-call instead of claiming Apple's validator blocks them.
• Offline pinned guideline citations
  (scripts/guideline-cite.sh). Pierre now
  quotes Apple's actual wording instead of his recollection of it. Quotes are pinned in
  guidelines-fingerprints.json next to the hashes that already detect when a section changes, so
  an out-of-date quote is a detectable condition rather than a silent lie — offline,
  deterministic, reviewable in git, and staleness-aware (STALE past
  GUIDELINE_CITE_STALE_DAYS, default 120). All 57 covered sections are pinned.
  Exit 3 / NO PINNED CITATION is the load-bearing case: Phase 3 now forbids reconstructing
  guideline text from memory and requires saying the wording could not be verified this run.
• guideline_url on every finding (text via the citation tool, plus JSON and SARIF), so a human
  can always open the section the finding cites. Parenthetical sub-items (5.1.1(v), 3.1.1(a))
  resolve to their anchor.
• scripts/guideline-drift.sh --quotes: maintainer step that fills in the citable quotes only.
  It never touches a fingerprint or reconciled_on, and refuses to re-quote a section that has
  drifted since its baseline — quoting a drifted section would take the new wording while the
  fingerprint still claimed the old, papering over the very change drift detection exists to catch.
• gd_section_quote in skills/appstore-precheck/scripts/lib/guideline-text.sh: original-case, sentence-bounded
  excerpts. The extraction was split into a shared _gd_section_raw, verified byte-identical
  against the live page for all 57 sections so no fingerprint moved.
• tests/test-evidence.sh (57 assertions) and tests/test-guideline-cite.sh. The evidence
  test enforces a completeness invariant: every rule scan.sh sets must carry both labels from the
  closed vocabularies, so a new check cannot ship unlabelled.

Changed

• Text output gains one indented evidence: … line under each FAIL:/WARN: (never under PASS:).
  ^FAIL:/^WARN: anchoring, verdict arithmetic, and the token are untouched; set
  APPSTORE_PRECHECK_NO_EVIDENCE=1 to suppress it.
• --format json findings gain evidence, confidence, needs_build_verification and
  guideline_url; the summary gains by_confidence and needs_build_verification. --format sarif carries the same labels in each result's properties bag, omitted entirely for an
  unclassified rule rather than guessed.
• Phase 3 of SKILL.md now requires a citation lookup per finding and requires Pierre to reflect
  the evidence label — especially to say, in plain words, when a finding rests on a grep rather than
  on the shipping build.
• docs: methodology.md gains "Evidence strength and confidence" (with the generated per-rule
  table) and "Guideline citations" (including why pinned snapshots beat a live fetch here);
  README gains both sections under "How it works".

Added — second pass: the remaining gaps

• SKIP: — a fourth line class for checks that could not run. FAIL/WARN/PASS all assert
  something about the build; a check that never ran asserts nothing, and reporting it as a PASS
  turns an unexamined surface into a clean bill of health. The screenshots check did exactly that
  ("assumed managed in App Store Connect"). SKIP is counted separately (skip= from
  verdict.sh, summary.not_audited in JSON), excluded from SARIF, and never moves the
  verdict — a gap in coverage is not a defect in the build.
• The store listing is no longer skipped in silence. With no fastlane/metadata directory, a dozen
  store-listing checks used to vanish without a word. SKIP: metadata now names how many did not
  run, counted from the evidence catalogue via the new rules_with_evidence so the number cannot
  rot. SKILL.md Phase 1 then asks the user to paste their App Store Connect listing — name,
  subtitle, description, keywords, promotional text, age rating, review notes, demo account — and
  audits it directly; those findings count. If the user declines, or there is no user (CI), they
  stay SKIP. Metadata is never invented.
• A mandatory "Not audited" section on every run, GREEN included: the checks that did not run,
  plus the fixed list of what a static tool can never see (runtime crashes, whether links resolve,
  whether Restore Purchases works, OAuth round trips, server-driven content, the built archive).
  A GREEN that never saw the store listing is a GREEN with a hole in it, and it now says so.
• §54 saturated-category (4.3(b)) — the guideline was not covered at all. Apple names the
  saturated categories ("dating, flashlight, sound effects, wallpaper, simple timers, and fortune
  telling ... we will not accept new submissions unless they offer a meaningfully different or
  improved experience", plus drinking games, kama sutra, fart and burp apps), so the rule matches
  Apple's own list rather than a guess. Matched against name / subtitle / keywords only — those
  say what the app is; the description says what it does, where "flashlight" or "timer" is an
  ordinary feature of an unrelated app. Word-boundary anchored, so fart does not match farther.
  WARN + judgment-call: being in a named category is exposure, not a violation.
• Deep-review check 30 (Tier B, 4.3) — the differentiation judgment the scanner cannot make:
  what does this app do that the incumbents do not, including 4.3(a) variant-bundle-ID patterns.
  Deep review is now 30 checks (23 Tier A + 7 Tier B); the scanner covers 54 vectors.
• guideline-cite.sh --verify-live — change-based staleness. Age was only a proxy, bad in both
  directions: an untouched section stays correct for years, a section edited yesterday is wrong and
  still looks fresh. This re-hashes the live section against the pinned fingerprint (one fetch,
  cached per day, so a whole run costs one request). Unchanged → the quote is current however old
  the pin is, and STALE is withdrawn. Changed → exit 4 and CHANGED, and Phase 3 requires
  saying Apple's text has moved instead of quoting the old wording. Any failure degrades to the
  offline path and never reports a verification that did not happen.
• Phase 0 scans Apple's announcements. developer.apple.com/news carries policy changes —
  deadlines, new required declarations, entitlement changes — often before the guideline text
  catches up. Items newer than the baseline reconciled_on produce a non-blocking WARN.

Changed — second pass

• The shared HTML parser moved from scripts/lib/ to
  skills/appstore-precheck/scripts/lib/guideline-text.sh. Only the skill directory is
  packaged, and --verify-live needs the parser at the user's machine. guideline-drift.sh and
  eval/rag/ingest.sh source it from the new location; there is still exactly one implementation.
• guidelines-fingerprints.json reconciled against the live page on 2026-09-01: all 57 previously
  pinned fingerprints verified unchanged, 4.3 added, 58 sections now carry quotes.

Fixed — adversarial review pass

Found by reviewing the two commits above as a hostile second reader, verifying each suspect by
running it rather than reading it.

• --verify-live cached the live page at a predictable path in shared /tmp and rm -f'd
  that path on failure. Any local user could pre-place a crafted page and have the tool "verify" a
  citation against it; and the very first test pointed the cache at /dev/null, which — run as
  root — would have deleted /dev/null. The cache now lives under the user's own
  ${XDG_CACHE_HOME:-~/.cache}/appstore-precheck (mode 0700), a non-regular file or symlink at the
  path is refused rather than read or removed, and only a file this run created is ever deleted.
  GUIDELINE_CITE_URL was added so the test fails the fetch **offline and ...

v1.17.0 — §53: EULA link required in the App Store description (3.1.2)

New IAP-gated scan vector §53 subscription-eula-metadata: every locale's App Store description.txt must contain a functional Terms of Use (EULA) URL. Apple's automated metadata review rejects auto-renewable-subscription submissions without one — a live rejection passed every existing check (§10 covers the in-app paywall links, but nothing covered the store description). Missing in any locale is a FAIL; a missing fastlane metadata dir downgrades to a verify-manually WARN.

Also: catalog slug subscription-eula-metadata, methodology row 53, SKILL.md/README counts 52 → 53, positive + negative test assertions, clean fixtures updated.

Full changelog: CHANGELOG.md

v1.16.0 — Grok Build support

Added

• Grok Build support: native .grok-plugin/ marketplace + plugin manifests so
  users can install with grok plugin marketplace add berkayturk/appstore-precheck and
  grok plugin install appstore-precheck --trust (or a direct path/GitHub install). The same
  SKILL.md and upload-guard hook work; Grok sets GROK_PLUGIN_ROOT and the CLAUDE_PLUGIN_ROOT
  alias used by hooks/hooks.json. The marketplace entry uses a URL source (Grok rejects a
  self-referential local source), so that flow installs GitHub main; install a path directly to
  test a local clone.
• install.sh grok: vendors the skill into .grok/skills/ (project or user scope). The
  default all target now also populates .grok/skills/ alongside .claude/skills/ and
  .agents/skills/.
• Docs, badge, AGENTS.md, version lockstep (check-versions), and install tests updated for the
  fifth host.

Fixed

• Upload guard no longer fails open on Grok Build (or Cursor): hooks/fastlane-guard.sh read
  only Claude Code's snake_case .tool_input.command, while Grok sends the same event camelCased
  as .toolInput.command and Cursor's shell event carries .command at the top level. The command
  came back empty and every fastlane deliver/pilot/release was allowed. Every plugin manifest
  wires this hook, so the guard now reads all three envelopes, and tests/test-guard.sh covers
  each.
• Block reason reaches the model on every host: the guard now also writes the deny as stdout
  JSON (decision: "deny" plus Claude Code's hookSpecificOutput.permissionDecision) alongside
  the stderr message. Grok honours a stdout deny for PreToolUse and documents stderr feedback
  only for Stop/SubagentStop, so Pierre's reason could previously be dropped there. Exit 2 is
  still the authority.

v1.15.0

Added

• §3/§16 tracking SDK list — 10 new signals: Unity Ads, Vungle / Liftoff, Chartboost,
  InMobi, Mintegral, Pangle (PAGAdSDK and the legacy BUAdSDK), Singular, Kochava, Tenjin,
  and a bare import AdSupport. A missing SDK here is a silent false negative: the 5.1.2 ATT
  check stays quiet on an app that tracks without an ATT prompt.
• §19 analytics SDK list — 8 new signals: PostHog, Heap, Countly, Matomo, Smartlook,
  Instabug, New Relic, Embrace. Same failure mode on the 5.1.1 privacy-manifest check.
• tests/test-sdk-signals.sh: one minimal app per signal, covering every entry in both
  lists including the pre-existing ones, plus five negative cases for the FP-prone words (a
  hand-rolled MinHeap, "Singularity", // Adjust the inset, // Embrace the whitespace,
  a plain SwiftUI screen). Suite is now 27 files.
• corpus/real/rationales.json: a one-line reason per <app>|<rule_id> behind every
  real-panel TP/FP verdict, so a label can be audited without re-deriving the evidence.

Fixed

• §2 photo-library read — fetchAssets false positive: the read signal matched a bare
  fetchAssets, so any method with that name raised a 5.1.1 NSPhotoLibraryUsageDescription
  FAIL — a remote-config loader in duckduckgo-ios, a wallpaper manager in firefox-ios. The
  signal is now PhotoKit-qualified (PHAsset already covers the real
  PHAsset.fetchAssets(...) call). Regression guard: tests/fixtures/fetch-assets-app.

Changed

• Real-app validation panel is fully labelled: the last 127 UNLABELED findings across all
  18 pinned apps now carry a human TP/FP verdict, so the panel reports tp=184 fp=122 unlabeled=0 and a real-code precision of 0.60. The earlier 0.85 was computed over the
  labelled third only. corpus/real/README.md documents the two rules that produce half the
  false positives (screenshot-dimensions, xcode-sdk-requirement) and the label-key
  collision rule (a key counts as TP only when every finding under it is a true positive).
• README: a RED-to-GREEN fix-loop demo above the fold, rendered from the new
  assets/demo-fix-loop.tape and assets/demo-fix-loop.patch so it is reproducible.

v1.14.1 — Norwegian locale false-positive fix

Fixed

• §9 autorenew-disclosure — Norwegian locale false positive: the App Store uses the locale code no while Xcode/xcstrings use nb (Bokmål), so the per-locale disclosure translation check reported a spurious "translation missing for 'no'" warning on apps that correctly localize into nb. The short-code lookup now maps no → nb. (#27)

Full changelog: https://github.com/berkayturk/appstore-precheck/blob/main/CHANGELOG.md

v1.14.0 — 10 new wording-level rejection vectors (43–52)

Added

• §42 permission-priming-cta (catalog vector 43, 5.1.1(iv)): static scan for custom
  pre-permission ("priming") screens whose consent CTA steers users toward granting access
  ("Allow and continue", "Grant access to start", bare "Enable notifications" buttons).
  Signal-gated on real permission-request APIs; scans String Catalog source-language values
  AND hardcoded Swift/ObjC literals; excludes post-denial "Enable X in Settings" guidance
  (Apple's own recommended pattern) and code comments. Driven by a real July 2026 App Review
  rejection of a GREEN-verdict app — the exact wording "Allow and continue" was rejected under
  5.1.1(iv) with Apple asking for "Continue"/"Next" (see corpus/outcomes/ledger.json,
  first missed record). Fixture: tests/fixtures/permission-priming-app.
• Pierre deep-review check 21 (Tier A, 5.1.1(iv)): pre-permission priming CTA neutrality —
  the semantic companion to scan §42 (is the flagged string really on a consent gate; does the
  flow stay usable on decline). Former checks 21–28 renumbered 22–29; Tier B set is now
  4, 5, 7, 10, 15, 29. Totals: 43 scan vectors, 29 deep-review checks (23 Tier A + 6 Tier B).
• First real App Store outcome recorded: corpus/outcomes/ledger.json seeds its first
  missed record (the 5.1.1(iv) rejection above) — the recall gap that motivated vector 43.
• §43 paywall-trial-emphasis (catalog vector 44, 3.1.2): flags paywall purchase CTAs that
  promote the free trial over the billed price ("Continue with free trial", "Start your free
  trial") and the "free-trial toggle" paywall pattern — the early-2026 App Review rejection wave
  under 3.1.2 (the field-proven fix is a neutral "Continue"/"Subscribe" CTA with the price and
  renewal term legible next to it). Signal-gated on IAP signals; scans String Catalog
  source-language values + hardcoded literals; a CTA that already carries a price is excluded.
  Fixture: tests/fixtures/trial-cta-paywall-app.
• §44 metadata-pricing-language (catalog vector 45, 2.3.1): flags "Free", "% off", "sale",
  "discount", or currency amounts in name.txt/subtitle.txt (2.3.1/2.3.7 accurate metadata) —
  the offline complement to fastlane precheck's pricing rules, which need ASC credentials.
  Hyphen compounds ("ad-free") excluded; keywords/descriptions not scanned. Fixture:
  tests/fixtures/promo-metadata-app.
• §45 generic-purpose-string (catalog vector 46, 5.1.1(ii)): flags non-empty
  NS*UsageDescription values that are very short or pure permission-restating boilerplate
  ("This app needs camera access") — vector 2 checks presence, this checks substance; the static
  complement to deep-review check 18. Fixture: tests/fixtures/generic-purpose-app.
• §46 ai-provider-consent (catalog vector 47, 5.1.1): an external AI endpoint/SDK
  (OpenAI, Anthropic, Gemini, Mistral, OpenRouter, Groq, Perplexity, Together) with no
  user-facing string naming the provider — the 2026 consent-screen requirement for sharing user
  data with third-party AI (5.1.1 / 5.1.2(i)). The endpoint URL literal itself does not count
  as a mention. Fixture: tests/fixtures/ai-chat-app.
• §47 paywall-urgency (catalog vector 48, 3.1.2): fake-urgency purchase pressure —
  "limited time" / "only today" / "last chance" copy (multilingual) and countdown timers
  combined with discount wording in paywall views (3.1.2 / 2.3.1). Fixture:
  tests/fixtures/urgency-paywall-app.
• §48 rating-sentiment-gate (catalog vector 49, 5.6.1): "Enjoying the app?"-style
  sentiment pre-filtering next to a rating-prompt API — routing only happy users to the review
  sheet is rating manipulation. Fixture: tests/fixtures/rating-gate-app.
• §49 forced-login (catalog vector 50, 5.1.1(v)): a credential login UI with no
  skip/guest/continue-without-account affordance (WARN-verify heuristic). Fixture:
  tests/fixtures/forced-login-app.
• §50 push-marketing-optout (catalog vector 51, 4.5.4): a marketing-push SDK (OneSignal,
  Braze, CleverTap, Iterable, Airship, MoEngage) registering for notifications without a
  notification-preferences/opt-out signal. Fixture: tests/fixtures/push-marketing-app.
• §51 xcode-sdk-requirement (catalog vector 52, 2.1): LastUpgradeCheck clearly pre-26
  vs the April 2026 iOS 26 SDK (Xcode 26) upload minimum (WARN-verify heuristic — the field
  tracks the upgrade-check, not the build toolchain). Fixture: tests/fixtures/old-xcode-app.
• Multilingual steering/trial patterns for vectors 43–44: the permission-priming (§42) and
  trial-CTA (§43) regexes now also match common Turkish/German/French/Spanish wording in both
  word orders ("İzin ver ve devam et", "Ücretsiz denemeyi başlat", "Kostenlos testen") — a
  non-English source language is no longer a blind spot for these checks.
• Pre-submit manual checklist extended with the ASC-side items the scanner cannot see:
  exact paywall↔ASC price match, trial configured as an Introductory Offer, the updated
  age-rating questionnaire (13+/16+/18+), EU DSA trader status, and the AI-consent screen.
  Totals: 52 scan vectors.

Fixed

• Version lockstep: package.json and both plugin manifests were left at 1.13.1 when
  SKILL.md moved to 1.14.0 (scripts/check-versions.sh was failing); all four now agree.
• Last-section fingerprint churn (guideline drift): gd_section_text ran to end-of-page
  for the last numbered section (5.6.4), gluing the "After You Submit" block, the
  "last updated" date, and the whole site footer onto its prose — so ANY footer edit fired a
  false "5.6.4 text drift" WARN (exactly what happened on 2026-07-23; the actual 5.6.4 prose
  was unchanged). Extraction now truncates at those page-chrome markers; fingerprints
  reconciled (only 5.6.4's hash changed).
• --reconcile wrote a literal placeholder date: reconciled_on was set to the string
  "RECONCILE_DATE" instead of today's date (single-quoted jq program); now --arg-injected.
• README drift from the vector-43 commit: the static-scan table was missing the
  5.1.1(iv) permission-priming row, the deep-review table was missing check 21 (priming CTA
  neutrality), and the Tier A count still read 22 (should be 23).

Added (eval, dev-only — not part of the distributed package)

• RAG-grounded Pierre experiment: eval/rag/ — ingests the full App Store Review
  Guidelines text (all ~125 sections, not just the officially-mapped subset), embeds it
  (Gemini gemini-embedding-001, MRL-truncated to 1024 dims), stores it in a local pgvector
  instance, and retrieves top-k relevant sections to ground Pierre's eval-harness prompt
  (eval/run.sh --rag, eval/lib/build_request.py --retrieved). Measured against the
  existing 21-case labeled dataset: grounded and ungrounded runs scored identically
  (F1 1.00 on both, all tiers) — a ceiling effect, not a broken pipeline (retrieval was
  independently confirmed to surface the correct guideline section per case). See
  docs/rag-eval-results.md and docs/specs/2026-07-17-rag-grounded-pierre-eval-design.md.

Fixed (eval, dev-only)

• Gemini embedding requests now actually apply taskType/outputDimensionality: the
  request builders nested both fields inside an embedContentConfig object, which the v1beta
  REST endpoint silently ignores (the documented curl examples put them at the top level) —
  the published 2026-07-17 run therefore used embeddings without
  RETRIEVAL_DOCUMENT/RETRIEVAL_QUERY task optimization, and the API's 3072-dim responses
  were only tamed by the client-side MRL truncation safety net. Fields moved to the top level;
  results doc amended (conclusion unchanged — both configurations were at the F1 ceiling).
• RAG corpus deliberately not committed: eval/rag/corpus/sections.json (full Apple
  guideline prose) is now gitignored with the copyright rationale documented in
  eval/rag/README.md; the design spec's "this file is committed" line is amended.
• RAG CLI/network edge cases hardened (dev-only): retrieve.py malformed args exit 64
  with a usage message instead of a traceback; a bare network failure (no HTTP response) in
  the shared Gemini client raises a clear error instead of an unhandled traceback; run.sh --rag no longer leaks its temp request file when retrieval fails.

Added (eval, dev-only — not part of the distributed package)

• Two guideline-drift-sensitive check-16 (4.8) cases — the follow-up proposed in
  docs/rag-eval-results.md: check16-eid-login-exempt (sole eID/BankID login — exempt under
  the current 4.8 wording, likely flagged by stale parametric memory; the first case where
  grounded and ungrounded runs may genuinely diverge) and check16-google-login-only (clear
  violation under any wording; control twin). Both label_confirmed: false (UNLABELED,
  excluded from all metrics) pending human label review; first eval coverage for check 16.
  Labels human-confirmed 2026-07-18.
• 23-case RAG re-run — first measured divergence: both configurations re-run with
  task-typed embeddings on the expanded dataset. F1 still 1.00 everywhere (ceiling), but
  consistency split 0.96 vs 1.00 — the single non-unanimous case was exactly the
  drift-sensitive check16-eid-login-exempt (ungrounded: pass/pass/not-applicable wording
  drift; grounded: unanimous pass, citing the retrieved 4.8 exemption, top-1 similarity
  0.76). A consistency effect, not an accuracy effect — documented with scope caveats in
  docs/rag-eval-results.md.

v1.13.1 — Pierre pass/not-applicable fix, eval-verified; three-model scorecard

Fixed

Pierre's pass-vs-not-applicable boundary. The deep-review output format now states explicitly: "not applicable" is only for checks whose subject matter is entirely absent from the project; material that exists and is clean gets a plain REVIEW-PASS with evidence.

This is the eval harness (v1.13.0) paying for itself on day one: the consistency metric caught 3 of 63 clean-case verdicts drifting to "not applicable" across Opus 4.8 and Fable 5 (binary verdicts were never wrong — only the wording boundary). After the one-paragraph clarification, full re-runs of all three models show consistency 1.00 across the board with no regression on genuine not-applicable cases. Measure → fix → re-measure, closed loop.

Added (eval)

• Multi-model scorecard: docs/llm-scorecard.md opens with a comparison table and renders every committed baseline — currently Claude Sonnet 5, Opus 4.8, and Fable 5, before and after the prompt clarification. The CI Tier-A F1 ≥ 0.80 floor gates each baseline separately.
• Fable/Mythos-tier support: eval/run.sh --model claude-fable-5 works (thinking field omitted, max_tokens headroom, both recorded in the manifest).
• Prompt fingerprinting + stricter resume: every manifest records a prompt_sha256 of the Pierre prompt; the resume guard refuses to mix models or prompt versions in one cache dir.

Full changelog: https://github.com/berkayturk/appstore-precheck/blob/main/CHANGELOG.md

v1.13.0 — LLM eval harness: the deep-review layer is now measured

Added

LLM eval harness (eval/) — Pierre's Phase 4 deep review (28 semantic checks, incl. the 6 heuristic Tier B checks) is now measured against a human-labelled dataset, not just described. Additive and opt-in: nothing in the default scan path changes, no new network calls, verdict logic untouched.

• Dataset: 21 human-labelled cases — positive+negative pairs per covered check, deliberate false-positive traps, borderline cases, pre-fetched-URL cases for the privacy-policy check. Unconfirmed labels report as UNLABELED and never enter headline metrics.
• Runner (eval/run.sh): pinned model + generation params recorded per run, --repeat for consistency measurement, response caching with resume. Per-model baseline dirs with a guard that refuses to mix models in one cache.
• Offline scorer (eval/score.py → docs/llm-scorecard.md): per-check and per-tier precision/recall/F1, Tier-B false-positive rate, majority-vote scoring, unanimity-based consistency.
• Committed baselines: Claude Sonnet 5 and Claude Opus 4.8, both 21/21 correct (Tier-A F1 1.00, Tier-B FP rate 0.00). Consistency: 1.00 (Sonnet) vs 0.95 (Opus — one pass/not-applicable wording drift). Sonnet is the pinned default: equally accurate here, more consistent, 2.5× cheaper.
• CI: blocking offline gate (dataset validity, scorecard freshness, Tier-A F1 ≥ 0.80 floor on the committed baseline) + a non-blocking live smoke job.

Honest caveat, straight from the scorecard: these numbers measure fidelity to this project's own labels on minimal synthetic fixtures — not agreement with Apple's actual review decisions. Counts are printed next to every rate so small samples stay visible.

Full changelog: https://github.com/berkayturk/appstore-precheck/blob/main/CHANGELOG.md

v1.12.2 — false-positive fix: build checkouts no longer scanned

Fixed

• iosSourceDir root scans no longer sweep build checkouts: the ~34 code-level greps scoped to $IOS_DIR (purpose-string, tracking/ATT, IAP, private-API, SIWA, and the other ${SRC_INC[@]}-based checks) now apply the same GREP_PRUNE exclude-dirs as the repo-wide passes. When iosSourceDir resolves to the repo root (e.g. "."), these greps previously swept gitignored build/, DerivedData/, SourcePackages/checkouts/, and .build/ output, misreading vendored SDK code as the app's own — observed in the wild as a false ATT 5.1.2 FAIL (RevenueCat mock under build/SourcePackages) and a false 2.5.1 private-API FAIL (sentry-cocoa checkout under .build/).

See CHANGELOG.md for details. (#20)

v1.12.1 — quality patch: false-RED fixes, ObjC coverage, --dir, CI hardening

Quality patch from a full fresh-eyes review (three independent review passes). No new checks — every change removes a false-RED path, fixes a real bug, or hardens the release surface.

Scanner accuracy (false-RED cluster):

• 3.1.2 Restore Purchases recognizes StoreKit 2 (AppStore.sync(), capitalized button labels)
• 3.1.2 Terms/Privacy links match human-readable labels and nonstandard URL paths
• RevenueCat/Adapty remote-configured paywalls: missing links WARN (verify in dashboard) instead of hard-FAIL
• 5.1.1 FileTimestamp anchored to real filesystem APIs (no more creationDate property false-FAILs)

Coverage & correctness:

• Objective-C blind spot closed: all code greps share one include-set (*.swift *.m *.mm *.h)
• --dir is authoritative (new scan.sh flag; CLI + Action wired) — monorepo subdir scans and SARIF paths fixed
• Rule 42 (screenshot-dimensions) is suppressible; verdict thresholds deduplicated; CLI exits 70 on a signal-killed scanner

Tests & CI: 3 new fixtures + monorepo/suppression/npm-pack regression tests (18 test files); least-privilege workflow permissions; SHA-pinned actions.

Full details: CHANGELOG.