Testing and Guardrails

Testing and Guardrails

Forsetti uses tests, scripts, and pull request workflows to keep runtime behavior, versioning, and architecture boundaries stable.

Verification Stack

flowchart TD
    Change["Code, documentation, or workflow change"] --> Unit["swift test"]
    Change --> Guardrails["Scripts/verify-forsetti-guardrails.sh"]
    Change --> Version["Scripts/update-pr-version.py checks"]
    Change --> Lint["SwiftLint when available"]
    Guardrails --> Package["Package dependency graph checks"]
    Guardrails --> Imports["Import boundary checks"]
    Guardrails --> Final["Final class checks"]
    Guardrails --> Manifest["Template manifest declarations"]
    Unit --> Result["Merge-readiness evidence"]
    Guardrails --> Result
    Version --> Result
    Lint --> Result

Loading

Test Targets

Target	Coverage
ForsettiCoreTests	SemVer, manifest loading, compatibility, registration records, runtime requirements, activation lifecycle, entitlements, UI surface manager, module communication, and logging.
ForsettiPlatformTests	StoreKit provider behavior, secure storage, file export, platform logging, and entitlement provider factory behavior.
ForsettiHostTemplateTests	Host controller boot, module lists, locked modules, restore purchases, service toggles, UI selection, and toolbar routing.
ForsettiArchitectureTests	Package graph, import rules, internal layering, final class expectations, and template manifest requirements.

Core Commands

swift test
Scripts/verify-forsetti-guardrails.sh

Run both before pushing runtime, package-boundary, manifest, or template changes.

Version Workflow Checks

Run this from the repository root:

python3 -m py_compile Scripts/update-pr-version.py
tmpdir="$(mktemp -d)"
cp -R version.txt README.md Sources "$tmpdir/"
(
  cd "$tmpdir"
  python3 "$OLDPWD/Scripts/update-pr-version.py" \
    --base-version 0.1.0 \
    --title "docs: align documentation surfaces"
)

Use the script check when editing PR version automation, version files, release configuration, or documentation that describes release behavior. A docs: title should produce the next patch version in the copied files; a chore: title should skip the update.

CI Workflows

Workflow	Purpose
guardrails.yml	Runs package tests and repository guardrails.
lint-pr.yml	Validates pull request titles against conventional commit-style formats.
pr-version.yml	Updates SemVer files on same-repository pull requests unless the PR title is chore:.

Wiki publishing is manual through the GitHub Wiki repository so curated pages, diagrams, and operational guidance stay separate from repository source changes.

Guardrail Logic

flowchart LR
    Files["Repository files"] --> Package["Package.swift graph"]
    Files --> Source["Sources imports"]
    Files --> Tests["Tests imports"]
    Files --> Manifests["Example and template manifests"]
    Package --> Rules{"Expected layering?"}
    Source --> Rules
    Tests --> Rules
    Manifests --> Rules
    Rules -- yes --> Pass["Pass"]
    Rules -- no --> Fail["Fail with exact violation"]

Loading

Manifest Guardrails

Check	Expected Result
Example and template manifests declare manifestTemplateVersion.	New manifests make their template contract explicit.
Template 1.1 manifests declare runtimeRequirements.	Service, UI, and data requirements are reviewable.
UI/app modules declare contributed IDs.	Activation can validate toolbar, view, slot, route, pointer, and theme IDs.
Service modules do not provide UI contributions.	Runtime boundaries stay clean.

Pull Request Quality Flow

sequenceDiagram
    participant Author as Author
    participant PR as Pull Request
    participant Version as PR Version Workflow
    participant Checks as Tests and Guardrails
    participant Main as main
    participant Release as Release Process

    Author->>PR: open PR with conventional title
    PR->>Version: parse title/body
    alt title is chore
        Version-->>PR: no version change
    else non-chore title
        Version-->>PR: update version files
    end
    PR->>Checks: run guardrails, tests, title lint
    Checks-->>PR: status
    PR->>Main: merge when checks pass
    Main->>Release: release work evaluates release state

Loading

Pre-Merge Quality Bar

• New runtime behavior has unit tests.
• New host controller behavior has host-template tests.
• New platform behavior has platform tests or a documented platform limitation.
• Architecture-affecting changes pass guardrail scripts.
• Manifest and template changes keep schema/template/runtime requirements aligned.
• Version automation changes are smoke-tested locally.
• Documentation updates accompany behavior changes.
• Wiki pages are updated manually for architecture, API, workflow, or integration changes.