Skip to content

Align crash benchmark audit with current launch and setup workflow #461

Description

@janicduplessis

Problem

The launch-crash collector rejects a run that successfully launches the crashing app, captures its unique error and source stack with stim logs --errors, repairs the injected throw, and proves the unchanged Settings screen.

Evidence

In the campaign tracked by #450, stim ios exits successfully after adopting the prepared simulator and using the cached app. It reports an error-level device-log record but does not print the unique JavaScript token inline. The immediately following stim logs --errors contains the unique token, RootLayout, and app/_layout.tsx:27:18. Source inspection and the minimal repair happen only after that log capture; Settings and recording audits pass. The collector reports launch-crash-initial-launch-evidence-missing.

Cause

launchCrashDiagnosis requires a successful Stim launch command to print the token itself, unlike control. Its pre-capture allowlist also predates the current guide/doctor/Git-worktree-plus-warm workflow. These requirements are stricter than the documented launch-then-capture task and prevent evaluating the current CLI workflow.

Fix idea

Accept successful launch evidence followed by a separate actionable, run-token-specific error capture for both arms. Align pre-capture setup with the current prompted workflow while continuing to reject application-source reads/edits before captured errors. Cover safe dependency-resolution diagnostics narrowly; do not allow arbitrary script evaluation. Re-audit the preserved attempt without resampling or changing its timing/evidence.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions