Skip to content

v0.5: The Grammar of Simulation

Choose a tag to compare

@yoheinakajima yoheinakajima released this 02 Nov 20:43
f65a964

The Grammar of Simulation, version 0.5 (2026-11-02). This revision answers the eight findings that Explore Science's third review (score 90) left open. Each of them asks for a second rater. Where no other person was available, it does the analyses that need no rater, and adds a blind check by a second model.

  • Dataset: sha256 3c96f240392d1eb3d49c3d5ca85d751283aba26a082850481fc2b460ed745d69.
  • Read it: at https://simgrammar.com/paper/, or as the attached PDF.
  • Cite it: CITATION.cff, with the dataset hash. The concept DOI 10.5281/zenodo.23846209 resolves to the newest version.

What changed since v0.4.1 (#72–#81):

  • A cross-model check. A second model, gpt-6-sol, coded and rated blind samples of every coding and rating decision under the same operator.
    • It agrees closely on families, simulation-state classes and technique maturity (κ 0.82 to 0.90), less on effect types (κ 0.73), and little on opportunity scores (α 0.28 to 0.48).
    • The paper reports it as agreement across models, not independent agreement.
  • The bets ranking does not replicate. The two readings' priority orders correlate at ρ = 0.12. The stored ranking stays the original rater's, and the abstract now says so.
  • Effect types. An audit applied the written §8.1 order to every flagged edge, relabelling 41 of them. The second model had already given 8 of the 9 sampled relabelled edges their new label. Replacing a mechanic's substrate is now the most common effect, and agents & behavior the most exposed family.
  • Edge recall. A keyword sweep of 231 cited sources found 1 missing edge. A blind recall check found 72% of the recorded edges and 53 extra ones, so the paper now treats an absent edge as weak evidence.
  • Mechanic state classes. Every mechanic is classed as simulation state, control, meta-structure or presentation. Clustering and co-occurrence are re-run on the simulation-state subset.
  • Family sensitivity. White space and exposure are re-run with borderline mechanics moved to their next-closest family. Small families now carry bootstrap intervals.
  • Rating reliability. A simulation finds that a reliable alpha would need 120 to 150 doubly rated mechanics. Rating noise at the observed agreement weakens the score correlations but keeps their sign.
  • Predictions. Each now has a pre-specified reference class, generic-diffusion baseline and catalog base rate, and three criteria gained size bars.
  • Prior art. A recorded search documents every novelty score. It lowered 16 scores and raised none, and three TRLs were corrected.
  • Videos. Eight narrated scenes are re-voiced to match. They are published separately from yoheinakajima/simgrammar-media.

Still open: agreement with a second person. tasks/second-rating.md lets one independent contributor run every sheet in a single pass.

🤖 Generated with Claude Code