v1.1.0: Self-reviewed working paper

Self-reviewed release of the working paper "Can LLM Agents Test Social-Science
Theory? A Confound-Aware Framework and a Seven-Generation Negative Result on
Bloom's 2-Sigma Problem in LLM Agents."

Status: working paper, not peer-reviewed.

What this release adds

A completed author self-review of all thirteen units of the work — every
experimental generation (v1–v3, v4, v5, v6, v7a, v7b, v8, v9b, v9c, v9c2, the
ablation), the cross-version confound audit, and the paper itself. Each reported
figure in the paper body was traced to the released databases; all of them
reconcile. The records are released under
experiments/bloom_1n_vs_1on1/results/reviews/, stating for each unit what was
verified and what is taken on trust.

Paper changes the review produced:

• Section 8 states the three null results — class size, learner ownership, and
  discussion value — as separate findings. They had been bundled into one
  sentence, and a null reported only as an interpretation-flag value is easy to
  miss.
• Section 9 gains three limitations: two of the four readiness-check items appear
  as worked examples, answers included, in the fixed lecture; A8's semantic judge
  runs but its verdict never reaches answer_correct, so no generation here has
  a validated L6 measurement; and one of the four learner types was never
  actually instantiated, because the constraint shuffles a list of facts while the
  solving procedure sits in a field it does not touch.
• The explanation attributing correction-outcome decoupling to a procedural
  deficit is withdrawn in all three places it appeared. The decoupling stands as
  an observation; we no longer offer a mechanism.
• Section 9 quantifies the run-to-run variation it had only acknowledged: on the
  same lecture and the same readiness check, one check type passed at 50%, 85%
  and 12% across three runs.
• The cross-version audit table is extended to v9c2, so the generation that
  resolved the pseudoreplication confound is recorded as having done so. The v8
  row is corrected — its whole-class condition was a full instance of F4, not a
  partial one; the shared prerequisite lecture had been counted in the
  denominator.
• One reference DOI did not resolve and is corrected.

Also in this release:

• Japanese translation of the paper
  (results/2026-06-16-paper-draft.ja.md)
• Typeset PDF (18 pages). The LaTeX source had never been built; it needed
  natbib and several table and line-breaking fixes.
• lib/report.rb now reports the path L6 was actually scored by, rather than
  printing a header and a per-row note that contradicted each other. Rows whose
  semantic verdict was not recorded read "not measured" instead of zero.
• The development log's account of the v1 ceiling and the v3 no-education figure
  is corrected in place.

Artifact

Code, configs, prompts, the synthetic Zarn Tokens domain, tests, per-run SQLite
databases and raw run directories. The ablation run's database was lost; its
generated report is released verbatim, with a note that the report generator
emits sections that run did not have.

A day-by-day research journal is published separately at
https://kotowari-modoki.github.io/experimental-commons/research/agent-based-theory-testing-2-sigma-problem/

AI involvement

The experiments, analyses and manuscript were produced end-to-end by LLM agents
under the human author's direction and review. The same model family serves as
the experimental subjects; the paper discusses this reflexivity and notes that
independent human replication of the confound audit is the obvious external
check.

v1.0.0: Working paper release

Working paper release for "Can LLM Agents Test Social-Science Theory? A Confound-Aware Framework and a Seven-Generation Negative Result on Bloom's 2-Sigma Problem in LLM Agents."\n\nDOI: https://doi.org/10.5281/zenodo.21186083\n\nStatus: working paper, not peer-reviewed.\n\nIncluded artifacts:\n- Final paper Markdown: experiments/bloom_1n_vs_1on1/results/2026-06-16-paper-draft.md\n- Final paper LaTeX: experiments/bloom_1n_vs_1on1/results/2026-06-16-paper-draft.tex\n- Experiment code, configs, prompts, tests, version-by-version analyses, and confound audit artifacts\n- Citation metadata: CITATION.cff\n- Zenodo metadata: .zenodo.json\n\nLicense: MIT.