Skip to content

Stage 0 - Pilot run and pre-registration (freeze the analysis plan before the main runs) #9

Description

@sebasmos

Goal. Make the whole study confirmatory rather than exploratory.

Scope / what to build.

  • A smoke run of about 50 twin pairs through solo mode to (a) verify cues actually bite for at least one model, (b) estimate effect sizes for a power analysis, and (c) lock the analysis plan.
  • A written pre-registration: the confirmatory endpoints, the tests, the exclusion rules, and the multiple-comparison policy, committed before the main runs. Anything added later is labeled exploratory.

Definition of done. The pilot shows a nonzero flip rate on at least one cue type; the power analysis sets sample sizes; the pre-registration document is committed and referenced by every later stage.

Depends on: Stage 0 harness, roster, dataset loaders, cue pipelines, statistics module

Model backend. For now every agent uses the Gemini API (multimodal). All model access goes through one gateway wrapper, so the roster can be extended to other model APIs later without changing any experiment code. Gemini is the default for now; no fine-tuning, models are used off-the-shelf.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions