Skip to content

0.4.6

Choose a tag to compare

@github-actions github-actions released this 21 Sep 21:27
· 17 commits to main since this release
  • Breaking: add data_id and prompt_id columns to every experiment_<provider> / try_experiment_<provider> table, written from mapping.csv by generate, update and try.
  • An older database without the two columns is refused on open: re-run experiment generate, or add and fill the columns in place manually using SQL (check migrations.md for details on the manual migration)
  • Breaking: analysis add-agreement now scores Cohen's kappa per prompt and pairs profiles rather than provider/model/profile triples.
  • Add intermediate_values_dir to pairwise_cohen_kappa, writing the rows behind each kappa to one CSV per prompt
  • Edit experiment try that no longer needs a generated database: it warns that the try will not be stored, then runs and saves the response file as usual. A --file naming a missing database still aborts.
  • Add --suggest to experiment try: prints one randomly chosen combination of data.csv, prompts/ and llms-for-experiment.csv as a ready command, and runs nothing.