You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Breaking: add data_id and prompt_id columns to every experiment_<provider> / try_experiment_<provider> table, written from mapping.csv by generate, update and try.
An older database without the two columns is refused on open: re-run experiment generate, or add and fill the columns in place manually using SQL (check migrations.md for details on the manual migration)
Breaking:analysis add-agreement now scores Cohen's kappa per prompt and pairs profiles rather than provider/model/profile triples.
Add intermediate_values_dir to pairwise_cohen_kappa, writing the rows behind each kappa to one CSV per prompt
Edit experiment try that no longer needs a generated database: it warns that the try will not be stored, then runs and saves the response file as usual. A --file naming a missing database still aborts.
Add --suggest to experiment try: prints one randomly chosen combination of data.csv, prompts/ and llms-for-experiment.csv as a ready command, and runs nothing.