0.4.8

• Add experiment view (alias show): shows llm-params.csv per provider (--params, the default), llms-for-experiment.csv (--llms) and a summary of data, prompts and mapping (--data).

0.4.7

• Add the openrouter LLM provider, reaching the OpenRouter <https://openrouter.ai/>_ gateway over its OpenAI-compatible endpoint. Only PROVIDER_OPENROUTER_KEY is required; the URL defaults to https://openrouter.ai/api/v1. Adds the openrouter_provider_order and openrouter_reasoning_effort hyperparameters in the experiment init templates.
• Add a spending cap to experiment run: OpenRouter call costs are read from usage.cost, logged to a new cost_logs table and counted against PROVIDER_OPENROUTER_MAX_SPEND (default 5.00 USD). Reaching it pauses the run gracefully, leaving the remaining rows pending. The tally is per run and resets when the command exits.
• Edit the litellm example profile of the experiment init template: deepseek-ai/DeepSeek-V4-Flash-0731 replaces minimax-m2.7:229b.
• Add a total_costs metric to experiment stats: the sum of cost_logs.cost_usd, shown as USD.
  0.4.6 (2026-09-21)

---

• Breaking: add data_id and prompt_id columns to every experiment_<provider> / try_experiment_<provider> table, written from mapping.csv by generate, update and try.
• An older database without the two columns is refused on open: re-run experiment generate, or add and fill the columns in place manually using SQL (check migrations.md for details on the manual migration)
• Breaking: analysis add-agreement now scores Cohen's kappa per prompt and pairs profiles rather than provider/model/profile triples.
• Add intermediate_values_dir to pairwise_cohen_kappa, writing the rows behind each kappa to one CSV per prompt
• Edit experiment try that no longer needs a generated database: it warns that the try will not be stored, then runs and saves the response file as usual. A --file naming a missing database still aborts.
• Add --suggest to experiment try: prints one randomly chosen combination of data.csv, prompts/ and llms-for-experiment.csv as a ready command, and runs nothing.

0.4.6

• Breaking: add data_id and prompt_id columns to every experiment_<provider> / try_experiment_<provider> table, written from mapping.csv by generate, update and try.
• An older database without the two columns is refused on open: re-run experiment generate, or add and fill the columns in place manually using SQL (check migrations.md for details on the manual migration)
• Breaking: analysis add-agreement now scores Cohen's kappa per prompt and pairs profiles rather than provider/model/profile triples.
• Add intermediate_values_dir to pairwise_cohen_kappa, writing the rows behind each kappa to one CSV per prompt
• Edit experiment try that no longer needs a generated database: it warns that the try will not be stored, then runs and saves the response file as usual. A --file naming a missing database still aborts.
• Add --suggest to experiment try: prints one randomly chosen combination of data.csv, prompts/ and llms-for-experiment.csv as a ready command, and runs nothing.

0.4.5

• Add project list: every project under .projects/ with the parts it holds (search, experiment, analysis, papers) and its creation date. Takes --sort-by and --desc.
• Edit project create that now also writes a README.md describing what the project holds and the llmexer version that created it.
• Edit project create that now takes its .gitignore and README.md from the bundled data/project/ files instead of strings in the code.
• Add experiment fix that makes JSON responses from LLM more processing-friendly by repairing them. Logging every change to the new datafix_logs table. Writes only with --apply.
• Add --ensure-json-format param to experiment fix, fixes the format of the LLM response by removing prose and converting from markdown to JSON.
• Add --rename-json-field OLD NEW param to experiment fix, renaming a top-level field of the answer JSON instead of repairing it.
• Edit experiment init that now takes its starter prompt from the bundled data/experiment/prompt01.txt instead of a string in the code. The example now asks for a JSON answer with yes/no fields.
• Add analysis add-agreement: a notebook reporting Cohen's kappa between every pair of provider, model and profile, per yes/no field. It loads and flattens the experiment itself, so it runs without the other notebooks.
• Added a new dependency to analysis dependency group - scikit-learn
• Edit search create that now defaults to a broader boolean query, written single-quoted on one line in the YAML.
• Edit project, experiment and search that called on their own now answer instead of printing the help: the first two report what is currently set, search lists the searches.

0.4.4

• experiment export: add --filter-code, a case-insensitive glob matched against the whole code, so S12_* selects every combination built from one data item.
• experiment export: add --filter-response, a case-insensitive regular expression searched anywhere in the answer. Rows that have not run carry no answer and drop out.

0.4.3

• experiment copy-search also copies the publication year, writing ID;Title;Abstract;year;doi;authors. A search file without a year column is now rejected.
• experiment export takes run's --filter-provider / --filter-model / --filter-profile. Each filter is appended to the HTML file name and named in the page header.
• project create now writes a .gitignore into the new project folder, keeping the generated files (e.g., databases, HTML, JSON responses, backups, etc.) out of version control.
• Add experiment compact: packs an experiment database into a .7z archive of the same name next to it, leaving the database in place.
• Added a new dependency - py7zr

0.4.2

• Breaking: the flattened answers exported by the experiment notebook now carry code, model_name, provider_name and profile_name ahead of the parsed values, so a row can be traced back to the combination that produced it. transform.flattened_only() adds them unconditionally; stats.value_counts_summary() leaves them out of the answer profile.
• Both analysis notebooks split their setup cell in three — folders, imports, then DB_FILE / SEARCHES alone with the prints, so the value a reader actually changes is easy to find.
• Add stats.answers_by_model() and plots.plot_answers_by_model(): one answer column crosstabbed against provider and model, as a table and as a chart. Both are wired into analyse_experiment.ipynb — re-run analysis init --rewrite to pick them up.

0.4.1

Full Changelog: 0.4.0...0.4.1

0.4.0

ADD: a new CLI category called `analysis` that should simplify analys…

0.3.10

FIX: documentation