v1.0.0

The first stable release. The Python API, the run.yaml and results.json formats, and the paperlab-space command now follow semantic versioning: configs and results from 1.0 keep working in every 1.x release.

• Text metrics: bleu and chrf (chrF/chrF++) via sacreBLEU, with the signature saved for your paper, and rouge1, rouge2, rougeL. Chinese, Japanese, Thai, Burmese, and other scripts without spaces are handled.
• LLM-as-judge: llm_judge grades each output against the reference or your own rubric, with the same rate limits, retries, resume, and estimates as model requests. The judge's reasoning is saved for every row.
• Confidence intervals: a 95% bootstrap interval for every score, in the Summary sheet, the CLI, and optionally table.tex.
• Significance tests: paired permutation (default) or bootstrap tests between models, Holm-corrected, in a new Significance sheet. With statistics.baseline, table.tex marks significant differences with a dagger.
• results.json files from 0.1 still load. New dependencies: sacrebleu and numpy.

Docs: LLM-as-judge · Statistics · Changelog

v0.1.6

• Stopped runs keep their progress: finished rows are saved to results.json as the run goes and when you press Ctrl+C, so a stopped run continues with paperlab-space run run.yaml --resume results-run/results.json. In Python, pass checkpoint= to run or arun.
• Documentation at paperlab.space/docs: guides, the CLI, a config file reference, a metrics reference, and a Python API reference generated from the source.
• Docstrings for every public field, shown in help() and in editors.
• The citation title is now "paperlab-space: LLM evaluation on spreadsheet data".

v0.1.5

• Rate-limit errors meaning the key has no quota for a model (Gemini limit: 0, OpenAI insufficient_quota) now stop that model immediately with a clear message instead of retrying every row.
• Progress now includes the finished RowOutput (row), so callers can checkpoint partial runs and resume them with previous=.
• Examples, README, and the init template use gemini/gemini-3.5-flash; Google retired Gemini 2.5 for new API keys.

v0.1.4

• Readable error messages: failed requests now show the provider's own message, e.g. Rate limit or quota exceeded at gemini: You exceeded your current quota, please check your plan and billing details., instead of LiteLLM's raw exception text. For OpenRouter, the upstream provider's message is used.
• Privacy: results.json now stores only the file name of the dataset, not its full local path, so it is safe to share (the same rule run.yaml already followed).

v0.1.3

Fix: list_sheets() now raises a clear DataError ("Could not open Excel file ...") for corrupt or non-Excel .xlsx files, instead of a raw BadZipFile error. Found while building the PaperLab.space upload flow.

v0.1.2

First release archived on Zenodo, so paperlab-space can now be cited with a DOI.

• Citation metadata now lists the author (Kaung Nay Lin Khant, ORCID 0009-0005-5612-5755) and an abstract
• No code changes since 0.1.1

pip install paperlab-space

v0.1.1

The paperlab-space command is now part of the normal install, so you no longer need the [cli] extra:

pip install paperlab-space
paperlab-space --help

pip install 'paperlab-space[cli]' still works and installs the same thing.

Also: the README now points to PyPI for installation.

v0.1.0

First public release of the open-source engine behind PaperLab.space: evaluate LLMs on your own spreadsheet data and get paper-ready results.

pip install 'paperlab-space[cli]'
paperlab-space init my_data.xlsx
paperlab-space run run.yaml

What's included

• Data: .xlsx (any sheet), .csv, .tsv; encoding detection and Unicode-safe handling for non-Latin scripts (tested with Burmese, Thai, Chinese, Japanese, Korean); validation for empty cells, duplicate IDs, and garbled text
• Models: OpenRouter (including free models), Gemini, OpenAI, Anthropic, and anything else LiteLLM supports, side by side, with your own API keys
• Runs: per-model rate limits tuned for free tiers, retries with backoff, resume after interruption, and a cost and time estimate before anything is sent
• Metrics: exact match, accuracy (with label extraction), precision/recall/F1, token F1
• Output: Excel workbook, booktabs LaTeX table, reproducibility YAML (model versions, prompt, seed, metric settings, data hash), full results JSON
• CLI: paperlab-space init | validate | estimate | run [--resume] | metrics

See the README for the quickstart and PLAN.md for what's next (BLEU/chrF/ROUGE, LLM-as-judge, significance tests).