v1.2.0: Calibrated on SlopBench, 14 more languages

One calibrated score, measured on a public benchmark, in English and 14 more languages.

New

• SlopBench: 3,252 texts. Half are human texts written before ChatGPT. The other half are AI texts on the same topics and at the same lengths, from 14 current models, across 15 kinds of writing: news, Wikipedia, arXiv, Stack Exchange, Reddit, reviews, student essays, non-native English, and literature from 1532 to 1915. Every script and score is in the repo.

• The score is calibrated on it. On kinds of writing it was not tuned on:

  	1.1.0	1.2.0
  AUC (English)	0.904	0.942
  Human texts called "Likely AI"	7.7%	1.7%
  Human texts flagged at all	14.1%	3.9%
  AUC (other languages)	0.731	0.910

• 14 more languages. Spanish, French, German, Italian, Portuguese, Dutch, Polish, Russian and Japanese are supported. Arabic and Korean are experimental. Hindi, Turkish and Chinese aren't reliable yet. Every report now says which language it scored and how far the score can be trusted in it.

• Bands you can read. A score of 45 is where the top 5% of human writing begins, 55 the top 2% and 80 the top 0.5%. So "Likely AI" means fewer than 2 in 100 human texts score that high.

• Feedback after an analysis: visitors can say who wrote the text. Only the answer and the scores are kept, never the text.

Faster

• Long texts: the slowest 10% of analyses finish about 40% sooner (56 s down to 34 s at peak traffic), with the same accuracy.
• Short texts get their own lane. At twice peak traffic, the worst wait fell from 134 s to 26 s.
• Text of any length is accepted and kept whole in the report.

Fixed

• Short Thai, Japanese and Chinese texts scored near 0 because their words were undercounted.
• Cached reports could carry a score from an older calibration.
• URL scans that couldn't be fetched returned 500 instead of 502.

Thank you

First contributions from @ST-SACHIN-KUM, @saad07072, @HarshRajSinghania, @nawaaaaaAaar, @sniertough, @AnjaliKumariPrasad, @GreedyC and @SouvickSarkar20: tests, /health engine status, a Python example client, pre-commit, Bugbear, SlopBench results by length, a compose healthcheck and the configuration docs. More good first issues are open.

Run it

docker run -p 8000:8000 -v sloptotal-models:/app/models ghcr.io/pablocaeg/sloptotal:1.2.0

Full changes: CHANGELOG