Skip to content

Releases

Model round 19 (flan-t5-large)

Choose a tag to compare

@boredland boredland released this 17 Oct 17:03
c4f50e4

Flan-T5 large (783M) auf den Daten von Runde 18, als wählbares genaueres Modell neben dem Standardmodell (Runde 17). Testsätze 96,6 % statt 94,8 % wortgleich; Schilderungen 10 statt 13 von 393 Sätzen mit Fehler (im Rahmen der Bewertungsschwankung). Download 1,9 GB statt 680 MB, etwa doppelt so langsam pro Satz. Trainiert auf einer AMD Instinct MI50.

Model round 18 (flan-t5-base)

Choose a tag to compare

@boredland boredland released this 03 Oct 11:10
dca3d3f

Not deployed. Single model: test exact match 0.960, user-reported sentences 15/15, separable 40/40, subclause 23/24, held-out accounts 19/393 (live round 17: 14), Tatoeba audit 6/545. Averaging it with recent rounds gave 14/393 again — no gain over round 17 beyond grader noise, so round 17 stays live.

Model round 17 (flan-t5-base)

Choose a tag to compare

@boredland boredland released this 03 Oct 06:58
dca3d3f

Deployed. No new training: uniform weight average of rounds 13-16. Held-out asylum-hearing and police-interview accounts: 14/393 sentences flagged (round 15: 15, round 16: 21, round 7: 101; repeated grading of one model varies by about 1). User-reported sentences 15/15, separable 40/40, subclause 23/24, challenge 20/20, test exact match 0.948, Tatoeba audit 4/546. Fine-tuned google/flan-t5-base (Apache-2.0); ONNX for Transformers.js (fp32 encoder, per-channel q8 decoder, fp32 lm_head). Unpack into web/ and run npm run build.

Model round 16 (flan-t5-base)

Choose a tag to compare

@boredland boredland released this 03 Oct 04:46
021a672

Deployed, with regressions against round 15. Held-out asylum-hearing and police-interview accounts: 21/393 sentences flagged (round 15: 15). User-reported set 14/15: plural agreement now right ('Ihm hätten zwei Personen gegenübergesessen'), but the apposition '– ein sachlicher,' is dropped. Separable probes 38/40 (round 15: 40), subclause 23/24, test exact match 0.952, Tatoeba audit 5/546. Fine-tuned google/flan-t5-base (Apache-2.0); ONNX for Transformers.js. Unpack into web/ and run npm run build.

Model round 15 (flan-t5-base)

Choose a tag to compare

@boredland boredland released this 03 Oct 01:07
5a54be5

Deployed. Held-out asylum-hearing and police-interview accounts: 15/393 sentences flagged (round 14: 21, round 7: 101). Test exact match 0.951, challenge 20/20, separable 40/40, subclause 24/24, Tatoeba audit 5/547. Regression: user-reported set 14/15 — 'Ihm habe zwei Personen gegenübergesessen' (singular auxiliary with plural subject after a fronted dative); round 14 got it right. Fine-tuned google/flan-t5-base (Apache-2.0); ONNX for Transformers.js (fp32 encoder, per-channel q8 decoder, fp32 lm_head). Unpack into web/ and run npm run build.

Model round 14 (flan-t5-base)

Choose a tag to compare

@boredland boredland released this 01 Oct 23:44
89b002c

Deployed. Held-out asylum-hearing and police-interview accounts: 21/393 sentences flagged (round 13: 25, round 7: 101). Test exact match 0.954, challenge 20/20, regression 15/15, separable 40/40, subclause 22/24, Tatoeba audit 4/547. Fine-tuned google/flan-t5-base (Apache-2.0); ONNX for Transformers.js (fp32 encoder, per-channel q8 decoder, fp32 lm_head). Unpack into web/ and run npm run build.

Model round 13 (flan-t5-base)

Choose a tag to compare

@boredland boredland released this 01 Oct 20:03
e138fde

Deployed. Held-out asylum-hearing and police-interview accounts: 25/393 sentences flagged (round 12: 34, round 7: 101). Test exact match 0.947, challenge 20/20, regression 15/15, separable 40/40, subclause 23/24, Tatoeba audit 6/546. Fine-tuned google/flan-t5-base (Apache-2.0); ONNX for Transformers.js (fp32 encoder, per-channel q8 decoder, fp32 lm_head). Unpack into web/ and run npm run build.

Model round 12 (flan-t5-base)

Choose a tag to compare

@boredland boredland released this 01 Oct 16:08
c4e67d5

Deployed. Test exact match 0.953 (round 11: 0.946), challenge 20/20, regression 15/15, separable 40/40, subclause 22/24. Held-out asylum-hearing and police-interview accounts: 34/393 sentences flagged (round 11: 31, within judge noise; round 7: 101). Tatoeba audit 4/546 (round 11: 6). Fine-tuned google/flan-t5-base (Apache-2.0); ONNX for Transformers.js (fp32 encoder, per-channel q8 decoder, fp32 lm_head). Unpack into web/ and run npm run build.

Model round 11 (flan-t5-base)

Choose a tag to compare

@boredland boredland released this 01 Oct 12:21
dea614d

Deployed. Held-out asylum-hearing and police-interview accounts: 31/393 sentences flagged (round 10: 51, round 7: 101). Test exact match 0.946 (round 10: 0.951), challenge 20/20, regression 15/15, separable 40/40, subclause 24/24, Tatoeba audit 6/547 (round 10: 2/547). Fine-tuned google/flan-t5-base (Apache-2.0); ONNX for Transformers.js (fp32 encoder, per-channel q8 decoder, fp32 lm_head). Unpack into web/ and run npm run build.

Model round 10 (flan-t5-base)

Choose a tag to compare

@boredland boredland released this 01 Oct 08:31
63549a3

Deployed. Test exact match 0.951, challenge 20/20, regression 14/15 (schwer gefallen vs. schwergefallen, both Duden-valid), separable 40/40, subclause 22/24. Held-out asylum-hearing and police-interview accounts: 51/393 sentences flagged (round 7: 101). Tatoeba audit 2/547. Fine-tuned google/flan-t5-base (Apache-2.0); ONNX for Transformers.js (fp32 encoder, per-channel q8 decoder, fp32 lm_head). Unpack into web/ and run npm run build.