Word-triple context tables

Per-language tables of three-word phrases, used to suggest the next word from the two words before the cursor instead of one (issue #334).

Format w1 w2 w3<TAB>count, lowercased, UTF-8, sorted by key — same shape and the same reason as the word-pair tables: the app binary-searches a flat blob, so key order means a linear load rather than sorting 100 000 keys per language. Read by frequency with sort -t$'\t' -k2 -rn <file>.

Why a separate tier rather than simply larger pair tables. Measured on held-out text: after a function word — "the", "to", "of", where 46 % of all predictions are asked for — five times as many pairs are worth exactly nothing (20.4 % → 20.4 % top-3 in English, 15.9 % → 15.9 % in German). Only a second word of context adds anything there. The converse holds too: after a rare word these triples are worth almost nothing and only more pairs help. They are complements, so both shipped together.

100 000 entries because the gain saturates early: 50 000 already buys 4.0 of the 5.3 points available after a function word, 100 000 buys 4.6, and the last 0.7 would cost two and a half times the bytes.

Sizes run 1.8–6.6 MB, and that spread is script, not vocabulary: every table holds the same number of entries, but a Georgian or Tamil character costs three UTF-8 bytes where a Latin one costs one. Tamil's is the one short table (66 149 rows) — its corpus is Wikipedia and the language is agglutinative, so fewer triples clear the minimum count of three.

Source Leipzig Corpora Collection (https://wortschatz-leipzig.de), CC BY — © Universität Leipzig / Sächsische Akademie der Wissenschaften / InfAI. Please cite D. Goldhahn, T. Eckart & U. Quasthoff (2012), "Building Large Monolingual Dictionaries at the Leipzig Corpora Collection: From 100 to 200 Languages", LREC 2012. Counted from each package's *-sentences.txt by tools/glide-dict/generate_ngrams.py; the package per language is recorded in tools/glide-dict/generate_ngrams_all.py.

English is not here — it ships inside the app, so the English keyboard is complete on install.