v0.24.0 — Telugu, Thai line breaking, French spacing, and a run of shaping fixes

A minor release: a new script, French typography, and a run of shaping and line-breaking fixes — most of
them found by diffing texish against HarfBuzz rather than by anyone reporting them.

Telugu

texish now sets Telugu (\font telugu, with a bold cut). Telugu builds downward: a consonant joined by
the virama is drawn as a subscript beneath the base, which inverts the base — a conjunct's base is its
first consonant, not its last — and a consonant with its vowel sign commonly fuses into a single drawn
glyph. A vowel sign belongs to the base, so it is moved back across the subscripts to reach it: ప్రేమ
joins its vowel to the pa, not to the subjoined ra. The ai sign is split into the two parts the font
draws, and a word-initial ra with a virama forms no reph.

Thirty everyday Telugu words shape glyph for glyph as hb-shape does. Noto Serif Telugu is bundled;
there is a guide page and a demo.

Thai line breaking

Thai is written without spaces between words, so a run of it reached the line breaker as one box and
overflowed the measure. The breaker now offers somewhere to break between orthographic clusters, never
parting a vowel or tone mark from its consonant nor stranding a pre-posed vowel from the consonant that
follows it. Proper Thai breaks fall at word boundaries and finding those needs a dictionary, which
texish does not carry, so a cluster break may land inside a word — the limitation is documented.

French spacing

French sets a space before the colon, semicolon, exclamation and question marks, and inside its
guillemets. texish set an ordinary interword space there, which is wrong twice over: it stretches when
the line is justified, so the mark drifts from the word it belongs to, and it offers the breaker a place
to break, so the mark is carried alone to the head of the next line.

Declaring the document French — \usehyphenation{fr}, or any regional tag built on it — now spaces
these marks the way French does: a full space before the colon, half that before the rest, neither
stretching nor breaking. Type the space or leave it out as you prefer, « Oui ! » and «Oui!» set
alike. The comma and full stop, which French sets tight as English does, are untouched, as is every
other language.

Breaking change

Typesetter.hyphenationLanguage is now Typesetter.language. It decides both the hyphenation
patterns the line breaker uses and the typography a language sets its punctuation by, so naming it after
hyphenation alone had stopped being true. No document syntax changes: \usehyphenation,
\loadhyphenation and \language are as they were.

Fixed: hyphenation patterns broke French, Spanish, Italian and Portuguese

A TeX comment runs from its % to the end of the line. The pattern-file reader dropped only the
whitespace-separated tokens that began with one, so the prose of a comment was read as patterns — the
French file heads its table with % phonetic patterns % etymological patterns % — and a brace inside a
comment ended the pattern block early, losing every pattern after it.

The English file is almost uncommented and so came out right, which is exactly why this went unseen.
Every other bundled language is heavily commented and broke words where no break belongs: French set
fr-ançais, pr-o-phète, par-ole, dif-fé-r-ent, ré-surr-ec-tion, none of them possible breaks,
and lost others entirely. All four now break their words where the language breaks them; English is
unchanged.

Fixed: whole GSUB lookups were being skipped

Two kinds, both silent, and both wider than the script that exposed them.

Context formats 1 and 2 were never read. Only the coverage-based format 3 was, so the rule- and
class-based lookups parsed to nothing and were passed over — and Indic fonts write a great deal of their
behaviour that way. Devanagari विद्या shaped to six glyphs where HarfBuzz gives four, the द्य never
forming.

A contextual lookup named by another contextual lookup was not followed. Noto Serif Telugu reaches
its subjoined ra through two levels of chaining context, so క్ర kept the generic subscript instead of
the variant drawn under a ka.

Fixed: vocalized Arabic stacked the shadda and its vowel the wrong way up

A letter carrying both is stored with the vowel first — that is canonical order, the vowel's combining
class being the lower — but is drawn the other way round, the shadda on the letter and the vowel above
it. The marks of a syllable are now put into drawing order before shaping, following Unicode's rules for
Arabic mark rendering, so both ways of typing a shadda and its vowel converge. In Amiri the wrong order
had also cost the word for God its calligraphic ligature.

Fixed: pointed Hebrew set the dagesh beside its letter, not inside it

A face cut for pointed Hebrew draws a letter and the point inside it as one glyph and gives no anchor for
placing that point alone. Hebrew ran no substitution at all, so in the bundled Ezra SIL — the scholarly
face for Biblical Hebrew, whose one job this is — the dagesh landed beside its letter. The font's own
composition feature now runs, with the points first put into drawing order so the rule can match, and
whether a pair combines is left to the font: Ezra combines a letter with its dagesh but leaves an alef
and its patah to be placed by anchor, and Noto Serif Hebrew combines nothing at all. Both are set
correctly without any choice on your part.

Fixed: line breaking

A tie could still be broken. The greedy setter — the fallback used when Knuth-Plass finds no solution
within the tolerance — treated every glue as a place to break, ignoring the flag that says otherwise.

A hyphenated word could be given a second hyphen. The optimal breaker hyphenated a word that already
carried one, and a break offered beside it ended the line with the word's own hyphen and the one added
for the break: Croyez--. The greedy setter never had the fault, breaking such words only at their own
hyphens; the two now agree.

A spacing mark lost its advance. Marks were assumed weightless, but Indic fonts carry spacing
subjoined forms that the font classes as marks, so syllables overlapped. The pen now moves by a glyph's
own advance unless the glyph attached to another.

Demos and docs

New: a Telugu guide page and demo, a Telugu section in the command reference, and an Ezra SIL section in
the Hebrew demo — which had only ever exercised the Noto face, and so never showed the fault above.