Grammar hygiene: variant diet and dead-rule fixes - #70
Merged
Conversation
NlSkýring, NlSkýringKomma and SvigaInnihald carried /pers, so every
apposition and parenthetical noun phrase was compiled in four persons.
The only person-dependent element inside them is the relative clause
of a parenthesis ("(sem ég er fylgjandi)"); a census of 10,000 parsed
news sentences shows 1st/2nd person winning there 3 times in total,
versus ~50 for relative clauses attached directly to the noun phrase,
which keep their person.
Make the appositional family third-person only, with the parenthetical
relative clause fixed to p3. The compiled grammar shrinks from 7,564
to 7,276 nonterminals and from 28,016 to 26,921 productions. On the
1,000-sentence baseline, 3 trees change (score ties), none lost.
Engine and GreynirCorrect test suites pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The fallback that lets an apposition or parenthesis in the wrong case
parse with an error tag ("Ég ræddi við forsætisráðherra (Sigmundur
Davíð)") was defined under the misspelt name Svigainnihald_* and thus
never referenced, although GreynirCorrect still has the handler for
its AðvörunSvigaInnihaldNl node.
Attach it as the last, lowest-priority alternative of SvigaInnihald
via a per-case helper. A separate definition block would have gotten
top priority and pruned the correct readings, since the priority
counter of a block is not shared with the generic definition.
On the 1000-sentence baseline 2 trees change, both improved (a
parenthesis "( átta )" now attaches as an apposition instead of
forcing a heading analysis). Engine and GreynirCorrect suites pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A usage census over 10,000 parsed news sentences found nonterminals
that never appeared in a winning tree. Four of them were meant to fire
on ordinary prose and lost to a wrong analysis or could not match:
- "sem:ao" does not exist in BÍN ('sem' is st or fs), so SemStendur
("sem stendur"), LoSem ("sem jöfnustu"), "svo sem" and one "sem
sagt" could never match. Use "sem:st".
- HjSögnLhNtFrumlag's first alternative ("Nú fer hælisleitendum
fækkandi") lacked the dative subject, so the sentence was parsed
with an impersonal verb instead.
- HjSögnVantaSkorta ("Okkur þótti vanta mýkt") lost on score to a
reading of 'vanta' as a noun; give it a bonus.
- "undir eins" was read as 'undir' + the comparison "eins og". Make
it a fixed phrase in Phrases.conf.
On the 1000-sentence baseline 5 trees change: 3 score ties and 2
improvements (e.g. "vel viðráðanleg sem stendur" no longer becomes a
relative clause), none lost.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
HjSögnVh matched singular subjunctive auxiliaries (hafi/gæti/mætti/ þætti/skyldi/myndi [ekki] hafa) in a single verb-initial impersonal alternative of SetningSoSagnliður. It never won a parse in a 10,000-sentence census, and all five of its canonical example sentences (including "Í dag gæti hafa rignt") parse identically via the generic auxiliary path, since so_et_p3 does not restrict mood. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
VeraInnskot ("vera [stöðugt] að") was an optional element of the
accusative-with-infinitive bottoms ÓpSagnarBotn and
MiðmyndarBotnFlókinn. It never won a parse in a 10,000-sentence
census, and probing its home contexts ("Mér sýndist kötturinn vera
að lepja mjólkina") shows the progressive reading always wins
through the generic infinitive path; all probes still parse
identically without it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
SetningAð handled infinitive-phrase subjects ("Að raða frímerkjum
er góð skemmtun") as an alternative of Setning_et_p3_hk, already
marked down with $score(-4), and SetningAðHugsaSér was reachable
only through it. Neither ever won a parse in a 10,000-sentence
census; the IP-INF-SUBJ path wins every probe with correct trees,
and all probes (including "Að hugsa sér!") parse identically
without them.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Inside SagnFramhald, the optional SagnKomma* prefix could never win:
KommaOgEðaEn already matches a bare comma, so a comma-separated verb
sequence always parses as a chain of SagnFramhald units, each earning
$score(+32), which the $score(+2) prefix path cannot beat. Removing
the prefix eliminates a spurious forest family per comma. SagnKomma
itself stays: it is the only path to three or more verbs in the
correlative alternatives (hvorki/bæði/ýmist/annaðhvort), where it
demonstrably wins ("Hann hvorki söng, dansaði né hló").
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ÁframSagt (Segir SegirÁfram Sagt) required a second ", segir X" attribution after Segjandi and allowed no colon, so it could not even match the canonical example in its own comment and never won a parse. The question alternative of Staðhæfing already uses SegirÁframSagt (SegirÁfram ":"? Sagt?), which does win; pointing the declarative alternative at the same rule makes "„Þetta er gott,“ segir Jón og bætir við: „Þetta er mjög gott.““ parse as a single attributed statement instead of being split at the continuation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Census-driven cleanup of the date family:
- Delete the Dagsetningarliður wrapper and its single use site (the
'dagsett + date' alternative of Lo_sb_nf): the participle reading
with a generic date adverbial always wins, and the children
(FöstDagsetning, AfstæðDagsetning) remain heavily used elsewhere.
- Delete Mánaðamót: the tokenizer delivers "ágúst-september" as a
single WORD token, so the rule ('mánaðamót' dagsafs '-' dagsafs)
could never match its target pattern. Handling fused month ranges
would be a tokenizer-level fix.
- Fix DaganaTímabilFast with $score(+8) so "dagana 13. október til
14. nóvember 2017" wins as one date period, and add a
raðnr-til-dagsafs alternative for the year-less
"dagana 5. til 7. ágúst".
FráTilTími stays: it wins today ("frá kl. 10 til kl. 14 á morgun").
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Census-driven cleanup of coordination/continuation helpers that never won a parse in 10,000 sentences and lose in their own home contexts under targeted probes: - OgSögn_mm_sagnb: "hafa reynst og nýst mér" wins via the generic sagnbót coordination - OgNhSögn_lhþt, OgNhSögn_expl_op: "að reistar yrðu og byggðar íbúðir" / "að rigna og snjóa" win via CP or generic paths (the OgNhSögn base family stays; it wins regularly) - HreinSögnAtv_nh: unused optional trailer in SagnliðurÁnF - AukafallÖfugSögn: unmatchable — 'gera' has no bare-dative frame, so "skal engum gert" never matched (a working replacement for the constitutional sentence follows separately) - FramhaldÁnFMeðKommu: 0 wins vs ~35 for the plain sibling despite $score(+8) - KommuSögn_bh: imperative sequences win as separate clauses; OgSögn_bh (which does win) stays The nine other census-flagged twins were probed alive and are kept. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Replaces the unmatchable AukafallÖfugSögn path deleted in the
previous commit with a working rule. Verbs.conf already carries the
'gera þgf |nh' frame ("Dómarinn gerði mér að greiða sektina"), so
"Engum skal gert að leysa nauðungarvinnu af hendi" parsed fine; what
failed was the topicalized variant in article 68 of the
constitution, "Nauðungarvinnu skal engum gert að leysa af hendi",
where the object of the embedded infinitive is fronted. A new
SetningAukafall alternative anchors on the literal 'gera:so'_gm_sagnb
with a dative NP and an infinitive whose object is the fronted
phrase.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Name the /hk, /kvk, /nf, /þf, /et, /p3 declarations for what they are — "eins gildis tilbrigði" — and explain the pinning idiom (Tilvísunarsetning/tala/p3/kyn) in the grammar comment and in CLAUDE.md, so it doesn't have to be rediscovered again. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #69, based on a usage census of the grammar over 10,000 parsed sentences (which nonterminals/productions ever appear in a winning parse). Three hygiene commits:
Drop the person variant from appositions and parentheses (
d4c4576):NlSkýring,NlSkýringKommaandSvigaInnihaldcarried a/persvariant that never influenced a winning parse; dropping it (and specializing the innerTilvísunarsetningtop3) shrinks the compiled grammar from 7,500 to 7,276 nonterminals and from 28,001 to 26,921 productions, with no change in parse results.Reach the wrong-case apposition fallback (
b321356): a typo (Svigainnihaldvs.SvigaInnihald) left the wrong-case parenthesis fallback unreachable since its introduction. The fix appends the fallback as the last priority alternative of the generic chain — a separate definition block would have (surprisingly) received top priority, since production priorities restart at 0 per definition block.Fix constructions that never won a parse (
a1277c1): six occurrences of"sem:ao"could never match (BÍN has sem only as a conjunction/preposition) — now"sem:st";HjSögnLhNtFrumlag("þeim fer fækkandi") had an unmatchable first alternative;HjSögnVantaSkorta("mig vantar X") always lost to an impersonal-verb reading and now carries a score boost; "undir eins" is now a static phrase soUndirEinscan win.Validation: full test suite passes (157 tests), GreynirCorrect suite passes (84 tests) against this branch, ruff + mypy clean. A fixed 1,000-sentence baseline of previously-parsing text shows a handful of neutral-or-better tree changes and no lost parses. New test
test_never_winning_constructionslocks in the resurrected constructions.🤖 Generated with Claude Code