Grammar coverage: verb frames, particles, clitics, articles, quotes, raising - #69
Merged
Conversation
Verbs.conf frames with a reflexive pronoun as an object or inside a
prepositional phrase ('átta |sig /á þgf', 'tryggja sér |þf') were
classified as 'complex' and silently skipped when loading, so verbs
whose only frames are reflexive (átta, ímynda, furða, notfæra,
trúlofa, ...) were unknown to the parser and could not take objects
at all, and frames such as 'tryggja sér |þf' never licensed
'tryggja e-m e-ð'. 89 verbs and 227 verb+case keys were affected.
Such frames are now registered under the pronoun's case and marked
as weak. The reducer penalizes a verb terminal whose cases are only
licensed by weak frames, so that e.g. 'á' in 'á pí-deginum' remains
a preposition, and verb_score() ignores the $score() pragmas of weak
frames when strong frames exist for the same key. Frames with fixed
phrases are still skipped; registering them licensed far too much
('eiga |þgf /til góða' -> generic 'eiga' + dative).
The 'átta so < töl' preference is strengthened so that the numeral
reading still wins in 'þrjátíu og átta fermetra'.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The grammar had no construction for a particle preceding the object: 'bjó til hús', 'kastaði upp boltanum', 'skilaði inn skýrslunni' and 'sagði upp störfum' failed to parse, while 'tók upp bókina' only parsed by misanalysing 'upp bókina' as a prepositional phrase. A new nonterminal SagnÖgn (tagged verb_particle) may now appear between the verb and its object in Sögn_1, Sögn_1_sagnb, Sögn_1_nh, Sögn_1_bh and NhSögn, all tagged apply_particle_bonus. The reducer notes the particle text at the SagnÖgn node, carries it up to the enclosing verb phrase and adjusts that family's score: a bonus if Verbs.conf lists the particle for the verb with these argument cases (*til, *upp, ...), using VerbFrame.matches_particle(), which had no caller before, and otherwise a penalty that outweighs the preposition penalty, so that 'kastaði upp á þakið' keeps its PP reading. 'búa |þf *til' is added to Verbs.conf; it was missing altogether. The expected tree in test_rel_clause_pp_attachment now has 'í umdæmi' in the dative, which is the correct reading. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Equally scored families of children were resolved by family index,
i.e. by the order in which the Earley parser completed the derivations
(the C++ forest prepends families, so the last completed derivation
won). That order shifts with unrelated grammar edits, so every change
to a common production flipped a large number of tied parses.
Ties are now broken by a content-based key: the production that
appears first in the grammar wins, and between derivations of the
same production, the one with the longest leading children wins.
Terminal/nonterminal indices are assigned by sorted name and the
production index by grammar position, so the outcome only changes
when an alternative is inserted between two competing alternatives.
Accordingly, give "í" + dative a small bonus over "í" + accusative,
so that locative readings win when the noun form is the same in
both cases ("í umdæmi", "í Reykjavík", "í höfn"); directional
readings governed by verb frames are unaffected.
Measured on 1000 parsed news sentences: 313 trees change once, all
score ties except 84 that improve via the í + þgf bonus; no
measurable speed difference. Engine and GreynirCorrect test suites
pass; PYTHONHASHSEED was verified not to affect results.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
BÍN lists interrogative verb forms with a cliticized subject
('spurnarmyndir': ertu, viltu, geturðu, veistu, komstu) tagged with
SP, and binparser already reserved a restrictive 'sp' variant for
them, but the grammar never used it, so sentences such as "Ertu
búinn?", "Viltu koma?" or "Hvernig komstu í kynni við hana?" failed
to parse (or, for 'ertu', parsed as an imperative of 'erta').
Add 'sp' as a fourth value of the /pers variant. The existing verb
phrase productions then yield Sagnliður_et_sp whose finite verb
terminals (so_..._et_sp) only match SP forms. A new SagnHluti_et_p2
alternative, SagnliðurSníkill, uses it as a verb-first clause with
an included subject, which covers questions, fronted adverbials
("Þá viltu það ekki") and the other verb-first contexts. Noun
phrases of the 'sp' person map to those of the second person.
Measured on the unparsed-sentence corpus: 802 of 1017 sentences
containing such forms now parse. On 1000 previously parsed news
sentences, 5 trees change (all containing clitic forms, all
improved). The grammar grows by ~10% (nonterminals 6808 -> 7513)
and parsing is ~5% slower. Engine and GreynirCorrect test suites pass.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
BÍN lists interrogative clitic forms (SP) for practically every verb,
including obscure ones whose form coincides with a common adjective
('næstu' = ná + þú, 'mestu' = meta + þú, 'elstu' = ala + þú) or with
a regular verb form ('áttu', 'yrðu', 'máttu' as 3rd person plural).
On 1500 parsed news sentences containing any such token, about 30
switched to a wrong clitic reading after the previous commit.
Penalize an sp terminal in the reducer when the token also has an
adjective reading (-40) or a non-clitic verb reading (-6), and give
SagnliðurSníkill a small negative score so that a regular analysis
wins whenever one is available. This brings the wrong readings down
to a handful, all in sentences whose previous parse was poor, while
genuine clitic questions ('Ertu ...?', 'Áttu von á ...?', 'Hvenær
áttu að eiga?') keep their new analysis. The cluster recovery
(802/1017) and the 1k baseline (5 improved trees) are unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…inbera)
The free article 'hinn/hin/hið' followed by an adjective and no noun
('hið sama', 'hið rétta', 'hið opinbera', 'hið versta', 'hið góða og
hið illa') had no analysis unless the article form happened to have
a pronoun reading in BÍN (as 'hinu', 'hins' do), so most such
sentences failed to parse.
Add NlGreinirLo (Greinir + LoLiður) as a NlStak_p3 alternative, and
HiðFyrsta, an adverbial phrase for the fixed superlative expressions
'hið fyrsta', 'hið snarasta', 'hið bráðasta', 'hið minnsta', 'hið
mesta' (with optional 'allra'), so that they don't become verb objects.
Also let the adjective reading of 'sama' survive the Phrases.conf
disambiguation of 'sama gildir/gilti/gildi', which otherwise blocks
'Hið sama gildir um ...'.
Measured on the unparsed-sentence corpus: 317 of 517 sentences with
such phrases now parse. On 1000 previously parsed news sentences,
3 trees change, all now using the article rather than the pronoun.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Direct speech ending in a question mark or exclamation mark is not
followed by a comma before the attribution ("Komst hann ekki í
hópinn?" sagði hann / "Við erum að falla á tíma!" skrifar Limbourg),
but Staðhæfing required one, so such sentences did not parse.
Add SagtSpurning (a question clause, or a sentence ending in "!") and
SegirÁnKommu (attribution with optional comma) as a second Staðhæfing
alternative, with an optional "og bætti við: ..." continuation. Add
the attribution verbs spyrja, svara, hrópa, kalla, tísta, hugsa and
útskýra, "bætir X við" as an attribution on its own, and gesture
continuations ("segir KK og hlær", "spurði hún og kinkaði kolli").
A proper noun as the attributed source (Heimild) is now third person
singular only, instead of getting an arbitrary person by tie-break.
Measured on the unparsed-sentence corpus: 241 of 448 sentences with a
quoted question/exclamation and attribution now parse. On 1000
previously parsed news sentences, 5 trees change (proper-noun
attributions now p3; one statement now analyzed as a quote), 0 lost.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The static phrase "þess í stað" was commented out in Phrases.conf, so
the three tokens were analyzed separately ('þess' as a genitive
object, 'í stað' as a prepositional phrase), which rarely fits the
sentence; sentences such as "Þess í stað fór hann heim" or "Hann fór
þess í stað að læra" did not parse. Enable the phrase, which then
matches an 'ao' terminal like the other fixed adverbials.
Measured on the unparsed-sentence corpus: 171 of 250 sentences with
'þess í stað' now parse. No change on 1000 previously parsed
news sentences.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The past participles sagður, talinn, álitinn and látinn followed by an
infinitive ("Maðurinn er sagður hafa stolið bílnum", "Hún er talin
vera í haldi") only had an analysis for "sögð vera + predicate", so
the common news pattern "er sagður hafa + supine" failed to parse
("sagður vera í haldi" happened to parse with 'vera' as a noun).
Add three SögnErLoBotn alternatives after SögnLhÞtVera: 'hafa' + a
supine verb phrase, a plain infinitive verb phrase, and vera/hafa
verið + a past participle, each with an optional adverb ("var ekki
talinn hafa misnotað aðstöðu sína").
Measured on the unparsed-sentence corpus: 241 of 369 sentences with
such phrases now parse. On 1000 previously parsed news sentences,
6 trees change (3 ties, 3 improved: "voru talin skekkja ímynd",
"var látinn vinna", "látin vita"), 0 lost.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
vthorsteinsson
commented
Aug 28, 2026
vthorsteinsson
left a comment
vthorsteinsson
left a comment
Member
Author
There was a problem hiding this comment.
A few questions/comments to consider
| SegirÁfram ":"? Sagt? | ||
|
|
||
| SagtSpurning → | ||
| Forskeyti? Upphrópun* Spurnarsetning |
Member
Author
There was a problem hiding this comment.
Might be better to include the Forskeyti? prefix in the parent production, instead of having it in both options of this production
Member
Author
There was a problem hiding this comment.
Done in b58f67b: Forskeyti? and the optional comma now sit on the Staðhæfing alternative, and Segir → "," SegirKjarni shares the attribution core with the question/exclamation case. 0 diffs on the 1k baseline; test suites green.
| | Forskeyti? Yfirsetning StLiður* "!" | ||
|
|
||
| SegirÁnKommu → | ||
| ","? SögnSegja/tala/pers Heimild_nf/tala/pers AtvFsAuka? |
Member
Author
There was a problem hiding this comment.
Might be better to include the ","? option in the parent production, instead of having it in both options of this production
Member
Author
There was a problem hiding this comment.
Done in b58f67b: Forskeyti? and the optional comma now sit on the Staðhæfing alternative, and Segir → "," SegirKjarni shares the attribution core with the question/exclamation case. 0 diffs on the 1k baseline; test suites green.
| # , segir/fullyrðir Jón Ásmundsson, fulltrúi samtakanna, í dag í útvarpinu | ||
| "," SögnSegja/tala/pers Heimild_nf/tala/pers AtvFsAuka? | ||
| # , bætir Jón við | ||
| | "," SögnBætaVið/tala/pers |
Member
Author
There was a problem hiding this comment.
Potentially include the "," in the parent production, instead of both options of Segir
Member
Author
There was a problem hiding this comment.
Done in b58f67b: Forskeyti? and the optional comma now sit on the Staðhæfing alternative, and Segir → "," SegirKjarni shares the attribution core with the question/exclamation case. 0 diffs on the 1k baseline; test suites green.
Per review: move Forskeyti? and the optional comma up into the Staðhæfing alternative, and share the attribution core (SegirKjarni) between the comma-prefixed Segir and the question/exclamation case, instead of repeating the prefixes in each alternative. No change in parse results on the 1000-sentence baseline. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Coverage improvements driven by an analysis of ~85k sentences that failed to parse in the Greynir scraper database (2026-08). Each change targets one construction cluster, and each is validated the same way: parse-failure recovery on the cluster, a tree/score diff on a fixed set of 1,000 previously parsed news sentences (0 sentences lost by any change), and the engine + GreynirCorrect test suites.
sig/sér/sín) instead of silently dropping them — 227 verb-frame keys were missing (átta_þf,tryggja_þgf_þf, …)SagnÖgn+ reducer particle bonus from Verbs.conf*tilí+þgf preferencespas a fourth/persvalue; penalties for BÍN's clitic homographs (næstu, mestu, áttu)hið fyrsta/snarastaadverbial; Phrases.confsama gildirno longer drops the adjective readingAlso verified along the way:
PYTHONHASHSEEDhas no effect on parse results; the tie-break change costs no measurable parse time; the clitic grammar extension costs ~5% parse time (grammar +10%).Notes for review
reducer.py: the tie-break key and the clitic penalties are the only non-grammar logic changes besides the verb-frame registration inverbframe.py.Greynir.grammar:/pers = p1 p2 p3 sp— see the discussion in a339bd5; the alternative (/perssp) was evaluated and rejected as saving ~2% grammar size for a large mechanical edit.test_parse(sentence 32 now includesvera,segjain its verb list — the raising analysis) and one intest_rel_clause_pp_attachment(relative clause attaches to the nearest noun,í umdæmidative by preference rather than luck)./persdiet on noun-phrase nonterminals.Test plan
uv run pytest— 157 passeduv run ruff check src/reynir,uv run mypy src/reynir— clean🤖 Generated with Claude Code