OPENNLP-547: Add a dependency parser component - #1236
Open
krickert wants to merge 18 commits into
Open
Conversation
krickert
added a commit
to ai-pipestream/opennlp
that referenced
this pull request
Aug 22, 2026
krickert
added a commit
that referenced
this pull request
Aug 26, 2026
…C helper base # Conflicts: # opennlp-docs/src/docbkx/dependency.xml
krickert
added a commit
that referenced
this pull request
Aug 26, 2026
…C helper base # Conflicts: # opennlp-docs/src/docbkx/dependency.xml
krickert
added a commit
that referenced
this pull request
Aug 30, 2026
krickert
added a commit
that referenced
this pull request
Sep 1, 2026
Contributor
|
@krickert Can you rebase this one onto main? Currently changes show up here that are now already included in main. |
… and UAS/LAS evaluator Adds opennlp.tools.depparse: the DependencyParser interface with DependencyArc, DependencyGraph, and DependencySample in opennlp-api, and a greedy transition-based implementation in the runtime: arc-standard state and static oracle, configuration features, an oracle-driven event stream, DependencyParserME over the existing maxent machinery, and a UAS/LAS evaluator. ConlluDependencySampleStream maps the basic dependency columns of CoNLL-U sentences so Universal Dependencies treebanks train directly; sentences whose multiword tokens were merged by ConlluStream are skipped because their syntactic words are not recoverable. Non-projective trees have no arc-standard derivation and are skipped during event generation. Model persistence and a DL-backed implementation are follow-ups behind the same interface. (cherry picked from commit 7448e34)
…ies evaluation Adds DependencyModel on the standard BaseModel machinery, so trained parsers serialize and load like every other tool model; the declared serialVersionUID is the serialver-computed value. Training now returns the model with the trainer's manifest entries, and DependencyParserME gains a model constructor beside the raw one. The round-trip test serializes, reloads, and re-parses. ConlluDependencyParserEvalTest trains on a Universal Dependencies treebank and scores UAS and LAS on its test split; it runs only when opennlp.depparse.ud.dir names the downloaded splits, keeps treebank data out of the repository, and asserts only a low regression floor, with the logged scores as the measurement. (cherry picked from commit c5a3b94)
…pendencies Rewrites ConlluDependencySampleStream to parse CoNLL-U content directly instead of consuming the merged ConlluSentence view: ConlluStream merges multiword token ranges with their syntactic words, which suits the token and lemma views but destroys the dependency annotation of every contraction-bearing sentence. The raw reader drops range lines and empty nodes while keeping the syntactic words, so those sentences train and evaluate normally; on the English EWT treebank this recovers 2192 training and 302 test sentences that were previously skipped, and evaluation now covers the full test set. (cherry picked from commit f22b0ac)
Extends the configuration features with the partial structure built so far: tags and relations of the leftmost and rightmost dependents of the top two stack tokens, their conjunctions with the neighboring tags, word and tag pairs in both directions, a second-order stack tag triple, buffer word context, and a distance-tag conjunction. ArcStandardState now tracks leftmost and rightmost dependents and exposes the assigned relation of an attached token. Measured together with the raw CoNLL-U reader on UD English EWT with gold UPOS tags: UAS 0.8259 and LAS 0.7929 over the full 25094-token test set, up from 0.7791 and 0.7103 over the 19394-token subset the merged reader could parse; training takes 91 seconds. (cherry picked from commit 6cf87b8)
…n the JVM Adds the neural transition parser as plain array arithmetic with no native runtime: FeedforwardDependencyModel holds embeddings for words, tags, and arc labels, one cube-activation hidden layer, and a transition output layer in a versioned binary format of our own; FeedforwardDependencyParser decodes greedily with applicability masking; and FeedforwardDependencyTrainer trains the whole network in Java with minibatch AdaGrad over softmax cross-entropy, inverted dropout, a learned unknown word embedding, and a fixed seed for reproducibility. The feature template embeds fourteen configuration positions with second-order children, words and tags for all, labels for the dependent positions. First run on UD English EWT with gold UPOS tags and default settings: UAS 0.8585 and LAS 0.8351 over the full 25094-token test set, above the best classical result (0.8579 and 0.8316 from quasi-Newton training) with 13 minutes of training against 38, parsing at roughly 3.5k tokens per second single-threaded. Untapped levers: pretrained embedding initialization from the static embedding tables, capacity and schedule tuning, and beam search. (cherry picked from commit 7425d11)
Adds a training overload taking a pretrained vector provider: vocabulary words the provider knows start from their pretrained vectors instead of random noise, stay trainable, and everything the provider does not know keeps the random initialization. The provider is a training-time ingredient only, since the learned embeddings ship inside the model, so parsing carries no dependency on the embedding source. This is the seam that lets the static embedding tables feed the parser. (cherry picked from commit 0084cf3)
The decoder keeps the best transition sequences side by side, scored by summed log-probabilities, so one locally attractive but globally wrong transition no longer commits the whole parse. Beam size one keeps the exact greedy fast path; every arc-standard derivation has the same length, so summed scores compare without normalization. (cherry picked from commit 2c30099)
After local training, sentences are re-decoded with a beam while the gold derivation is tracked through it; the moment gold falls out an early update pushes the model toward keeping it, under a conditional likelihood over the beam's candidate paths. Paths are scored exactly as the beamed parser scores them, so training optimizes the quantity decoding uses. Refinement mutates the model in place, deterministic for a fixed seed. (cherry picked from commit bd663ba)
…iner and parser javadoc
… a download helper
…and read blanks and case through the project seams
…d make model bytes reproducible Routing normalization through the plain per-code-point mapping broke case-insensitive vocabulary matching for Greek: treebank-derived keys spell a word-final sigma as U+03C2, and an uppercase surface form mapped per code point ends in the medial U+03C3 instead, missing the vocabulary. Normalization now applies the Final_Sigma condition of the Unicode SpecialCasing file, restricted to a single token, on top of the per-code-point mapping, and returns already-lowercase words unchanged without allocating, which is the common case on the parse hot path. Serialized vocabularies are written in ascending id order because the iteration order of the immutable maps is salted per JVM launch, so serializing the same model now produces the same bytes on every run. Transition and dependency-graph relation labels judge blankness under the project whitespace definition, and the copy javadoc no longer calls the cloned transition array immutable.
… models The scorer re-derived every feature's hidden-layer contribution from its embedding on every configuration, although for a frozen model the contribution of a (template position, embedding row) pair is a fixed vector. Parsers now turn on a bounded lazy cache that computes each pair's vector once on first sight and adds it thereafter, the adaptive form of the precomputation described for this architecture by Chen and Manning (2014): tag and label rows are fully cached within a document or two and word rows follow their frequency. Measured on realistic dimensions (20k words, embedding 50, hidden 400, 77 transitions): 5,462 to 71,954 scored states per second, 13.2x. Training and refinement work on uncached copies, copies never carry a cache, concurrent readers are safe by idempotent fill, and a test pins cached-versus-direct agreement to float rounding with identical winning transitions.
Replaces the two private blank helpers with the shared predicate; behavior is identical since both already followed the toolkit whitespace definition.
…ed example Add docbkx/dependency.xml, wire it into the manual, and cite ConlluDependencyParserUsageTest.
…e constants - DependencyParserME decodes the model outcome inventory once in the constructor and keeps it as a Transition[]. Decoding a sentence now indexes that array instead of parsing an outcome string per configuration and per outcome, and a model trained for another task is rejected with IllegalArgumentException when the parser is built rather than surfacing as an IllegalStateException in the middle of a sentence. Both constructors document the new failure, and a pinning test builds a parser over a MaxentModel whose outcomes are POS tags to prove the rejection happens up front. - Removed the private isBlank copy from ConlluDependencySampleStream and routed sentence separation through StringUtil.isBlank, so the toolkit whitespace definition lives in one place. The rationale the copy carried moved into the nextSentence javadoc, which also gained its missing @return and @throws. DependencyArc validates its relation through the same predicate, so an arc label made only of a no-break space is rejected exactly as a CoNLL-U separator line is. - StringUtil.isBlank rejects a null argument with IllegalArgumentException instead of letting a NullPointerException escape the loop, and documents it. Its test became a parameterized case list plus an explicit null case. - Extracted the magic values of DependencyContextGenerator into named constants: the word/tag and position separators, the feature count the list is sized to, the valency and distance bounds, and the long-distance feature value. The distance feature is computed once rather than twice. - Named the *ROOT* vocabulary key ROOT_SYMBOL and the "*" special-symbol prefix SPECIAL_SYMBOL_PREFIX on FeedforwardDependencyModel and used them from FeedforwardContext and FeedforwardDependencyTrainer, which spelled all three as literals. FeedforwardContext names the first dependent position instead of indexing the template at a bare 6. - Folded the three repeated special-symbol loops in the trainer vocabulary builder into addSpecialSymbols, and hoisted the repeated model.transitions() call out of the transition-decoding loop. - FeedforwardDependencyModel.score and featureIds validate their array argument, and lookup fails loudly when a vocabulary carries no *UNK* row to fall back on instead of returning null and unboxing to a NullPointerException later. - Documented the package-private accessors of FeedforwardDependencyModel, saying which of them hand out the live arrays the trainer writes into. - Trimmed commentary to what the code does: enableScoringCache drops the literature reference and the restatement of how caching pays off, and DependencyEvaluator.processSample uses {@inheritdoc} plus only what the override adds over the Evaluator contract. - Replaced fully qualified java.util.Arrays, java.util.function.Function, java.io.ByteArrayInputStream, java.io.ByteArrayOutputStream and MaxentModel uses with imports in the trainer, the arc-standard state and the tests, and dropped stray blank lines in DependencyGraph and Transition. - Moved the sample() and corpus() helpers, copied verbatim in three test classes, into a shared DependencyTestSamples fixture with the repetition count as a named constant. - Added pinning tests: a relation of U+00A0 alone is rejected while a label such as nmod:poss is kept, an empty DependencySample is rejected, and the arc and graph relation accessors return what was passed in.
Add reference links for CoNLL-U, the arc-standard system (Nivre 2004), the feedforward architecture and training recipe (Chen and Manning 2014), early update (Collins and Roark 2004), and the Unicode SpecialCasing file. State that thread safety is implementation specific on the DependencyParser interface. Add javadoc to the remaining private helpers in main and test code. Fold the duplicated corpus reading in the feedforward trainer into a readAll helper, and read the corrupt-model test fixture as UTF-8. The DependencyModel serialVersionUID was verified to equal the serialver default.
krickert
force-pushed
the
OPENNLP-547-dependency-parser
branch
from
September 1, 2026 07:54
71da201 to
83bc298
Compare
krickert
marked this pull request as ready for review
September 2, 2026 00:39
krickert
added a commit
that referenced
this pull request
Sep 4, 2026
# Conflicts: # opennlp-docs/src/docbkx/dependency.xml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stack
This draft is temporarily based on OPENNLP-1888-DocumentShape so the later Document annotation PRs form a clean review stack. The dependency parser itself does not require the Document API. After #1182 merges, this PR can target main.
Verification
The top of the stack passes the relevant Maven reactor with tests, checkstyle, forbidden APIs, Javadocs, documentation generation, and license checks.