Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adds relative occurrence #49

Merged
merged 32 commits into from
Oct 21, 2019
Merged

Adds relative occurrence #49

merged 32 commits into from
Oct 21, 2019

Conversation

da1nerd
Copy link
Contributor

@da1nerd da1nerd commented Oct 21, 2019 •
edited
Loading

Related to unfoldingWord/translationCore#6237 (comment)

The Alignment Position algorithm was proving to be too strong and it was producing false positives when words with multiple occurrences where aligned together. In some cases wordMAP would predict out of order word occurrences.

  • This update adds a new algorithm that balances out the alignment position by dynamically evaluating the relative occurrence between two ngrams.
  • Also improve the algorithm execution performance and fixes some bugs.

This change is Reviewable

da1nerd and others added 30 commits November 19, 2018 10:59
@da1nerd
removed old saved alignment wording
ec71d98
@da1nerd
added custom key retrieval from ngram and alignment indicies
e0b8f4a
@da1nerd
added lemma key to alignments
ef35700
@da1nerd
added lemma ngram frequency algorithm
1888a23
@da1nerd
fixed bugs in lemma fallback in ngram frequency
681f232
@da1nerd
added lemma fallback to uniqueness algorithm
ef82079
@da1nerd
adding test for uniqueness and fixing bugs in uniqueness algorithm
f04d03e
@da1nerd
added tests for uniqueness
c113734
@da1nerd
finished tests for lemma ngram frequency
ff36ce9
@da1nerd
added tests for alignment occurrences
e42ce92
@da1nerd
added tests for phrase plausibility
1e7d4c7
@da1nerd
added lemma score keys
b27c688
@da1nerd
connected last algorithm to map
b9d39a2
@da1nerd
Merge branch 'develop' into enhancement/joel/5267/lemma-fallback
ef20215
@da1nerd
added toJSON method to alignments
2fbec1f
@da1nerd
enabled passing one alignment at a time to map
c974401
@da1nerd
merged 0.4.3 into master
c8840aa
@da1nerd
added lock files
cdc21f8
@da1nerd
Merge tag 'v0.4.3' into merging
286afd2
@da1nerd
fixed tests
1286382
@da1nerd
stubbed out new algorithm
2f72414
@PhotoNomad0
removed warnings
876b2b3
@PhotoNomad0
added option to enable nmap warnings in log.
507cce6
@PhotoNomad0
updated unit tests to pass on Travis.
73a4090
@PhotoNomad0
documentation and cleanup.
3627bee
@mannycolon
Merge pull request #48 from translationCoreApps/feature-mcleanb-3579
3b59dc4 
Feature 3579/Optimization
@da1nerd
added test for relative occurrence
d2943a3
@da1nerd
updated lexer dependency
3b1f7ed
@da1nerd
added algorithm to throttle the alignment position algorithm
9b3ddf8
@da1nerd
refactored algorithm registration
3a72619
@da1nerd da1nerd merged commit c9ba591 into master Oct 21, 2019
@delete-merged-branch delete-merged-branch bot deleted the enhancement/joel/relative-occurrence branch October 21, 2019 09:14
@da1nerd da1nerd mentioned this pull request Oct 23, 2019
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@da1nerd @PhotoNomad0 @mannycolon