Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support spaCy v2.2 #432

Merged
merged 21 commits into from
Jun 7, 2020
Merged

Support spaCy v2.2 #432

merged 21 commits into from
Jun 7, 2020

Conversation

HiromuHota
Copy link
Contributor

This will close #384.

This PR

  • Expand the version requirement for spaCy to include v2.2.
  • Adapt the unit tests to the language model change from v2.1 to v2.2
  • Remove some checks from e2e that are checked elsewhere (ie test_candidates.py and test_parser.py)
  • Switch from mecab-python3 to fugashi as an extra dependency for Japanese as spaCy did as of v2.2.4.

@HiromuHota HiromuHota marked this pull request as ready for review June 2, 2020 20:14
@HiromuHota HiromuHota marked this pull request as draft June 2, 2020 20:17
@codecov-commenter
Copy link

codecov-commenter commented Jun 2, 2020 •
edited
Loading

Codecov Report

Merging #432 into master will not change coverage.
The diff coverage is n/a.

Impacted file tree graph

@@           Coverage Diff           @@
##           master     #432   +/-   ##
=======================================
  Coverage   83.17%   83.17%           
=======================================
  Files          88       88           
  Lines        4548     4548           
  Branches      837      837           
=======================================
  Hits         3783     3783           
  Misses        572      572           
  Partials      193      193
Flag Coverage Δ
#unittests 83.17% <ø> (ø)
Impacted Files Coverage Δ
setup.py 0.00% <ø> (ø)

Hiromu Hota added 19 commits June 4, 2020 08:34
Support spaCy v2.2
bfb1704
Choose a sentence whose words get "stable" NER tags
95fa37f 
"Stable" tags means they are unlikely to change even when a lang model changes
Adapt the number of parts to the fix introduced at spaCy v2.2
c027d8b 
Reported by explosion/spaCy#2926
BC546/D is tokenized as ["BC546/D"] as of spaCy v2.0&v2.1 and ["BC546"], "/", "D"] as of spaCy v2.2
NER like Person also changes as the lang model changes
f040c13
Stop checking # of parsed sentences, tables, etc. that are checked in…
1a53def 
… test_parser.py
Stop checking # of extracted mentions and candidates that are checked…
bb57856 
… in test_candidates.py
Adapt # of FeatureKey/LabelKey/GoldLabelKey to spaCy v2.2
a55bb57
Drop a feature key that is used even if spaCy v2.2 is used
dbf0a76
Adapt the german test to the spaCy v2.2 german model
7ac7d68
spacy switched to fugashi at v2.2.4 as a backend Japanese tokenizer
c756aeb
Update CHANGELOG
7c8dfd5
No need to install Mecab as it comes with fugashi
1d546bc
Adapt test_incremental.py to spaCy v2.2
0eab4db
Update CHANGELOG
d4b1428
fugashi 0.2.2+ comes with wheel for Mac
517ffdc
fugashi for macOS still requires mecab
ac64a01
Update FAQ how to install fonduer[spacy_ja]
2ce035e
Revert "fugashi for macOS still requires mecab"
550c0d6 
This reverts commit 7f62212.
Wheel for mac as of fugashi>=0.2.3 bundles Mecab
02f35e4
@HiromuHota HiromuHota marked this pull request as ready for review June 4, 2020 17:33
lukehsiao
lukehsiao reviewed Jun 4, 2020
View reviewed changes
tests/e2e/test_e2e.py Show resolved Hide resolved
tests/e2e/test_e2e.py Show resolved Hide resolved
lukehsiao
lukehsiao approved these changes Jun 7, 2020
View reviewed changes
Copy link
Contributor

@lukehsiao lukehsiao left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@lukehsiao lukehsiao merged commit a893e31 into HazyResearch:master Jun 7, 2020
@lukehsiao lukehsiao added this to the v0.8.3 milestone Jun 7, 2020
@lukehsiao lukehsiao added the clean-up Cleaning up the code or refactoring label Jun 7, 2020
@HiromuHota HiromuHota deleted the fix/384 branch June 11, 2020 15:57
wajdikhattel added a commit to wajdikhattel/fonduer that referenced this pull request Jun 26, 2020
@wajdikhattel
build(spacy): add support for spaCy v2.2 (HazyResearch#432)
aaea81e
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@lukehsiao lukehsiao lukehsiao approved these changes

Assignees
No one assigned
Labels
clean-up Cleaning up the code or refactoring
Projects
None yet
Milestone
v0.8.3
Development

Successfully merging this pull request may close these issues.

Upgrade spaCy to v2.2.4 or later for easier installation of Japanese backend
3 participants
@HiromuHota @codecov-commenter @lukehsiao