✨ Add regen wikification

[GabrielePicco]

Add easy Wikification for the Regen model

Status	Type	⚠️ Core Change	Issue
Hold	Feature	No	#11

Description

• Added a pre-computed prefix trie for wikipedia in the HG hub: https://huggingface.co/ibm/regen-disambiguation/blob/main/wikipedia_trie.pkl
• Add utils method to easy load the trie and generate wikipedia links
• Added option to use precomputed trie

Examples

import spacy

from zshot import PipelineConfig
from zshot.linker.linker_regen.linker_regen import LinkerRegen
from zshot.linker.linker_regen.utils import load_wikipedia_trie, spans_to_wikipedia
from zshot.mentions_extractor import MentionsExtractorSpacy
from zshot.tests.config import EX_DOCS

nlp = spacy.load("en_core_web_sm")
trie = load_wikipedia_trie()
config = PipelineConfig(
    mentions_extractor=MentionsExtractorSpacy(),
    linker=LinkerRegen(trie=trie),
)
nlp.add_pipe("zshot", config=config, last=True)
assert "zshot" in nlp.pipe_names

doc = nlp(EX_DOCS[1])
print(doc.ents)
print(spans_to_wikipedia(doc._.spans))

Issues

Closes #11

[codecov]

Codecov Report

Base: 92.93% // Head: 93.03% // Increases project coverage by +0.10% 🎉

Coverage data is based on head (07f7920) compared to base (b2c2a27).
Patch coverage: 98.48% of modified lines in pull request are covered.

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #44      +/-   ##
==========================================
+ Coverage   92.93%   93.03%   +0.10%     
==========================================
  Files          67       67              
  Lines        2773     2830      +57     
==========================================
+ Hits         2577     2633      +56     
- Misses        196      197       +1     

Impacted Files	Coverage Δ
zshot/linker/linker_regen/utils.py	78.04% <96.55%> (+44.71%)	⬆️
zshot/linker/linker_regen/linker_regen.py	98.30% <100.00%> (+0.05%)	⬆️
zshot/linker/linker_regen/trie.py	100.00% <100.00%> (ø)
zshot/tests/linker/test_regen_linker.py	100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.