Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Thai-NNER #665

Merged
merged 3 commits into from
May 7, 2022
Merged

Add Thai-NNER #665

merged 3 commits into from
May 7, 2022

Conversation

wannaphong
Copy link
Member

@wannaphong wannaphong commented May 7, 2022

What does this changes

This pull request include thai-nner to PyThaiNLP. It is pythainlp.tag.NNER class. It was use thai_nner library and include model by pythainlp-corpus.

This issue is #664.

Thai-NNER: https://github.com/vistec-AI/Thai-NNER

from pythainlp.tag.named_entity import NNER
nner = NNER()
nner.tag("แมวทำอะไรตอนห้าโมงเช้า")
# output: (['<s>', '', 'แมว', 'ทํา', '', 'อะไร', 'ตอน', '', 'ห้า', '', 'โมง', '', 'เช้า', '</s>'],
# [{'text': ['', 'ห้า'], 'span': [7, 9], 'entity_type': 'cardinal'}, {'text': ['', 'ห้า', '', 'โมง'], 'span': [7, 11], 'entity_type': 'time'}, {'text': ['', 'โมง'], 'span': [9, 11], 'entity_type': 'unit'}])

Your checklist for this pull request

🚨Please review the guidelines for contributing to this repository.

  • Passed code styles and structures
  • Passed code linting checks and unit test

@pep8speaks
Copy link

pep8speaks commented May 7, 2022

Hello @wannaphong! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

There are currently no PEP 8 issues detected in this Pull Request. Cheers! 🍻

Comment last updated at 2022-05-07 08:01:37 UTC

@wannaphong wannaphong changed the title Add thai nner [WIP] Add Thai-NNER May 7, 2022
@coveralls
Copy link

coveralls commented May 7, 2022

Coverage Status

Coverage increased (+0.009%) to 97.207% when pulling 29c2d03 on add-thai_nner into f63042b on dev.

@wannaphong wannaphong changed the title [WIP] Add Thai-NNER Add Thai-NNER May 7, 2022
@wannaphong wannaphong merged commit b9a0330 into dev May 7, 2022
@wannaphong wannaphong deleted the add-thai_nner branch May 7, 2022 18:41
@wannaphong wannaphong added this to the 3.1 milestone May 16, 2022
wannaphong added a commit that referenced this pull request Sep 1, 2022
See [3.1 Milestone](https://github.com/PyThaiNLP/pythainlp/milestone/16).

## What is new?
### Deprecation and other API changes
#687 Remove deprecated function
- pythainlp.word_vector; doesnt_match, get_model, most_similar_cosmul, sentence_vectorizer, similarity. use WordVector class instead
- pythainlp.util.delete_tone. use pythainlp.util.remove_tonemark instead
- Remove pythainlp.util.time_time. use pythainlp.util.time_to_thaiword instead
- pythainlp.tokenize.syllable_tokenize. use pythainlp.tokenize.subword_tokenize instead

### Name Entity Tagging
- #665 Add Thai-NNER `pythainlp.tag.NNER`
- #658 Add LST20NER onnx model. It is LST20NER model to onnx model from fine-turning by [WangchanBERTa model](https://huggingface.co/airesearch/wangchanberta-base-att-spm-uncased).

### Transliteration
- #659 Add ISO 11940 transliteration
- #660 Add Thai W2P v0.2
- #686 Add wunsen
- #694 Wunsen Mandarin and Japanese update

### PyThaiNLP Corpus downloader
- #656 Add support zip/tar.gz to download corpus

### Text normalization
- #673 Add a normalising rule for Lakkhangyao ๅ

### Translate
- #674 add gpu option

### Text summarize
- #679 Add mt5 cpe kmutt thai sentence sum

### Util
- #682 Add live-dead syllable classification
- #684 Add live dead syllable classify
- #690 Add tone detector

### Other
- #689 map NG tag to PART
- #691 Remove TinyDB as a dependency
- #692 Fix notifications that newer versions of corpora are available
@wannaphong wannaphong mentioned this pull request Sep 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants