Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add LST20NER onnx model #658

Merged
merged 11 commits into from
Apr 26, 2022
Merged

Add LST20NER onnx model #658

merged 11 commits into from
Apr 26, 2022

Conversation

wannaphong
Copy link
Member

@wannaphong wannaphong commented Mar 16, 2022

I port LST20NER model to onnx model from fine-turning by WangchanBERTa model.

Your checklist for this pull request

🚨Please review the guidelines for contributing to this repository.

  • Passed code styles and structures
  • Passed code linting checks and unit test

@wannaphong
Copy link
Member Author

This is a part of #639.

@wannaphong
Copy link
Member Author

@wannaphong
Copy link
Member Author

wannaphong commented Mar 16, 2022

>>> from pythainlp.tag import NER
>>> ner = NER("lst20_onnx")
>>> ner.tag("วันที่ 12 เมษายน พ.ศ.2565 เรามีประชุมที่สำนักงานนายกรัฐมนตรี")
[('วันที่', 'B_DTM'), (' ', 'I_DTM'), ('12', 'I_DTM'), (' ', 'I_DTM'), ('เมษายน', 'I_DTM'), (' ', 'I_DTM'), ('พ', 'E_DTM'), ('.', 'E_DTM'), ('ศ', 'I_DTM'), ('.', 'E_DTM'), ('25', 'E_DTM'), ('65', 'E_DTM'), ('เรามี', 'O'), ('ประชุม', 'O'), ('ที่', 'O'), ('สํานักงาน', 'O'), ('นาย', 'O'), ('กรัฐมนตรี', 'O')]
>>> ner.tag("วันที่ 12 เมษายน พ.ศ.2565 เรามีประชุมที่สำนักงานนายกรัฐมนตรี",tag=True)
'<DTM>วันที่ 12 เมษายน พ.ศ.2565</DTM>เรามีประชุมที่สํานักงานนายกรัฐมนตรี'

@pep8speaks
Copy link

pep8speaks commented Mar 16, 2022

Hello @wannaphong! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:

Line 7:80: E501 line too long (80 > 79 characters)

Line 95:80: E501 line too long (83 > 79 characters)

Line 13:80: E501 line too long (133 > 79 characters)
Line 17:80: E501 line too long (85 > 79 characters)

Comment last updated at 2022-04-26 07:25:38 UTC

@coveralls
Copy link

coveralls commented Mar 16, 2022

Coverage Status

Coverage decreased (-0.3%) to 97.177% when pulling 60fcd96 on add-lst20-ner-onnx into 647c31b on dev.

@wannaphong wannaphong changed the title [WIP] Add LST20NER onnx model Add LST20NER onnx model Apr 26, 2022
@wannaphong wannaphong merged commit f63042b into dev Apr 26, 2022
@wannaphong wannaphong added this to the 3.1 milestone May 16, 2022
wannaphong added a commit that referenced this pull request Sep 1, 2022
See [3.1 Milestone](https://github.com/PyThaiNLP/pythainlp/milestone/16).

## What is new?
### Deprecation and other API changes
#687 Remove deprecated function
- pythainlp.word_vector; doesnt_match, get_model, most_similar_cosmul, sentence_vectorizer, similarity. use WordVector class instead
- pythainlp.util.delete_tone. use pythainlp.util.remove_tonemark instead
- Remove pythainlp.util.time_time. use pythainlp.util.time_to_thaiword instead
- pythainlp.tokenize.syllable_tokenize. use pythainlp.tokenize.subword_tokenize instead

### Name Entity Tagging
- #665 Add Thai-NNER `pythainlp.tag.NNER`
- #658 Add LST20NER onnx model. It is LST20NER model to onnx model from fine-turning by [WangchanBERTa model](https://huggingface.co/airesearch/wangchanberta-base-att-spm-uncased).

### Transliteration
- #659 Add ISO 11940 transliteration
- #660 Add Thai W2P v0.2
- #686 Add wunsen
- #694 Wunsen Mandarin and Japanese update

### PyThaiNLP Corpus downloader
- #656 Add support zip/tar.gz to download corpus

### Text normalization
- #673 Add a normalising rule for Lakkhangyao ๅ

### Translate
- #674 add gpu option

### Text summarize
- #679 Add mt5 cpe kmutt thai sentence sum

### Util
- #682 Add live-dead syllable classification
- #684 Add live dead syllable classify
- #690 Add tone detector

### Other
- #689 map NG tag to PART
- #691 Remove TinyDB as a dependency
- #692 Fix notifications that newer versions of corpora are available
@wannaphong wannaphong mentioned this pull request Sep 1, 2022
@wannaphong wannaphong deleted the add-lst20-ner-onnx branch September 19, 2022 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants