You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
transformers is currently the de facto way to train NLP models (maybe speech and image soon?). For Thai language, we have some difficulties using the default settings; for example, tokenization for sequence-based metrics such as BLEU is based on space tokenization. We also want to include some quality-of-life functions such as easily loading datasets into datasets objects and preprocessing functions that are available in the tutorial notebooks.
transformers
is currently the de facto way to train NLP models (maybe speech and image soon?). For Thai language, we have some difficulties using the default settings; for example, tokenization for sequence-based metrics such as BLEU is based on space tokenization. We also want to include some quality-of-life functions such as easily loading datasets intodatasets
objects and preprocessing functions that are available in the tutorial notebooks.PR #67
The text was updated successfully, but these errors were encountered: