Machine Translation - Transformer

Introduction

Machine Translation is a task that we need to translate a sentence from a language to another language. Particularly, in this repository, I have leveraged the Transformer model to translate from Vietnamese (vi) to English (en).

Model architecture

Transformer is a very powerful model because of using the self-attention mechanism under the hood. It can improve the weakness of many types of recurrent neural network (RNN). Moreover, the model can be trained much faster since it can feed all the data once only, instead of one-by-one.

Transformer architecture

Gallery

Inference examples

Usage

Install libraries

Firstly, please do install all the required libraries:

cd src
pip install -r requirements.txt

Also, because we need to use tokenizer for Vietnames, we will install the model for spacy library:

pip install https://gitlab.com/trungtv/vi_spacy/-/raw/master/vi_core_news_lg/dist/vi_core_news_lg-0.0.1.tar.gz

And the model for English:

python -m spacy download en_core_web_sm

Train model

To train the model, you can either use the provided script train.sh, or use the following command:

python3 train.py --data_folder "../data" \
    --model_folder "../models"

Inference

In the same way, you can use the inference.sh script to run, or:

python3 inference.py --model_folder "../models" \
    --prompt "Ngày hôm qua, tôi thấy có một con mèo ở ngoài sân. Nó leo lên cây, nhưng nó đã bị té." 

Contribution

The repository is implemented following the instruction of pbcquoc/transformer and improved by phuc16102001.

Reference

Thank you very much for a very detail instruction of pbcquoc and the pretrained tokenizer of trungtv.

License

MIT