Neural-Machine-Translation

This repo serves as building of machine translation model using Seq-to-Seq model in Pytorch.

Translation: German to English
Dataset: Multi30k containing English and German captions for images from Flickr
Tokenization: sentencepiece
Prediction: Greedy

Baseline Model

LSTM based Seq-to-Seq model

Encoder: Bidirectional LSTM with each direction having 2 layers
Decoder: 2 layer LSTM
Accuracy: 50.50%
Bleu Score: 18.76

Baseline model sample predictions:

Seq-to-Seq model with attention

Neural Machine Translation by Jointly Learning to Align and Translate

Encoder: Bidirectional LSTM with each direction having 2 layers
Decoder: 2 layer LSTM with attention mechanism
Accuracy: 67.70%
Bleu Score: 38.45

Attention model sample predictions:

Adding attention mechanism to the decoder gave a significant increase in accuracy and bleu score, the translations were much clear than the baseline model.