Skip to content
Multilingual Neural Machine Translation using Transformers with Conditional Normalization.
Jupyter Notebook Python
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
mlt
.gitignore
LICENSE
README.md
config.yaml
environment.yml
environment_114.yml
many_to_many_demo.ipynb
many_to_many_fine_tune_demo.ipynb
many_to_one_demo.ipynb
one_to_many_demo.ipynb
setup.py

README.md

mlt

Multilingual Neural Machine Translation using Transformers with Conditional Normalization.

Transformers with One-to-Many, Many-to-One, Many-to-Many translation, with another demo for fine-tuning, by replacing regular layer norm in both encoder and decoder with Conditional Norm.

Conditional Norm is a technique from Style Transfer, used to train a single network with multiple styles, as well as mix styles, for example

Broader set of demos at https://github.com/suyash/stylizer/blob/master/image_stylization_demo.ipynb

The goal here is similar, make the rest of the network learn a common representation, while making the normalization parameters learn language specific semantics.

The One-to-Many and Many-to-One models are trained for English to French, German, Italian and Spanish Translation and Vice-Versa.

The Many to Many model is trained on English-French, French-English, English-German and German-English.

The image stylization paper specifies how a N-style network can pick up an N+1th style through fine-tuning an existing model. Similarly, I fine-tune my Many-to-Many model to pick up Portuguese.

I also provide demos for Few-Shot and Zero-Shot translation.

Have provided SavedModels, as well as standalone notebooks that run in Google Colaboratory.

Code and Demos for the following use-cases

One to Many

Demo: https://colab.research.google.com/github/suyash/mlt/blob/master/one_to_many_demo.ipynb

Code: https://github.com/suyash/mlt/blob/master/mlt/one_to_many.py

Model: https://dl.dropboxusercontent.com/s/bssvce8t00zjwxj/one_to_many.zip?dl=0

English to French, German, Spanish and Italian translation.

This uses split vocabularies of size 8192 each.

Many to One

Demo: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_one_demo.ipynb

Code: https://github.com/suyash/mlt/blob/master/mlt/many_to_one.py

Model: https://dl.dropboxusercontent.com/s/xmroc5u6k17t5ds/many_to_one.zip?dl=0

French, German, Spanish and Italian to English translation.

This uses split vocabularies of size 8192 each.

Many to Many

Demo: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_many_demo.ipynb

Code: https://github.com/suyash/mlt/blob/master/mlt/many_to_many.py

Model: https://dl.dropboxusercontent.com/s/kzxryp6snnx5dw6/many_to_many.zip?dl=0

English, French and German on both ends.

This uses a shared vocabulary of size 40,960.

Many to Many Fine-Tuned

Demo: https://colab.research.google.com/github/suyash/mlt/blob/master/many_to_many_fine_tune_demo.ipynb

Code: https://github.com/suyash/mlt/blob/master/mlt//many_to_many_fine_tune.py

Model: https://dl.dropboxusercontent.com/s/3ixjojbpho2emd8/many_to_many_fine_tune.zip?dl=0

Portuguese added from ted_hrlr_translate to the Many to Many model.

You can’t perform that action at this time.