Open Source Neural Machine Translation in Torch
Clone or download
Latest commit 27251da Jul 18, 2018
Permalink
Failed to load latest commit information.
benchmark introduce log_tag option (#517) Feb 9, 2018
data invalid conversion of │n corrected (#511) Feb 1, 2018
docs typofix found while reading docs (#535) Apr 12, 2018
hooks Integrate hooks with the rest translation server (#460) Dec 19, 2017
onmt Fix condition to detect LM training Jul 18, 2018
rocks [WIP] Hooks mechanism (#436) Dec 7, 2017
test WIP introduce token batch size limit (#506) Mar 5, 2018
tools Fix learn_bpe.lua for Lua 5.2 Jun 14, 2018
.dockerignore Docker: copy sources instead of using git clone Mar 5, 2018
.dokx autogen cli docs Nov 27, 2016
.gitignore add generated docs in gitignore Dec 5, 2016
.luacheckrc Hide more luacheck warnings Mar 6, 2017
.luacov Extend automated tests (#212) Apr 18, 2017
.travis.yml No longer build and push the Docker image from Travis Feb 27, 2018
CHANGELOG.md Update version Jul 18, 2018
CONTRIBUTING.md Update CONTRIBUTING.md Jun 16, 2017
Dockerfile Docker: add missing xavante dependency Mar 14, 2018
LICENSE.md rename license file Nov 16, 2016
README.md Update documentation (#419) Nov 2, 2017
STYLE.md Add short guide for contribution Jun 2, 2017
codecov.yml Extend automated tests (#212) Apr 18, 2017
lm.lua introduce log_tag option (#517) Feb 9, 2018
mkdocs.yml [WIP] Hooks mechanism (#436) Dec 7, 2017
preprocess.lua introduce log_tag option (#517) Feb 9, 2018
tag.lua introduce log_tag option (#517) Feb 9, 2018
train.lua WIP introduce token batch size limit (#506) Mar 5, 2018
translate.lua introduce log_tag option (#517) Feb 9, 2018

README.md

Build Status codecov

OpenNMT: Open-Source Neural Machine Translation

OpenNMT is a full-featured, open-source (MIT) neural machine translation system utilizing the Torch mathematical toolkit.

The system is designed to be simple to use and easy to extend, while maintaining efficiency and state-of-the-art translation accuracy. Features include:

  • Speed and memory optimizations for high-performance GPU training.
  • Simple general-purpose interface, only requires and source/target data files.
  • C++ implementation of the translator for easy deployment.
  • Extensions to allow other sequence generation tasks such as summarization and image captioning.

Installation

OpenNMT only requires a Torch installation with few dependencies.

  1. Install Torch
  2. Install additional packages:
luarocks install tds
luarocks install bit32 # if using LuaJIT

For other installation methods including Docker, visit the documentation.

Quickstart

OpenNMT consists of three commands:

  1. Preprocess the data.
th preprocess.lua -train_src data/src-train.txt -train_tgt data/tgt-train.txt -valid_src data/src-val.txt -valid_tgt data/tgt-val.txt -save_data data/demo
  1. Train the model.
th train.lua -data data/demo-train.t7 -save_model model
  1. Translate sentences.
th translate.lua -model model_final.t7 -src data/src-test.txt -output pred.txt

For more details, visit the documentation.

Citation

A technical report on OpenNMT is available. If you use the system for academic work, please cite:

@ARTICLE{2017opennmt,
  author = {{Klein}, G. and {Kim}, Y. and {Deng}, Y. and {Senellart}, J. and {Rush}, A.~M.},
  title = "{OpenNMT: Open-Source Toolkit for Neural Machine Translation}",
  journal = {ArXiv e-prints},
  eprint = {1701.02810}
}

Acknowledgments

Our implementation utilizes code from the following:

Additional resources