Skip to content
Facebook AI Research Sequence-to-Sequence Toolkit written in Python.
Branch: master
Clone or download
myleott and facebook-github-bot Remove unused import
Summary: Pull Request resolved: #587

Differential Revision: D14517597

Pulled By: myleott

fbshipit-source-id: 4831ea5a9da1c2e207529a4ab3c4d0b070f5f34e
Latest commit b26d6b5 Mar 19, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
docs 0.6.1 -> 0.6.2 (#577) Mar 15, 2019
examples
fairseq 0.6.1 -> 0.6.2 (#577) Mar 15, 2019
fairseq_cli Add fairseq to PyPI (#495) Feb 9, 2019
scripts Update scoring script for MoE paper Mar 19, 2019
tests Handle 3+ dimensional input in sequence_generator + nits Mar 12, 2019
.gitignore ignore data files in .gitignore Mar 1, 2019
CONTRIBUTING.md Architecture settings and readme updates Sep 15, 2017
LICENSE Initial commit Sep 15, 2017
PATENTS Initial commit Sep 15, 2017
README.md
eval_lm.py 0.6.1 -> 0.6.2 (#577) Mar 15, 2019
fairseq.gif Initial commit Sep 15, 2017
fairseq_logo.png
generate.py
interactive.py
preprocess.py
score.py Move string line encoding logic from tokenizer to Dictionary (unified… Feb 28, 2019
setup.py
train.py

README.md

Introduction

Fairseq(-py) is a sequence modeling toolkit that allows researchers and developers to train custom models for translation, summarization, language modeling and other text generation tasks. It provides reference implementations of various sequence-to-sequence models, including:

Fairseq features:

  • multi-GPU (distributed) training on one machine or across multiple machines
  • fast generation on both CPU and GPU with multiple search algorithms implemented:
  • large mini-batch training even on a single GPU via delayed updates
  • fast half-precision floating point (FP16) training
  • extensible: easily register new models, criterions, tasks, optimizers and learning rate schedulers

We also provide pre-trained models for several benchmark translation and language modeling datasets.

Model

Requirements and Installation

  • PyTorch version >= 1.0.0
  • Python version >= 3.6
  • For training new models, you'll also need an NVIDIA GPU and NCCL

Please follow the instructions here to install PyTorch: https://github.com/pytorch/pytorch#installation.

If you use Docker make sure to increase the shared memory size either with --ipc=host or --shm-size as command line options to nvidia-docker run.

After PyTorch is installed, you can install fairseq with pip:

pip install fairseq

Installing from source

To install fairseq from source and develop locally:

git clone https://github.com/pytorch/fairseq
cd fairseq
pip install --editable .

Improved training speed

Training speed can be further improved by installing NVIDIA's apex library with the --cuda_ext option. fairseq will automatically switch to the faster modules provided by apex.

Getting Started

The full documentation contains instructions for getting started, training new models and extending fairseq with new model types and tasks.

Pre-trained models and examples

We provide pre-trained models and pre-processed, binarized test sets for several tasks listed below, as well as example training and evaluation commands.

We also have more detailed READMEs to reproduce results from specific papers:

Join the fairseq community

License

fairseq(-py) is BSD-licensed. The license applies to the pre-trained models as well. We also provide an additional patent grant.

Credits

This is a PyTorch version of fairseq, a sequence-to-sequence learning toolkit from Facebook AI Research. The original authors of this reimplementation are (in no particular order) Sergey Edunov, Myle Ott, and Sam Gross.

You can’t perform that action at this time.