Skip to content
Branch: master
Find file History
nng555 and facebook-github-bot Fix year in noisy channel citation (#842)
Summary:
2018->2019
Pull Request resolved: fairinternal/fairseq-py#842

Differential Revision: D16973530

Pulled By: nng555

fbshipit-source-id: 00207b79821ac0257a53a0581a84582130e1bff5
Latest commit d4c9136 Aug 22, 2019
Permalink
Type Name Latest commit message Commit time
..
Failed to load latest commit information.
README.md Fix year in noisy channel citation (#842) Aug 22, 2019
__init__.py Small fixes Aug 19, 2019
rerank.py Small fixes Aug 19, 2019
rerank_generate.py Small fixes Aug 19, 2019
rerank_options.py Small fixes Aug 19, 2019
rerank_score_bw.py Backward reranking public (#667) Aug 15, 2019
rerank_score_lm.py Backward reranking public (#667) Aug 15, 2019
rerank_tune.py Small fixes Aug 19, 2019
rerank_utils.py Small fixes Aug 19, 2019

README.md

Simple and Effective Noisy Channel Modeling for Neural Machine Translation (Yee et al., 2019)

This page contains pointers to pre-trained models as well as instructions on how to run the reranking scripts.

Citation:

@inproceedings{yee2019simple,
  title = {Simple and Effective Noisy Channel Modeling for Neural Machine Translation},
  author = {Kyra Yee and Yann Dauphin and Michael Auli},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2019},
}

Pre-trained Models:

Model Description Download
transformer.noisychannel.de-en De->En Forward Model download (.tar.gz)
transformer.noisychannel.en-de En->De Channel Model download (.tar.gz)
transformer_lm.noisychannel.en En Language model download (.tar.gz)

Test Data: newstest_wmt17

Example usage

mkdir rerank_example
curl https://dl.fbaipublicfiles.com/fairseq/models/noisychannel/forward_de2en.tar.bz2 | tar xvjf - -C rerank_example
curl https://dl.fbaipublicfiles.com/fairseq/models/noisychannel/backward_en2de.tar.bz2 | tar xvjf - -C rerank_example
curl https://dl.fbaipublicfiles.com/fairseq/models/noisychannel/reranking_en_lm.tar.bz2 | tar xvjf - -C rerank_example
curl https://dl.fbaipublicfiles.com/fairseq/models/noisychannel/wmt17test.tar.bz2 | tar xvjf - -C rerank_example

beam=50
num_trials=1000
fw_name=fw_model_ex
bw_name=bw_model_ex
lm_name=lm_ex
data_dir=rerank_example/hyphen-splitting-mixed-case-wmt17test-wmt14bpe
data_dir_name=wmt17
lm=rerank_example/lm/checkpoint_best.pt
lm_bpe_code=rerank_example/lm/bpe32k.code
lm_dict=rerank_example/lm/dict.txt
batch_size=32
bw=rerank_example/backward_en2de.pt
fw=rerank_example/forward_de2en.pt

# reranking with P(T|S) P(S|T) and P(T)
python examples/noisychannel/rerank_tune.py $data_dir  --tune-param lenpen weight1 weight3  \
    --lower-bound 0 0 0 --upper-bound 3 3 3 --data-dir-name $data_dir_name  \ 
    --num-trials $num_trials  --source-lang de --target-lang en --gen-model $fw \
    -n $beam --batch-size $batch_size --score-model2 $fw --score-model1 $bw \
    --backwards1 --weight2 1 \
    -lm $lm  --lm-dict $lm_dict  --lm-name en_newscrawl --lm-bpe-code $lm_bpe_code \
    --model2-name $fw_name --model1-name $bw_name --gen-model-name $fw_name

# reranking with P(T|S) and P(T)
python examples/noisychannel/rerank_tune.py $data_dir  --tune-param lenpen weight3 \
    --lower-bound 0 0 --upper-bound 3 3  --data-dir-name $data_dir_name  \
    --num-trials $num_trials  --source-lang de --target-lang en --gen-model $fw \
    -n $beam --batch-size $batch_size --score-model1 $fw \
    -lm $lm  --lm-dict $lm_dict  --lm-name en_newscrawl --lm-bpe-code $lm_bpe_code \
    --model1-name $fw_name --gen-model-name $fw_name

# to run with a preconfigured set of hyperparameters for the lenpen and model weights, using rerank.py instead.
python examples/noisychannel/rerank.py $data_dir \
    --lenpen 0.269 --weight1 1 --weight2 0.929 --weight3 0.831  \
    --data-dir-name $data_dir_name  --source-lang de --target-lang en --gen-model $fw \
    -n $beam --batch-size $batch_size --score-model2 $fw --score-model1 $bw --backwards1  \
    -lm $lm  --lm-dict $lm_dict  --lm-name en_newscrawl --lm-bpe-code $lm_bpe_code \
    --model2-name $fw_name --model1-name $bw_name --gen-model-name $fw_name
You can’t perform that action at this time.