G-Transformer

This code is for ACL 2021 paper G-Transformer for Document-level Machine Translation.

Python Version: Python3.6

Package Requirements: torch==1.4.0 tensorboardX numpy==1.19.0

Framework: Our model and experiments are built upon fairseq.

Before running the scripts, please install fairseq dependencies by:

    pip install --editable .

Please also follow the readmes under folder raw_data and mbart.cc25 to download raw data and pretrained model.

Non-pretraining Settings

G-Transformer random initialized

• Prepare data:

    mkdir exp_randinit
    bash exp_gtrans/run-all.sh prepare-randinit exp_randinit

• Train model:

    CUDA_VISIBLE_DEVICES=0,1,2,3 bash exp_gtrans/run-all.sh run-randinit train exp_randinit

• Evaluate model:

    bash exp_gtrans/run-all.sh run-randinit test exp_randinit

G-Transformer fine-tuned on sent Transformer

• Prepare data:

    mkdir exp_finetune
    bash exp_gtrans/run-all.sh prepare-finetune exp_finetune

• Train model:

    CUDA_VISIBLE_DEVICES=0,1,2,3 bash exp_gtrans/run-all.sh run-finetune train exp_finetune

• Evaluate model:

    bash exp_gtrans/run-all.sh run-finetune test exp_finetune

Pretraining Settings

G-Transformer fine-tuned on mBART25

• Prepare data:

    mkdir exp_mbart
    bash exp_gtrans/run-all.sh prepare-mbart exp_mbart

• Train model:

    CUDA_VISIBLE_DEVICES=0,1,2,3 bash exp_gtrans/run-all.sh run-mbart train exp_mbart

• Evaluate model:

    bash exp_gtrans/run-all.sh run-mbart test exp_mbart