Skip to content
/ ARN Public

ARN - A lightweight attention structure for speeding up Transformer

8 stars 1 fork Branches Tags Activity
Star
Notifications You must be signed in to change notification settings

Kaixin-Wu-for-Open-Source/ARN

BranchesTags

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

11 Commits
examples
examples
 
 
fairseq
fairseq
 
 
runs
runs
 
 
scripts
scripts
 
 
tests
tests
 
 
CONTRIBUTING.md
CONTRIBUTING.md
 
 
PATENTS
PATENTS
 
 
README.md
README.md
 
 
distributed_train.py
distributed_train.py
 
 
eval_lm.py
eval_lm.py
 
 
fairseq.gif
fairseq.gif
 
 
generate.py
generate.py
 
 
interactive.py
interactive.py
 
 
multiprocessing_train.py
multiprocessing_train.py
 
 
preprocess.py
preprocess.py
 
 
requirements.txt
requirements.txt
 
 
score.py
score.py
 
 
setup.py
setup.py
 
 
train.py
train.py
 
 

Repository files navigation

Introduction

The Implementation of 《Speeding Up Transformer Decoding via an Attention Refinement Network》 [COLING2022，Long Paper]（Kaixin Wu, Yue Zhang, Bojie Hu, Tong Zhang）

Data Preparation

Download WMT14 En->De data to ARN/ directory.

sh runs/prepare-wmt14-en2de.sh

Model Training

  • Transformer Baseline
sh runs/run-wmt14-en2de-baseline.sh
  • ARN
sh runs/run-wmt14-en2de-arn.sh

About

ARN - A lightweight attention structure for speeding up Transformer

Resources

Readme
Activity

Stars

8 stars

Watchers

1 watching

Forks

1 fork
Report repository

Releases

No releases published

Packages

No packages published

Contributors 2

  •  
  •  

Languages