Skip to content

SimulEval: A General Evaluation Toolkit for Simultaneous Translation


Notifications You must be signed in to change notification settings



SimulEval is a general evaluation framework for simultaneous translation on text and speech. Full documentation can be found here.


git clone
cd SimulEval
pip install -e .

Quick Start

Following is the evaluation of a dummy agent which operates wait-k (k = 3) policy and generates random words until the length of the generated words is the same as the number of all the source words.

cd examples/quick_start
simuleval --source source.txt --target target.txt --agent


SimulEval is licensed under Creative Commons BY-SA 4.0.


Please cite as:

  title = {Simuleval: An evaluation toolkit for simultaneous translation},
  author = {Xutai Ma, Mohammad Javad Dousti, Changhan Wang, Jiatao Gu, Juan Pino},
  booktitle = {Proceedings of the EMNLP},
  year = {2020},