JANUS - a Joint Agreement Neural transdUction System for sequence2sequence learning
This toolkit implements an agreement model between left-to-right and right-to-left recurrent neural networks (with LSTM hiddent units), which converts an input sequence to an output sequence. It is mainly developed by Lemao Liu @ NICT Japan, following the paper .
Currently, it supports:
- A single left to right model;
- A single right to left model;
- The agreement model between a single left-to-right and a single right-to-left model.
In order to get more powerful performance as shown in our paper, one might need the agreement model between the ensembles of left-to-right and right-to-left models, which will be available soon (on going).
- XML::Simple and XML::Twig Perl Modules - It is only used for a evaluation script in performance. To to install it, one can use cpan. Of course, one shoule install Perl (with Perl 5.10 verified) at first.
python nn.py config.ini
In the current directory, a sample of config.ini is available. Generally, to obtain the *.xml files as shown in config.ini, please use scripts/wrapper_src_trg.py:
python scripts/wrapper_src_trg.py dev.ja dev.en >dev.ref.ja-en.xml 2>dev.src.ja-en.xml
After the training, there will be a file (eval.txt) in the working directory. This file contains four evaluation metrics (ACC, Mean-F-score, MRR and MAP_ref) for three models (left-to-right, right-to-left, and their agreement models) at each iteration, and some lines in this file are as follows:
evaluate for 66 iterations ------------- Performance for left-to-right model dev ACC: 0.270000 Mean F-score: 0.779298 MRR: 0.270000 MAP_ref: 0.270000 Performance for right-to-left model dev ACC: 0.460000 Mean F-score: 0.832969 MRR: 0.460000 MAP_ref: 0.460000 Performance for agreement model dev ACC: 0.630000 Mean F-score: 0.912477 MRR: 0.630000 MAP_ref: 0.630000 Performance for left-to-right model tst ACC: 0.260000 Mean F-score: 0.767345 MRR: 0.260000 MAP_ref: 0.260000 Performance for right-to-left model tst ACC: 0.450000 Mean F-score: 0.811715 MRR: 0.450000 MAP_ref: 0.450000 Performance for agreement model tst ACC: 0.690000 Mean F-score: 0.901153 MRR: 0.690000 MAP_ref: 0.690000
Note that to get the real testing performance, one should select the best iteration according the performance on the dev set and then report its corresponding testing performance on the test set.
About the JANUS and its LOGO
Both the name and its logo are given by Andrew Finch
Any bugs or questions
Please feel free to contact firstname.lastname@example.org. Your feedbacks are invaluable to make the toolkit powerful. Thanks!
Thanks to the tutorial at http://deeplearning.net/tutorial/lstm.html, where we follow the implementaion.
. Lemao Liu, Andrew Finch, Masao Utiyama, Eiichiro Sumita. Agreement on Target-bidirectional LSTMs for Sequence-to-Sequence Learning. In Proceedings of AAAI, 2016.