An implementation of a paper "Get To The Point: Summarization with Pointer-Generator Networks" with a few modifications added.
Evaluating on ria-news dataset.
Baseline\Metric | rouge-1-f | rouge-1-p | rouge-1-r | rouge-2-f | rouge-2-p | rouge-2-r | rouge-l-f | rouge-l-p | rouge-l-r |
---|---|---|---|---|---|---|---|---|---|
First sentence | 0.23394544 | 0.16467702 | 0.44054993 | 0.10301857 | 0.07112514 | 0.20716452 | 0.16290575 | 0.15132185 | 0.40389810 |
3 sentences | 0.15235018 | 0.09042597 | 0.53039291 | 0.05835634 | 0.03373963 | 0.24088673 | 0.08697666 | 0.08455916 | 0.49655811 |
Seq2seq+attn | 0.41582201 | 0.43718827 | 0.40561483 | 0.24086001 | 0.25307833 | 0.23581065 | 0.38644251 | 0.41508011 | 0.38504612 |
Pointer-Generator | 0.41094998 | 0.43332178 | 0.39906258 | 0.24201008 | 0.25566738 | 0.23547465 | 0.38392726 | 0.41334106 | 0.38063485 |
Seq2seq+attn+glove | 0.41925908 | 0.44047456 | 0.40921883 | 0.24513830 | 0.25763531 | 0.23992956 | 0.39001231 | 0.41847043 | 0.38874970 |
Pointer-Generator+Glove | 0.41242970 | 0.43542033 | 0.40028182 | 0.24193624 | 0.25587737 | 0.23518939 | 0.38503439 | 0.41518183 | 0.38158816 |
Seq2Seq+attn+unsup | 0.27785800 | 0.29142466 | 0.27177418 | 0.13108816 | 0.13720493 | 0.12893772 | 0.25623700 | 0.27484959 | 0.25632151 |
Seq2Seq+attn+ner | 0.15539128 | 0.16671571 | 0.14977648 | 0.05272446 | 0.05646063 | 0.05105246 | 0.14240497 | 0.15737464 | 0.14127297 |
Seq2Seq+attn+1sent | 0.38811201 | 0.40609767 | 0.37947066 | 0.22280892 | 0.23364022 | 0.21827619 | 0.36166143 | 0.38600110 | 0.36074401 |
Seq2Seq+attn+3sent | 0.19930877 | 0.21369066 | 0.19196596 | 0.08043450 | 0.08500956 | 0.07857434 | 0.18417329 | 0.20316124 | 0.18246018 |