Unsupervised Neural Single-Document Summarization of Reviews via Learning Latent Discourse Structure and its Ranking

Corresponding paper:

https://arxiv.org/abs/1906.05691
Masaru Isonuma, Juncihiro Mori and Ichiro Sakata (The University of Tokyo)
Accepted in ACL 2019 as a long paper

Requirements

Python 3.6
Tensorflow 1.8.0

Usage

Preprocessing

The pre-processed Amazon Sports & Outdoors review data can be downloaded at:
Put it at /path/to/data

To use other dataset, run:

python create_dataframe.py --input_path /path/to/gzip/file --output_path /path/to/dataframe
python preprocess_data.py --input_path /path/to/dataframe --output_path /path/to/data --word_vec_path /path/to/vec

Other dataset can be downloaded at http://jmcauley.ucsd.edu/data/amazon/
Put raw gzip file at /path/to/gzip/file
Also, download the pretrained FastText word vector file crawl-300d-2M.vec from the url below and put it at /path/to/vec
https://fasttext.cc/docs/en/english-vectors.html

Training

To train your model, run:

python cli.py --mode train --data /path/to/data --modeldir /dir/of/model

Parameters and logs are saved in /path/to/a/model/directory
The hyper parameters can be chaneged as described in cli.py

Evaluation

To write out and evaluate the summaries generated by your model, run:

python cli.py --mode eval --data /path/to/data --modeldir /dir/of/model --refdir /dir/of/refs --outdir /dir/of/outputs

References and output summaries are respectively saved in /dir/of/refs and /dir/of/outputs
The same hyper parameters must be set as used for training