An implementation of Submodular Optimization for Extractive Multi-Document Summarization
(Or rather, two implementations, there is one in Java, and one in Python).
Both the Java and the Python version can use the CLUTO binary for clustering. It can be freely downloaded from:
If not available, the Python version will use kmeans from scipy.
Both versions also rely on a stopwords-list (there is one distributed with ROUGE that can be used). In the Java version, this can be specified on the command line. See usage below.
At the time of writing, you have to specify the paths for these files in the source code. For the Java version, the paths are in SubSumBase.java, and for the Python version, the paths are in subsum.py and subsum_clustering.py.
Using several similarity measures at the same time
The default behaviour when specifying several similarity measures at the same time, is that they will be element-wise multiplied. This corresponds to a conjunctive combination of kernels in the field of kernel methods (even though these similarity measures are not necessarily PSD kernels).
The Java version is similar to the python version, but has been used more on the command line, and thus, it has more command line parameters supported.
java SubSum.java --matrix-file [--matrix-file [--matrix-file [...]]] --document-file file-or-directory --stopwords-file filename [--collection-for-idfs directory] [--word-clusters-file filename] [--nb-sentences summarysize] [--nb-words summarysize] [--nb-characters summarysize] [--lambda lambda] [--disable-bigrams] [--print-as-text]
pip install stemming
Gensim for reading pretrained word2vec vectors (optional):
pip install gensim
Run multsum.py once without arguments to get basic usage info for the current version.
If --spl is specified the input files are assumed to already be split into sentences, one sentence per line. Otherwise, sentence splitting will be performed for you.
subsum.py --s sentences-file [--m matrix-file1 [--m matrix-file2 [...]]]
Will summarize the document in sentences-file, using the sentence similarity matrices in the matrix-files. You can specify an arbitrary number of matrix-files, but in this mode, there has to be one sentences-file.
All input files are plain text files.
The matrix-files must contain square matrices of the same dimension as the number of sentences in the sentences-file. Elements are separated by commas, rows separated by semi-colons:
a11, a12, a13; a21, a22, a23; a31, a32, a33;
subsum.py sentences-file1[, sentences-file2[, sentences-file3[, ...]]]
The summarizer will use all sentences as input and compute a summary that's representative for all input documents.