Automatic summarisation of web pages
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
autosummary
static
templates
.gitignore
README.md
__init__.py
app.py
requirements.txt

README.md

autosummary

A simple algorithm for summarising the content of web pages

Extraction based summarisation

Words in the text are projected in a vector space using word2vec trained on the brown corpus in nltk. Then the meaning of sentences is captured as the mean of the individual term vectors. The text is then represented as a fully connected weighted graph where sentences are nodes and edges between them are assigned a weight equal to the cosine distance between the sentence's vectors.

Each sentence is assigned an individual score given by the sum of distances from each other sentence in the text.

The final summary is built by ranking the sentences according to their final score and using the top 20%

TODO

  • text as input
  • input percentage of text to keep (slider)
  • train word2vec from scratch
  • improve parser
  • code TextRank