some improvements

[olivier-compilatio]

Hi,
After reading the paper, I wanted to try the software. I like the idea of using WE vectors instead of just creating vectors from the yet-to-summarize text.
It didn't work at first so I dug into the code and made some changes :
1 : the stopword remover wasn't working : stopword_remover.add_keyword(stopword, "") makes it replace stopword by itself, because "" is understood as none. I just did a quick fix but it works.
2 : In centroid_word_embeddings.py , the word_vector_cache were bugged : the dictionary in the first if was new, so empty, and the condition was never entered. I modified the dict with the embedding model dict.
3 : I updated the load_gensim_embedding_model function. Seems the loading is simpler now than it was when you wrote the code!
I hope those fixes will help you,
Olivier

[A-Gulati]

Hi Oliver, any chance you could show me how to run the model on a sample subset of documents? I'm trying to work everything out!

[olivier-compilatio]

It's quite simple, here is a python script you can try and modify.
`from text_summarizer.centroid_word_embeddings import CentroidWordEmbeddingsSummarizer
from text_summarizer.centroid_word_embeddings import load_gensim_embedding_model
import gensim.downloader as api

api.info() # show info about available models/datasets
model = api.load("glove-wiki-gigaword-300")

cwes = CentroidWordEmbeddingsSummarizer(
model, debug=True, bow_param=2, length_param=5, position_param=4
)
with open("sample.txt") as f:
text = f.read().replace("\n", " ")
cwes.summarize(text)
`

[A-Gulati]

After installing some dependencies, this worked. Thanks Oliver!

[blazejdolicki]

Oh I wish I read this merge request before trying out the repo, instead of spending a few hours to fix the same bugs you solved.