Filtering Words not working in build_corpus method

[ettoreaquino]

Description

While building a Corpus, using the litstudy.build_corpus() method I have found that remove_word argument is not acting how it should be:

remove_words = ["institution", "authors", "licensee", "recent", "years", "john", "wiley"]
Corpus = litstudy.build_corpus(docs=db,
                               remove_words=remove_words,
                               min_word_length=3,
                               min_docs=10,
                               max_docs_ratio=0.75,
                               max_tokens=1000,
                               replace_words=None,
                               custom_bigrams=None,
                               ngram_threshold=0.7)

Expected behavior

The expected behavior is pretty straightforward: passing a list of words in remove_words should filter them from the word frequency vector of each document, thus removing them from the Corpus object. Which could be checked by not finding them in the Corpus.dictionary.items():

remove_words = ["institution", "authors", "licensee", "recent", "years", "john", "wiley"]
assert [] == [item for item in Corpus.dictionary.items() if item[1] in remove_words]

Observed behavior

The words are not removed at all:

remove_words = ["institution", "authors", "licensee", "recent", "years", "john", "wiley"]
assert [] == [item for item in Corpus.dictionary.items() if item[1] in remove_words]

Raises:

---------------------------------------------------------------------------
AssertionError                            Traceback (most recent call last)
Cell In[28], line 2
      1 remove_words = ["institution", "authors", "licensee", "recent", "years", "john", "wiley"]
----> 2 assert [] == [item for item in Corpus.dictionary.items() if item[1] in remove_words]

AssertionError: 

[stijnh]

Thanks. Fixed by #39.