DeprecationWarning: numpy sum :: np.sum(generator) is deprecated - LDA

[raviolli]

Warning:
score += np.sum(cnt * logsumexp(Elogthetad + Elogbeta[:, int(id)]) for id, cnt in doc)
/usr/local/lib/python2.7/dist-packages/gensim/models/ldamodel.py:1077: DeprecationWarning: Calling np.sum(generator) is deprecated, and in the future will give a different result. Use np.sum(np.from_iter(generator)) or the python sum builtin instead.
score += np.sum(cnt * logsumexp(Elogthetad + Elogbeta[:, int(id)]) for id, cnt in doc)

Versions

import platform; print(platform.platform())
Linux-4.16.0-x86_64-with-Ubuntu-18.04-bionic
import sys; print("Python", sys.version)
('Python', '2.7.15rc1 (default, Nov 12 2018, 14:31:15) \n[GCC 7.3.0]')
import numpy; print("NumPy", numpy.version)
('NumPy', '1.15.4')
import scipy; print("SciPy", scipy.version)
('SciPy', '1.1.0')
import gensim; print("gensim", gensim.version)
('gensim', '3.6.0')
from gensim.models import word2vec;print("FAST_VERSION", word2vec.FAST_VERSION)
('FAST_VERSION', 1)

[shravik]

I am facing the similar problem for LDA using gensim, Hope to receive some help for this.

Code:
lda_model = gensim.models.ldamodel.LdaModel(corpus=doc_term_matrix,
id2word=id2word,
num_topics=20,
random_state=100,
update_every=1,
chunksize=100,
passes=10,
alpha='auto',
per_word_topics=True)

Warning

C:\Users\shrav\Anaconda3\lib\site-packages\gensim\models\ldamodel.py:826: DeprecationWarning: Calling np.sum(generator) is deprecated, and in the future will give a different result. Use np.sum(np.from_iter(generator)) or the python sum builtin instead.
score += np.sum(cnt * logsumexp(Elogthetad + Elogbeta[:, int(id)]) for id, cnt in doc)
Traceback (most recent call last):

File "", line 9, in
per_word_topics=True)

File "C:\Users\shrav\Anaconda3\lib\site-packages\gensim\models\ldamodel.py", line 371, in init
self.update(corpus, chunks_as_numpy=use_numpy)

File "C:\Users\shrav\Anaconda3\lib\site-packages\gensim\models\ldamodel.py", line 734, in update
self.do_mstep(rho(), other, pass_ > 0)

File "C:\Users\shrav\Anaconda3\lib\site-packages\gensim\models\ldamodel.py", line 781, in do_mstep
self.print_topics(5)

File "C:\Users\shrav\Anaconda3\lib\site-packages\gensim\models\basemodel.py", line 36, in print_topics
return self.show_topics(num_topics=num_topics, num_words=num_words, log=True)

File "C:\Users\shrav\Anaconda3\lib\site-packages\gensim\models\ldamodel.py", line 883, in show_topics
topic_ = topic_ / topic_.sum() # normalize to probability distribution

File "C:\Users\shrav\Anaconda3\lib\site-packages\numpy\core_methods.py", line 36, in _sum
return umr_sum(a, axis, dtype, out, keepdims, initial)

TypeError: reduce() takes at most 5 arguments (6 given)

[menshikh-iv]

@raviolli thanks for the report!

@shravik looking into your stack trace, something goes wrong with your numpy, try to re-install it together with gensim.

[haskarb]

Is there any solution available? I recently installed the all the packages. Also the result varies with every run. The coherence value of topic gets changed with every run.

[menshikh-iv]

Also the result varies with every run. The coherence value of topic gets changed with every run.

This can happen if you don't pin seed, i.e. expected situation.
The warning can be easily fixed in the way that mentioned directly in warning. Feel free to contribute!

[rsdel2007]

I would like to contribute 😃 .

[menshikh-iv]

@rsdel2007 you are welcome :)

[shishir13]

I am still getting this issue /usr/local/lib/python3.6/dist-packages/gensim/models/ldamodel.py:1077: DeprecationWarning: Calling np.sum(generator) is deprecated, and in the future will give a different result. Use np.sum(np.fromiter(generator)) or the python sum builtin instead. score += np.sum(cnt * logsumexp(Elogthetad + Elogbeta[:, int(id)]) for id, cnt in doc)
Any solution for this ?

[piskvorky]

@shishir13 this issue is closed. If you want to report a similar issue, please open a new ticket, with full information (follow the issue template).