sklearn 0.19.1 OverflowError: signed integer is greater than maximum

[xiaokc]

hi, I augment my data set in order to update my model, this error occurs after running a perioid of time, I saw this issue #9147
and this: #6183
I upgrade my version from 0.18 to 0.19.1, but it doesn't work. Now I don't know how to deal with it. Any thoughts? thanks very much.

Description

File "/usr/lib64/python2.7/site-packages/sklearn/feature_extraction/text.py", line 1361, in fit
X = super(TfidfVectorizer, self).fit_transform(raw_documents)
File "/usr/lib64/python2.7/site-packages/sklearn/feature_extraction/text.py", line 869, in fit_transform
self.fixed_vocabulary_)
File "/usr/lib64/python2.7/site-packages/sklearn/feature_extraction/text.py", line 805, in _count_vocab
indptr.append(len(j_indices))
OverflowError: signed integer is greater than maximum

[rth]

This was fixed after the 0.19.1 was released so you would need to use the development version.

Once this large document term matrix is created several algorithms will still fail to work with 64 bit indexed sparse arrays cf #2969

[qinhanmin2014]

@xiaokc Thanks for the issue. I think it's resolved in #9147, please try current master.
Please reopen with standalone snippet if the problem still exists.

[ghost]

scikit 0.19.1

[qinhanmin2014]

@KeL3vRa See above comments. It's resolved in master and will be included in the next release.

[skasturi]

@qinhanmin2014 , Can you please let me know if this is released?

[jnothman]

Yes, the fix has been released.