'pattern' library getting automatically invoked if installed when using LdaModel.load()

[deakkon]

While loading previously saved LDA model with LdaModel.load() 'pattern' library gets invoked automatically (if installed).

Error message:

2015-09-21 18:11:23,052 : INFO : loading LdaModel object from /Users/jurica/Documents/workspace/eclipse/pubMedUBS/HR_Corpus/ldaModel.lda

2015-09-21 18:11:23,553 : INFO : loading id2word recursively from /Users/jurica/Documents/workspace/eclipse/pubMedUBS/HR_Corpus/ldaModel.lda.id2word.* with mmap=r

2015-09-21 18:11:23,553 : INFO : loading expElogbeta from /Users/jurica/Documents/workspace/eclipse/pubMedUBS/HR_Corpus/ldaModel.lda.expElogbeta.npy with mmap=r

Traceback (most recent call last):

  File "/Users/jurica/Documents/workspace/eclipse/pubMedUBS/UMLSdb/processMetaMappedMEDLINEBaseline.py", line 372, in <module>

    processResults.analyzeLDA()

  File "/Users/jurica/Documents/workspace/eclipse/pubMedUBS/UMLSdb/processMetaMappedMEDLINEBaseline.py", line 348, in analyzeLDA

    model = LdaModel.load(self.__destination+model,  mmap='r')

  File "/Users/jurica/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/gensim/models/ldamodel.py", line 887, in load

    result = super(LdaModel, cls).load(fname, *args, **kwargs)

  File "/Users/jurica/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/gensim/utils.py", line 256, in load

    obj._load_specials(fname, mmap, compress, subname)

  File "/Users/jurica/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/gensim/utils.py", line 287, in _load_specials

    val = numpy.load(subname(fname, attrib), mmap_mode=mmap)

  File "/Users/jurica/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/numpy/lib/npyio.py", line 391, in load

    return format.open_memmap(file, mode=mmap_mode)

  File "/Users/jurica/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/numpy/lib/format.py", line 722, in open_memmap

    shape, fortran_order, dtype = _read_array_header(fp, version)

  File "/Users/jurica/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/numpy/lib/format.py", line 479, in _read_array_header

    d = safe_eval(header)

  File "/Users/jurica/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/numpy/lib/utils.py", line 1168, in safe_eval

    ast = compiler.parse(source, mode="eval")

  File "/Applications/Canopy.app/appdata/canopy-1.5.5.3123.macosx-x86_64/Canopy.app/Contents/lib/python2.7/compiler/transformer.py", line 53, in parse

    return Transformer().parseexpr(buf)

  File "/Applications/Canopy.app/appdata/canopy-1.5.5.3123.macosx-x86_64/Canopy.app/Contents/lib/python2.7/compiler/transformer.py", line 132, in parseexpr

    return self.transform(parser.expr(text))

AttributeError: 'module' object has no attribute 'expr'¸

Environment:
OS X Yosemite (10.10.5) 64bit,
Canopy Python distribution with:
pattern 2.6
gensim 0.12.2
numpy 1.9.2
scipy 0.16
sklearn 0.16.1

with Python 2.7

[ziky90]

I'm trying to fix this bug, since it sometimes bothers me as well.

What do you think would be more elegant approach:

1. split the utils.py file (for example move lemmatize out of utils.py)

2. put the pattern library check inside of lemmatize and throw some exception when user attempts to call lemmatize without installed pattern library

I am more for the option 2) though I don't think that it's clear solution either. Proposed fix is implemented in #528

[ziky90]

Found another approach number 3) that I have implemented in the PR #528