Refactor current gensim code by PEP8

[menshikh-iv]

We have a lot of old code which doesn't satisfy PEP8, it's time to fix it!

What we want:

• PEP8 (without a hard 80-char constraint for long lines)
• No vertical indents (only hanging)
• map/filter -> list/set/dict comprehension
• logging message arguments as method arguments (not direct formatting)

[zsef123]

Hi, I'm trying to refactor by PEP8

E731 ( do not assign a lambda expression, use a def these error ) is often seen during refactoring

ex) blas = lambda name, ndarray: scipy.linalg.get_blas_funcs((name,), (ndarray,))[0]

It looks like code writer prefer lambda expresion than def

Do you think you should change lambda expression to def in case of E731 error?

[menshikh-iv]

Hi @zsef123, we ignore E731 in gensim, look at this line.

[zsef123]

I send PR #1550
but I got some problems.

return buffer[start_index : start_index + nbytes].view(dtype).reshape(shape, order=order)

In case E203 error occurs in pep8 library( https://pypi.python.org/pypi/pep8 ), but it's fine in documentation( https://www.python.org/dev/peps/pep-0008/#other-recommendations )

I think E203 error corresponds to scope delimeter( like if True: )

So First, no change
return buffer[start_index : start_index + nbytes].view(dtype).reshape(shape, order=order)
these line still occurs error in pep8 library but it's fine in docs

Second, line changed to
return buffer[start_index:start_index + nbytes].view(dtype).reshape(shape, order=order)
these line not occurs error in pep8 library and fine too

Which of the two is better?

[piskvorky]

The second option does not conform to PEP8 (see the "No:" section in https://www.python.org/dev/peps/pep-0008/#pet-peeves).

Basically, the : operator has a lower priority than e.g. +, and so shouldn't have less whitespace because that implies more priority in Python, by convention.

[zsef123]

What about ignore F405 and F403 error

F405 : name may be undefined, or defined from star imports: module
F403 : ‘from module import *’ used; unable to detect undefined names

In test_parsing.py

from gensim.parsing.preprocessing import *
error occurs F403
self.assertEqual(strip_numeric("salut les amis du 59"), "salut les amis du ")
error occurs F405 because strip_numeric may be undefined, or defined from star

[menshikh-iv]

Resolved[1] in #1550, will be continued from me (double checking)

[menshikh-iv]

Continued in #1569