bimdp autogen breaks with sklearn 0.14.1 #8

pberkes · 2013-08-27T13:45:55Z

It is likely caused by a docstring in sklearn containing a unicode character.

Traceback (most recent call last):
 File "/Users/vagrant/src/buildsystem-git/recipes/entest_utils.py", line 268, in test1_imports
   exec 'import ' + module
 File "<string>", line 1, in <module>
 File "/Users/vagrant/src/master-env/lib/python2.7/site-packages/bimdp/__init__.py", line 91, in <module>
   from inspection import *
 File "/Users/vagrant/src/master-env/lib/python2.7/site-packages/bimdp/inspection/__init__.py", line 5, in <module>
   from tracer import (
 File "/Users/vagrant/src/master-env/lib/python2.7/site-packages/bimdp/inspection/tracer.py", line 32, in <module>
   from bimdp.hinet import BiFlowNode, CloneBiLayer
 File "/Users/vagrant/src/master-env/lib/python2.7/site-packages/bimdp/hinet/__init__.py", line 5, in <module>
   from bihtmlvisitor import BiHiNetHTMLVisitor, show_biflow
 File "/Users/vagrant/src/master-env/lib/python2.7/site-packages/bimdp/hinet/bihtmlvisitor.py", line 12, in <module>
   from bimdp.nodes import SenderBiNode
 File "/Users/vagrant/src/master-env/lib/python2.7/site-packages/bimdp/nodes/__init__.py", line 3, in <module>
   exec binodes_code()
 File "/Users/vagrant/src/master-env/lib/python2.7/site-packages/bimdp/nodes/autogen.py", line 111, in binodes_code
   _binode_module(fid, nodes)
 File "/Users/vagrant/src/master-env/lib/python2.7/site-packages/bimdp/nodes/autogen.py", line 102, in _binode_module
   old_classname=old_classname)
 File "/Users/vagrant/src/master-env/lib/python2.7/site-packages/bimdp/nodes/autogen.py", line 70, in _binode_code
   fid.write('\n        """%s"""' % docstring)
UnicodeEncodeError: 'ascii' codec can't encode character u'\u2013' in position 62: ordinal not in range(128)

The text was updated successfully, but these errors were encountered:

pberkes · 2013-08-27T13:46:37Z

Actually, the whole autogeneration looks fragile wrt unicode characters.

otizonaizit · 2013-08-27T14:03:06Z

This actually could be considered a bug in sklearn. There is a recent thread in sklear mailing list about non-ASCII characters in source code:
http://sourceforge.net/mailarchive/forum.php?thread_name=20130826125219.GD19784%40phare.normalesup.org&forum_name=scikit-learn-general

Maybe you could report it on their issue tracker?

pberkes · 2013-08-28T08:45:48Z

It's not impossible to fix on our side, and it will prevent similar issues in the future.
As it stands, with sklearn 0.14.1 everything breaks with just import bimdp

otizonaizit · 2013-08-28T08:48:24Z

If you think you can fix it so that the lib2to3-generated Python3
version still works without manual adjustments, then go for it!

On Wed 28 Aug, 01:45, Pietro Berkes notifications@github.com wrote:

It's not impossible to fix on our side, and it will prevent similar issues in the future.
As it stands, with sklearn 0.14.1 everything breaks with just import bimdp

Reply to this email directly or view it on GitHub:
#8 (comment)

pberkes · 2013-08-28T09:21:10Z

How about we fix it next week? ;-) Niko knows what that code is doing...

On Wed, Aug 28, 2013 at 9:48 AM, Tiziano Zito notifications@github.comwrote:

If you think you can fix it so that the lib2to3-generated Python3
version still works without manual adjustments, then go for it!

On Wed 28 Aug, 01:45, Pietro Berkes notifications@github.com wrote:

It's not impossible to fix on our side, and it will prevent similar
issues in the future.
As it stands, with sklearn 0.14.1 everything breaks with just import bimdp

Reply to this email directly or view it on GitHub:

#8 (comment)

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/8#issuecomment-23399692
.

nwilbert · 2013-08-28T12:00:44Z

Yes, sure, I can work on it next week. I'm also planing to finally take a
look the CUDA context manager issue then.

Cheers,
Niko

On Wed, Aug 28, 2013 at 11:21 AM, Pietro Berkes notifications@github.comwrote:

How about we fix it next week? ;-) Niko knows what that code is doing...

On Wed, Aug 28, 2013 at 9:48 AM, Tiziano Zito notifications@github.comwrote:

If you think you can fix it so that the lib2to3-generated Python3
version still works without manual adjustments, then go for it!

On Wed 28 Aug, 01:45, Pietro Berkes notifications@github.com wrote:

It's not impossible to fix on our side, and it will prevent similar
issues in the future.
As it stands, with sklearn 0.14.1 everything breaks with just import bimdp

Reply to this email directly or view it on GitHub:

#8 (comment)

—
Reply to this email directly or view it on GitHub<
https://github.com/mdp-toolkit/mdp-toolkit/issues/8#issuecomment-23399692>

.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/8#issuecomment-23401334
.

otizonaizit · 2014-03-08T12:49:00Z

This is now hitting people (and kicked MDP out of Debian testing: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=724146 ). We should really fix it ;)

nwilbert · 2014-03-08T17:32:54Z

Oh, I had completely forgotten about that. I don't remember if we did anything on our side to fix this (but I remember that I did look at the CUDA stuff, so we might have discussed this).

otizonaizit · 2014-03-08T17:39:52Z

Oh, I had completely forgotten about that. I don't remember if we
did anything on our side to fix this (but I remember that I did
look at the CUDA stuff, so we might have discussed this).

yes, someone had a fix, maybe Pietro?, but I don't find
a corresponding branch on github. There is one named
sklearn-unicode-docstrings, but refers to something similar but much
older... As this only happens with bimdp, maybe you can try to fix
it there?

nwilbert · 2014-03-08T17:44:25Z

Ok, I can probably work on it next Friday (sorry, I probably won't make it before that).

nwilbert · 2014-03-14T14:43:49Z

I think this problem was fixed some time ago by Pietro (#9), but apparently we didn't do a new release after that.

So maybe let's just do a new release after fixing #12.

otizonaizit · 2014-03-14T14:49:10Z

You sure it really fixed the problem? With current master I am still getting:

Python 2.7.6 (default, Feb 26 2014, 00:34:35) 
[GCC 4.8.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import mdp
/usr/lib/python2.7/dist-packages/sklearn/pls.py:7: DeprecationWarning: This module has been moved to cross_decomposition and will be removed in 0.16 "removed in 0.16", DeprecationWarning)
>>> mdp.config
          python: 2.7.6.final.0
             mdp: 3.4, MDP-3.3-17-gf10d974
 parallel python: 1.6.4
          shogun: v1.1.0_02ce3cd_2011-12-12_08:17_
          libsvm: libsvm.so.3
          joblib: 0.7.1
         sklearn: 0.14.1
            numx: scipy 0.12.0
          symeig: scipy.linalg.eigh
>>> import bimdp
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "bimdp/__init__.py", line 86, in <module>
    from inspection import *
  File "bimdp/inspection/__init__.py", line 5, in <module>
    from tracer import (
  File "bimdp/inspection/tracer.py", line 32, in <module>
    from bimdp.hinet import BiFlowNode, CloneBiLayer
  File "bimdp/hinet/__init__.py", line 5, in <module>
    from bihtmlvisitor import BiHiNetHTMLVisitor, show_biflow
  File "bimdp/hinet/bihtmlvisitor.py", line 12, in <module>
    from bimdp.nodes import SenderBiNode
  File "bimdp/nodes/__init__.py", line 3, in <module>
    exec binodes_code()
  File "bimdp/nodes/autogen.py", line 110, in binodes_code
    return fid.getvalue()
  File "/usr/lib/python2.7/StringIO.py", line 271, in getvalue
    self.buf += ''.join(self.buflist)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 630: ordinal not in range(128)
>>>

nwilbert · 2014-03-14T15:16:36Z

Hmm, I can't reproduce it here, maybe it is masked by the other bug? I assume you have scipy installed?

otizonaizit · 2014-03-14T15:19:24Z

Hmm, I can't reproduce it here, maybe it is masked by the other bug? I assume you have scipy installed?
yes, I have scipy installed. But didn't you say the problem with
scipy is only with python3? we are talking here about the python2
case... In any case you can force MDP not to load scipy by setting
MDPNUMX=numpy environment variable...

nwilbert · 2014-03-14T15:36:08Z

Ops, you are right, I'm confused. I got the error under Python 2 now. I'll do the fix now...

nwilbert · 2014-03-14T16:36:34Z

Fix was pushed in 4fc2b74.

Update from upstream.

otizonaizit mentioned this issue Mar 8, 2014

As I am importing bimpd I get following error. #11

Closed

otizonaizit closed this as completed Mar 14, 2014

Stewori added a commit that referenced this issue Dec 14, 2019

Merge pull request #8 from mdp-toolkit/master

04cbc04

Update from upstream.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bimdp autogen breaks with sklearn 0.14.1 #8

bimdp autogen breaks with sklearn 0.14.1 #8

pberkes commented Aug 27, 2013

pberkes commented Aug 27, 2013

otizonaizit commented Aug 27, 2013

pberkes commented Aug 28, 2013

otizonaizit commented Aug 28, 2013

pberkes commented Aug 28, 2013

nwilbert commented Aug 28, 2013

otizonaizit commented Mar 8, 2014

nwilbert commented Mar 8, 2014

otizonaizit commented Mar 8, 2014

nwilbert commented Mar 8, 2014

nwilbert commented Mar 14, 2014

otizonaizit commented Mar 14, 2014

nwilbert commented Mar 14, 2014

otizonaizit commented Mar 14, 2014

nwilbert commented Mar 14, 2014

nwilbert commented Mar 14, 2014

bimdp autogen breaks with sklearn 0.14.1 #8

bimdp autogen breaks with sklearn 0.14.1 #8

Comments

pberkes commented Aug 27, 2013

pberkes commented Aug 27, 2013

otizonaizit commented Aug 27, 2013

pberkes commented Aug 28, 2013

otizonaizit commented Aug 28, 2013

pberkes commented Aug 28, 2013

nwilbert commented Aug 28, 2013

otizonaizit commented Mar 8, 2014

nwilbert commented Mar 8, 2014

otizonaizit commented Mar 8, 2014

nwilbert commented Mar 8, 2014

nwilbert commented Mar 14, 2014

otizonaizit commented Mar 14, 2014

nwilbert commented Mar 14, 2014

otizonaizit commented Mar 14, 2014

nwilbert commented Mar 14, 2014

nwilbert commented Mar 14, 2014