New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Crash on unicode scripts #43

Closed
Wilfred opened this Issue Apr 22, 2013 · 10 comments

Comments

Projects
None yet
2 participants
@Wilfred

Wilfred commented Apr 22, 2013

Given the following Python script:

# coding=utf-8

def function():
    """€

    """
    pass

Putting point on function and editing it gives me:

deferred error : (error "UnicodeDecodeError('ascii', 'function()\\\\n\\\\n\\xe2\\x82\\xac\\\\n\\\\n    ', 14, 15, 'ordinal not in range(128)')")

I believe my emacs-jedi setup is correct, it generally works well. This bug is just a slight annoynce when I'm editing this file, since my minibuffer keeps flashing up.

@tkf

This comment has been minimized.

Show comment
Hide comment
@tkf

tkf Apr 22, 2013

Owner

Can you try dev version of Jedi? You need to clone https://github.com/davidhalter/jedi, check out dev branch and then install it. I guess this is could be the bug which is already fixed in the dev version.

Owner

tkf commented Apr 22, 2013

Can you try dev version of Jedi? You need to clone https://github.com/davidhalter/jedi, check out dev branch and then install it. I guess this is could be the bug which is already fixed in the dev version.

@Wilfred

This comment has been minimized.

Show comment
Hide comment
@Wilfred

Wilfred Apr 23, 2013

The dev version of jedi gives me the same behaviour. I ran the following commands:

$ git clone git://github.com/davidhalter/jedi.git
$ cd jedi
$ git checkout dev
$ git show
commit 406e70d51ae15316adb827363bc75cdfa4131aad
Merge: 336d35e 07075a7
Author: David Halter <davidhalter88@gmail.com>
Date:   Sat Apr 20 12:49:30 2013 -0700

    Merge pull request #204 from davidhalter/testing

    Make run.py testing reusable
$ sudo pip install .

I still get the traceback above.

Wilfred commented Apr 23, 2013

The dev version of jedi gives me the same behaviour. I ran the following commands:

$ git clone git://github.com/davidhalter/jedi.git
$ cd jedi
$ git checkout dev
$ git show
commit 406e70d51ae15316adb827363bc75cdfa4131aad
Merge: 336d35e 07075a7
Author: David Halter <davidhalter88@gmail.com>
Date:   Sat Apr 20 12:49:30 2013 -0700

    Merge pull request #204 from davidhalter/testing

    Make run.py testing reusable
$ sudo pip install .

I still get the traceback above.

@tkf

This comment has been minimized.

Show comment
Hide comment
@tkf

tkf Apr 23, 2013

Owner

Hmm... Can you try to full stack trace from Python? You can get it by playing with the option for jediepcserver.py:
http://tkf.github.io/emacs-jedi/#jedi:server-args

Or pahapse the easiest (becase it is documented) way to do is to run in debug mode:
http://tkf.github.io/emacs-jedi/#jedi:toggle-debug-server

Also please run M-x jedi:show-jedi-version and paste the result here.

Owner

tkf commented Apr 23, 2013

Hmm... Can you try to full stack trace from Python? You can get it by playing with the option for jediepcserver.py:
http://tkf.github.io/emacs-jedi/#jedi:server-args

Or pahapse the easiest (becase it is documented) way to do is to run in debug mode:
http://tkf.github.io/emacs-jedi/#jedi:toggle-debug-server

Also please run M-x jedi:show-jedi-version and paste the result here.

@Wilfred

This comment has been minimized.

Show comment
Hide comment
@Wilfred

Wilfred Apr 24, 2013

jedi version:

((:version "2.7.4 (default, Apr  6 2013, 19:20:36) \n[GCC 4.8.0]" :name "sys" :file nil)
 (:version "0.5.5" :name "jedi" :file "/usr/lib/python2.7/site-packages/jedi/__init__.pyc")
 (:version "0.0.3" :name "epc" :file "/usr/lib/python2.7/site-packages/epc/__init__.pyc")
 (:version "0.0.2" :name "sexpdata" :file "/usr/lib/python2.7/site-packages/sexpdata.pyc"))

I'm not sure how to get a stack trace. I ran M-x jedi:toggled-debug-server, then ran the command:

$ python2 jediepcserver.py --port-file jedi-port.log --ipdb

in a shell. I can then see the port in the jedi-port.log file. However, it's not clear to me how to get Emacs to talk to this jediepcserver instance.

I also tried executing M-x jedi:stop-server, M-x jedi:start-dedicated-server and passed --pdb as an argument. This doesn't seem to change anything. Passing --help to jedi:start-dedicated-server works as expected, so I'm not sure why I'm not getting a stacktrace.

Wilfred commented Apr 24, 2013

jedi version:

((:version "2.7.4 (default, Apr  6 2013, 19:20:36) \n[GCC 4.8.0]" :name "sys" :file nil)
 (:version "0.5.5" :name "jedi" :file "/usr/lib/python2.7/site-packages/jedi/__init__.pyc")
 (:version "0.0.3" :name "epc" :file "/usr/lib/python2.7/site-packages/epc/__init__.pyc")
 (:version "0.0.2" :name "sexpdata" :file "/usr/lib/python2.7/site-packages/sexpdata.pyc"))

I'm not sure how to get a stack trace. I ran M-x jedi:toggled-debug-server, then ran the command:

$ python2 jediepcserver.py --port-file jedi-port.log --ipdb

in a shell. I can then see the port in the jedi-port.log file. However, it's not clear to me how to get Emacs to talk to this jediepcserver instance.

I also tried executing M-x jedi:stop-server, M-x jedi:start-dedicated-server and passed --pdb as an argument. This doesn't seem to change anything. Passing --help to jedi:start-dedicated-server works as expected, so I'm not sure why I'm not getting a stacktrace.

@tkf

This comment has been minimized.

Show comment
Hide comment
@tkf

tkf Apr 24, 2013

Owner

Here is how to use jedi:toggle-debug-server:

  1. Go to the buffer with the problematic code sample (the one with unicode).
  2. Type M-x jedi:toggle-debug-server RET. It should copy command line to the kill-ring. You will see the message saying so.
  3. Open terminal, paste the command line and hit enter. This should start the EPC server.
  4. Go back to the buffer in Emacs then execute some Jedi commands. You should see many log lines in the terminal when you execute the commands.
  5. Do some problematic command (requesting document for function with unicode document or something). This should print the stack trace in the terminal. Also you should see the pdb (or ipdb) prompt. At this point, Jedi stops working until you exit the pdb session by C-d (or q RET).
Owner

tkf commented Apr 24, 2013

Here is how to use jedi:toggle-debug-server:

  1. Go to the buffer with the problematic code sample (the one with unicode).
  2. Type M-x jedi:toggle-debug-server RET. It should copy command line to the kill-ring. You will see the message saying so.
  3. Open terminal, paste the command line and hit enter. This should start the EPC server.
  4. Go back to the buffer in Emacs then execute some Jedi commands. You should see many log lines in the terminal when you execute the commands.
  5. Do some problematic command (requesting document for function with unicode document or something). This should print the stack trace in the terminal. Also you should see the pdb (or ipdb) prompt. At this point, Jedi stops working until you exit the pdb session by C-d (or q RET).
@Wilfred

This comment has been minimized.

Show comment
Hide comment
@Wilfred

Wilfred Apr 25, 2013

Perfect. Here's the traceback:

$ python2 jediepcserver.py --port-file jedi-port.log --ipdb
(AutoLog) Called: EPCHandler.setup()
(AutoLog) Returns: EPCHandler.setup(...) = None
(AutoLog) Called: EPCHandler.handle()
receiving...
received: length = 174; data = '(call 4 complete ("# coding=utf-8\n\nimport os\n\ndef functionjkjl():\n    \\"\\"\\"\xe2\x82\xacjjkl\n\n    \\"\\"\\"\n    os.path.split()\n    pass\n" 5 16 "/home/wilfred/scratch/some_unicode.py"))\n'
(AutoLog) Called: EPCHandler._handle('(call 4 complete ("# coding=utf-8\n\nimport os\n\ndef functionjkjl():\n    \\"\\"\\"\xe2\x82\xacjjkl\n\n    \\"\\"\\"\n    os.path.split()\n    pass\n" 5 16 "/home/wilfred/scratch/some_unicode.py"))\n')
(AutoLog) Called: EPCHandler._handle_call(4, Symbol(u'complete'), [u'# coding=utf-8\n\nimport os\n\ndef functionjkjl():\n    """\u20acjjkl\n\n    """\n    os.path.split()\n    pass\n', 5, 16, u'/home/wilfred/scratch/some_unicode.py'])
(AutoLog) Returns: EPCHandler._handle_call(...) = ['return', 4, [{'doc': 'functionjkjl()\n\n\xe2\x82\xacjjkl\n\n    ', 'symbol': 'f', 'word': u'functionjkjl', 'description': u'function: some_unicode.functionjkjl'}]]
(AutoLog) Called: EPCHandler._send('return', 4, [{'doc': 'functionjkjl()\n\n\xe2\x82\xacjjkl\n\n    ', 'symbol': 'f', 'word': u'functionjkjl', 'description': u'function: some_unicode.functionjkjl'}])
ERROR:epc:UnicodeDecodeError('ascii', 'functionjkjl()\\n\\n\xe2\x82\xacjjkl\\n\\n    ', 18, 19, 'ordinal not in range(128)')
UnicodeDecodeError('ascii', 'functionjkjl()\\n\\n\xe2\x82\xacjjkl\\n\\n    ', 18, 19, 'ordinal not in range(128)')
> /usr/lib/python2.7/site-packages/sexpdata.py(90)uformat()
     89     """Alias of ``unicode(s).format(...)``."""
---> 90     return unicode(s).format(*args, **kwds)
     91 

Wilfred commented Apr 25, 2013

Perfect. Here's the traceback:

$ python2 jediepcserver.py --port-file jedi-port.log --ipdb
(AutoLog) Called: EPCHandler.setup()
(AutoLog) Returns: EPCHandler.setup(...) = None
(AutoLog) Called: EPCHandler.handle()
receiving...
received: length = 174; data = '(call 4 complete ("# coding=utf-8\n\nimport os\n\ndef functionjkjl():\n    \\"\\"\\"\xe2\x82\xacjjkl\n\n    \\"\\"\\"\n    os.path.split()\n    pass\n" 5 16 "/home/wilfred/scratch/some_unicode.py"))\n'
(AutoLog) Called: EPCHandler._handle('(call 4 complete ("# coding=utf-8\n\nimport os\n\ndef functionjkjl():\n    \\"\\"\\"\xe2\x82\xacjjkl\n\n    \\"\\"\\"\n    os.path.split()\n    pass\n" 5 16 "/home/wilfred/scratch/some_unicode.py"))\n')
(AutoLog) Called: EPCHandler._handle_call(4, Symbol(u'complete'), [u'# coding=utf-8\n\nimport os\n\ndef functionjkjl():\n    """\u20acjjkl\n\n    """\n    os.path.split()\n    pass\n', 5, 16, u'/home/wilfred/scratch/some_unicode.py'])
(AutoLog) Returns: EPCHandler._handle_call(...) = ['return', 4, [{'doc': 'functionjkjl()\n\n\xe2\x82\xacjjkl\n\n    ', 'symbol': 'f', 'word': u'functionjkjl', 'description': u'function: some_unicode.functionjkjl'}]]
(AutoLog) Called: EPCHandler._send('return', 4, [{'doc': 'functionjkjl()\n\n\xe2\x82\xacjjkl\n\n    ', 'symbol': 'f', 'word': u'functionjkjl', 'description': u'function: some_unicode.functionjkjl'}])
ERROR:epc:UnicodeDecodeError('ascii', 'functionjkjl()\\n\\n\xe2\x82\xacjjkl\\n\\n    ', 18, 19, 'ordinal not in range(128)')
UnicodeDecodeError('ascii', 'functionjkjl()\\n\\n\xe2\x82\xacjjkl\\n\\n    ', 18, 19, 'ordinal not in range(128)')
> /usr/lib/python2.7/site-packages/sexpdata.py(90)uformat()
     89     """Alias of ``unicode(s).format(...)``."""
---> 90     return unicode(s).format(*args, **kwds)
     91 
@tkf

This comment has been minimized.

Show comment
Hide comment
@tkf

tkf Apr 25, 2013

Owner

Thank you. I think now I know how to reproduce the problem in my machine. But I don't have enough time right now so it may take sometime to fix.

Owner

tkf commented Apr 25, 2013

Thank you. I think now I know how to reproduce the problem in my machine. But I don't have enough time right now so it may take sometime to fix.

@tkf

This comment has been minimized.

Show comment
Hide comment
@tkf

tkf May 19, 2013

Owner

It turned out to be sexpdata's bug. Upgrade sexpdata then it should be fine. If you use make to setup Python requirements, use make clean requirements to upgrade.

Owner

tkf commented May 19, 2013

It turned out to be sexpdata's bug. Upgrade sexpdata then it should be fine. If you use make to setup Python requirements, use make clean requirements to upgrade.

@tkf tkf closed this May 19, 2013

@Wilfred

This comment has been minimized.

Show comment
Hide comment
@Wilfred

Wilfred Jun 18, 2013

Great, I can confirm this works. Could you please update epc to depend on sexpdata>=0.0.3? Otherwise, installing from the requirements doesn't change the installed version and users will stay on the old version.

Wilfred commented Jun 18, 2013

Great, I can confirm this works. Could you please update epc to depend on sexpdata>=0.0.3? Otherwise, installing from the requirements doesn't change the installed version and users will stay on the old version.

@tkf

This comment has been minimized.

Show comment
Hide comment
@tkf

tkf Jun 18, 2013

Owner

Thanks. Good point. I will do it.

Owner

tkf commented Jun 18, 2013

Thanks. Good point. I will do it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment