emacs freezes when tab is hit in ipython with latest python-mode #1015

Merged
merged 5 commits into from Nov 24, 2011

8 participants

@paykroyd

This relates to the issue I filed: #1010

python-mode has it's own completion function mapped to the tab key. When that mapping wins, the tab will cause that function to fire and it will hang emacs. I've used a defadvice to redirect every call to python-mode's completion function instead to ipython's.

Honestly, I'm not sure if this is the best way since python-mode seems to try to make all configuration buffer-local and this is more of a global solution. But to get ipython working with python-mode you current have to bypass that anyway.

Suggestions welcome.

(I've got @deftpunk's patch in here too because I needed that to get ipython working with the most recent python-mode. Sorry if that's messy.)

paykroyd added some commits Nov 18, 2011
@paykroyd paykroyd merging deftpunk's patch for python-mode f54e9e3
@paykroyd paykroyd Ensures that ipython-complete is called.
python-mode defines a key-binding for the tab key for its own
completion function (py-shell-complete) if that one wins, then
py-shell-complete will hang emacs as it tries to communicate
with the python shell process which does not exist. This uses
def-advice to instead call ipython-complete when py-shell-complete
is called.
4a983ca
@paykroyd paykroyd Switches to use a better completion API and send the whole line.
* sends the whole line of code to get better results
* not updated for xemacs yet
65ce75f
@fperez
IPython member

@paykroyd, the changes look good, but I'm struggling to make it work easily and fully with python-mode-6.0.4. Bear with me, I'm pretty ignorant of elisp... :)

problem 1

I can get it to load ipython instead of python with C-c C-!, but only if I put

(setq py-shell-name "ipython")

before python-mode is loaded at all, by anything. It means it took quite a bit of trial and error moving this line around in my config files; ultimately I stuck it just at the very beginning of my file. This didn't use to be the case long ago, one could just

(require 'ipython)

and everything would work. It would be nice to understand what's going on here, so we can give users some guidance.

problem 2

All my attempts at configuring the arguments, to load with pylab mode active, failed. I put

(setq py-python-command-args '("--pylab" "--colors" "LightBG"))

everywhere in my emacs init files, from the start to the end, and no matter what I do, the value of that variable is reported as just -i. Furthermore, I see there's code in ipython.el to auto-detect the color scheme and set --colorsappropriately, but it doesn't seem to be working, because the variablepy-python-command-args` is also not being set automatically (I tested by not setting it myself, and nothing happened).

I temporarily disabled all other python-related files, and tested that the codepath for auto-detection of the background is being taken. In fact, py-python-command-args is (--colors=LightBG -i) as I verified with a print statement in the elisp. But strangely, once the emacs session starts, it's back to being only -i, so something is overwriting it later...

The only way I could make it work was by setting it interactively with M-x set-variable. I went as far as completely nuking (temporarily) my entire emacs configuration, leaving a bare .emacs file with nothing but this:

(add-to-list 'load-path "/home/fperez/.emacs.conf.d/lisp")
(add-to-list 'load-path "/home/fperez/.emacs.conf.d/lisp/python-mode")
(setq py-shell-name "ipython")
(require 'ipython)
(setq py-python-command-args '("--colors" "LightBG"))

and this did make ipython start, but still the colors didn't work right.

So we seem to have a problem where the order of loading of the files is getting things rewritten, no idea why.

In any case, if you can point me to a bare .emacs file with the minimum amount that makes it work, and I can reproduce it, I'd be thrilled to add these fixes. But today I've already spent ~4 hours fighting with emacs and getting nowhere... I hope we can find a solution for this, and really appreciate your help.

@fperez
IPython member

One more data point: with your fixes and the bare .emacs file above, %debug now works better, which is great! But it will only track files with %colors nocolor, it seems the ansi color escapes confuse the regexp matcher. I get

error in process filter: not: Symbol's value as variable is void: py-pydbtrack-stack-entry-regexp

and a bunch of naked ansi junk on the debug buffer:

> �[0;32m/home/fperez/scratch/error.py�[0m(45)�[0;36mRampNum�[0;34m()�[0m
�[0;32m     44 �[0;31m    �[0mstep�[0m �[0;34m=�[0m �[0;34m(�[0m�[0mend�[0m�[0;34m-�[0m�[0mstart�[0m�[0;34m)�[0m�[0;34m/�[0m�[0;34m(�[0m�[0msize�[0m�[0;34m-�[0m�[0;36m1�[0m�[0;34m-�[0m�[0mtmp�[0m�[0;34m)�[0m�[0;34m�[0m�[0m
�[0m�[0;32m---> 45 �[0;31m    �[0mresult�[0m�[0;34m[�[0m�[0;34m:�[0m�[0;34m]�[0m �[0;34m=�[0m �[0marange�[0m�[0;34m(�[0m�[0msize�[0m�[0;34m)�[0m�[0;34m*�[0m�[0mstep�[0m �[0;34m+�[0m �[0mstart�[0m�[0;34m�[0m�[0m
�[0m�[0;32m     46 �[0;31m�[0;34m�[0m�[0m
�[0m
ipdb> 

If this can't be fixed that's OK, we can just advertise that to use %debug within emacs, people should simply temporarily turn coloring off.

@fperez
IPython member

One more: this small change should probably be applied:

diff --git a/docs/emacs/ipython.el b/docs/emacs/ipython.el
index 1b11e5f..f56a1fd 100644
--- a/docs/emacs/ipython.el
+++ b/docs/emacs/ipython.el
@@ -218,7 +218,7 @@ the second for a 'normal' command, and the third for a multiline command.")
           py-shell-input-prompt-2-regexp "^   [.][.][.]+: *" )
     ;; select a suitable color-scheme
     (unless (delq nil
-                  (mapcar (lambda (x) (eq (string-match "^--colors=*" x) 0))
+                  (mapcar (lambda (x) (eq (string-match "^--colors*" x) 0))
                           py-python-command-args))
       (push (format "--colors=%s"
                     (cond

b/c the = sign is now optional.

@paykroyd

Good feedback. I had sort of forgotten that I had made some changes to my environment just to get it all working together. I'll try to address those problems.

I'm still testing the %debug stuff. I found that with %pdb on it would jump to the right file, but the location in that file was not correct. I'm still working on that.

Based on a note in ipython.el, I also modified the completion code to send the entire line as well the current item being completed and switched it to use get_ipython().complete(). That works nicely, it can now complete things import statements. I still need to make that change for the xemacs completion function.

@fperez
IPython member

Awesome! It looks like we're finally getting out of the rut we've been in with emacs support! Many thanks to you (and @defunkt) for pitching in, welcome to the team :)

paykroyd added some commits Nov 21, 2011
@paykroyd paykroyd Gets ipython working with python-mode with little config.
* sets the default value for py-shell-name to the ipython-command
* fixes color scheme setting to set a default value
cbf9083
@paykroyd paykroyd Changes color config detection based on feed back from @fperez. 32a579d
@paykroyd

@fperez, ok I think that I have the config problem fixed. I tested my changes with a vanilla install. One of the problems was that python-mode makes these configuration variables buffer-local. So setting them once isn't always effective.

Here is the complete .emacs configuration that I used to test:

(add-to-list 'load-path "~/.emacs.d/python-mode")
(add-to-list 'load-path "~/.emacs.d/ipython/docs/emacs")
(require 'ipython)

I think the color issues are sorted as well and I incorporated the change that you suggested for detecting whether a color option had been set.

There are still some issues to be sorted with pdb, I think, and with some making completion smoother, but maybe that's a different pull request?

Let me know if you have any issues.

@paykroyd

Oh I forgot to mention, to test manually configuring the parameters to ipython, see the documentation change. Use setq-default instead of setq.

@deftpunk

paykroyd: Given that python-mode makes some variables buffer-local, is it even necessary for ipython.el to do

(require 'python-mode) 

anymore? Is it possible to do

(setq-default somevar somevalue)

on all of those variables directly in ipython.el - this might help with python.el compatibility.

@paykroyd

deftpunk: I think it still needs to require it because ipython.el modifies the py-mode-map as well as a few other variables defined in python-mode.el. I'm pretty green to elisp though, is there a better way?

@jenshnielsen

Seems to work fine for me.

One bug: It used to be possible to complete from a buffer and not only the terminal inside emacs. When I do that now
I get a Buffer *.py has no process error

One other long standing bug that I noticed is that one cannot complete stuff that contains a - because ship-chars-backwards does not skip it. If one changes l.394 to this it works.
(beg (save-excursion (skip-chars-backward "a-z0-9A-Z_./\-" (point-at-bol))
The slash is to escape the - in the regex

@fperez
IPython member

Hi folks, I'll try to finish the merge for this one soon, but I wanted to point your attention to #22, which also deals with emacs-related issues. If you have a chance to look at that code and you find anything worth doing with it (presuming the author is willing to relicense it as BSD for ipython), that would be great. And if everything there has already been done, we can simply close that issue.

@fperez fperez added a commit that referenced this pull request Nov 24, 2011
@fperez fperez Add '-' to escaped characters for emacs completion.
After feedback from @jenshnielsen in #1015.
9eb1305
@fperez fperez merged commit 32a579d into ipython:master Nov 24, 2011
@fperez
IPython member

Guys, I went ahead and merged it because it's already a big improvement over the previous situation. Oddly, tab completion isn't working at all for me. But because everything else is already so much better, I'd rather merge it now. If you guys think you know what could be the problem, we can open another PR for a new fix. I tried with the absolutely minimal .emacs file as noted above by @paykroyd, and still completion doesn't work even for something like 'import'.

But in any case, thanks a lot for the work, things are already in much better shape than before!

@takluyver
IPython member

Emacs users: can you have a look at the docs about ipython.el, and update anything that needs it? In particular, I hope the issue with Python 2.2 has been solved now. ;-)

The relevant section is here: https://github.com/ipython/ipython/blob/master/docs/source/config/editors.txt#L40

@paykroyd

@fperez, just to clarify, you were trying to do an import in the ipython shell, right? Were there any messages in the Messages buffer after it failed? If you start to do an import and then type in M-x ipython-complete does it work?

Thanks.

@fperez
IPython member

@paykroyd: yes, this is trying to complete by typing 'imp', which should unambigously complete to 'import'.

I tried both ways, with and with M-x ipython-complete, and both cases led to the same message in the *Messages* buffer:

Can't find completion for "imp" based on line imp [2 times]

HTH

@paykroyd

@fperez, that's odd. I was hoping for an error message. This is the line of code that runs in ipython:

print(';'.join(get_ipython().complete('%s', '%s')[1])) #PYTHON-MODE SILENT\n

Where the first string is the current word ("imp") and the next is the whole line of code. Does that API call make sense? Are you on emacs or xemacs? I just found a bug in the xemacs hanlding.

@fperez
IPython member
@andreas-roehler

Hi all,
seems something to do still at the python-mode.el side.
Linked a bug-report to this

https://bugs.launchpad.net/python-mode/+bug/934022

Cheers,
Andreas

@andreas-roehler

Hi,

if these kind of things happen, please alert us at
http://launchpad.net/python-mode
Let's prevent the evil at it's source :)

Cheers,
Andreas

@ipstone

hello, is this matter resolved? the ipython.el is indeed outdated, I tried multiple times in the past, and gave up for a while. just tried it again today on 2012-12-11, it froze my emacs 24.2 on windows ... it's a pity, maybe I'll go with vim then...

@tkf

My impression is that ipython.el is not maintained. Both python.el and python-mode.el support IPython now. So I guess there is no need for ipython.el now. If you are using Emacs 24.2, you have the new python.el which supports ipython and ipdb.

@takluyver
IPython member
@mattvonrocketstein mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this pull request Nov 3, 2014
@fperez fperez Add '-' to escaped characters for emacs completion.
After feedback from @jenshnielsen in #1015.
e7650fd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment