Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP

Loading…

emacs freezes when tab is hit in ipython with latest python-mode #1015

Merged
merged 5 commits into from

8 participants

@paykroyd

This relates to the issue I filed: #1010

python-mode has it's own completion function mapped to the tab key. When that mapping wins, the tab will cause that function to fire and it will hang emacs. I've used a defadvice to redirect every call to python-mode's completion function instead to ipython's.

Honestly, I'm not sure if this is the best way since python-mode seems to try to make all configuration buffer-local and this is more of a global solution. But to get ipython working with python-mode you current have to bypass that anyway.

Suggestions welcome.

(I've got @deftpunk's patch in here too because I needed that to get ipython working with the most recent python-mode. Sorry if that's messy.)

paykroyd added some commits
@paykroyd paykroyd merging deftpunk's patch for python-mode f54e9e3
@paykroyd paykroyd Ensures that ipython-complete is called.
python-mode defines a key-binding for the tab key for its own
completion function (py-shell-complete) if that one wins, then
py-shell-complete will hang emacs as it tries to communicate
with the python shell process which does not exist. This uses
def-advice to instead call ipython-complete when py-shell-complete
is called.
4a983ca
@paykroyd paykroyd Switches to use a better completion API and send the whole line.
* sends the whole line of code to get better results
* not updated for xemacs yet
65ce75f
@fperez
Owner

@paykroyd, the changes look good, but I'm struggling to make it work easily and fully with python-mode-6.0.4. Bear with me, I'm pretty ignorant of elisp... :)

problem 1

I can get it to load ipython instead of python with C-c C-!, but only if I put

(setq py-shell-name "ipython")

before python-mode is loaded at all, by anything. It means it took quite a bit of trial and error moving this line around in my config files; ultimately I stuck it just at the very beginning of my file. This didn't use to be the case long ago, one could just

(require 'ipython)

and everything would work. It would be nice to understand what's going on here, so we can give users some guidance.

problem 2

All my attempts at configuring the arguments, to load with pylab mode active, failed. I put

(setq py-python-command-args '("--pylab" "--colors" "LightBG"))

everywhere in my emacs init files, from the start to the end, and no matter what I do, the value of that variable is reported as just -i. Furthermore, I see there's code in ipython.el to auto-detect the color scheme and set --colorsappropriately, but it doesn't seem to be working, because the variablepy-python-command-args` is also not being set automatically (I tested by not setting it myself, and nothing happened).

I temporarily disabled all other python-related files, and tested that the codepath for auto-detection of the background is being taken. In fact, py-python-command-args is (--colors=LightBG -i) as I verified with a print statement in the elisp. But strangely, once the emacs session starts, it's back to being only -i, so something is overwriting it later...

The only way I could make it work was by setting it interactively with M-x set-variable. I went as far as completely nuking (temporarily) my entire emacs configuration, leaving a bare .emacs file with nothing but this:

(add-to-list 'load-path "/home/fperez/.emacs.conf.d/lisp")
(add-to-list 'load-path "/home/fperez/.emacs.conf.d/lisp/python-mode")
(setq py-shell-name "ipython")
(require 'ipython)
(setq py-python-command-args '("--colors" "LightBG"))

and this did make ipython start, but still the colors didn't work right.

So we seem to have a problem where the order of loading of the files is getting things rewritten, no idea why.

In any case, if you can point me to a bare .emacs file with the minimum amount that makes it work, and I can reproduce it, I'd be thrilled to add these fixes. But today I've already spent ~4 hours fighting with emacs and getting nowhere... I hope we can find a solution for this, and really appreciate your help.

@fperez
Owner

One more data point: with your fixes and the bare .emacs file above, %debug now works better, which is great! But it will only track files with %colors nocolor, it seems the ansi color escapes confuse the regexp matcher. I get

error in process filter: not: Symbol's value as variable is void: py-pydbtrack-stack-entry-regexp

and a bunch of naked ansi junk on the debug buffer:

> �[0;32m/home/fperez/scratch/error.py�[0m(45)�[0;36mRampNum�[0;34m()�[0m
�[0;32m     44 �[0;31m    �[0mstep�[0m �[0;34m=�[0m �[0;34m(�[0m�[0mend�[0m�[0;34m-�[0m�[0mstart�[0m�[0;34m)�[0m�[0;34m/�[0m�[0;34m(�[0m�[0msize�[0m�[0;34m-�[0m�[0;36m1�[0m�[0;34m-�[0m�[0mtmp�[0m�[0;34m)�[0m�[0;34m�[0m�[0m
�[0m�[0;32m---> 45 �[0;31m    �[0mresult�[0m�[0;34m[�[0m�[0;34m:�[0m�[0;34m]�[0m �[0;34m=�[0m �[0marange�[0m�[0;34m(�[0m�[0msize�[0m�[0;34m)�[0m�[0;34m*�[0m�[0mstep�[0m �[0;34m+�[0m �[0mstart�[0m�[0;34m�[0m�[0m
�[0m�[0;32m     46 �[0;31m�[0;34m�[0m�[0m
�[0m
ipdb> 

If this can't be fixed that's OK, we can just advertise that to use %debug within emacs, people should simply temporarily turn coloring off.

@fperez
Owner

One more: this small change should probably be applied:

diff --git a/docs/emacs/ipython.el b/docs/emacs/ipython.el
index 1b11e5f..f56a1fd 100644
--- a/docs/emacs/ipython.el
+++ b/docs/emacs/ipython.el
@@ -218,7 +218,7 @@ the second for a 'normal' command, and the third for a multiline command.")
           py-shell-input-prompt-2-regexp "^   [.][.][.]+: *" )
     ;; select a suitable color-scheme
     (unless (delq nil
-                  (mapcar (lambda (x) (eq (string-match "^--colors=*" x) 0))
+                  (mapcar (lambda (x) (eq (string-match "^--colors*" x) 0))
                           py-python-command-args))
       (push (format "--colors=%s"
                     (cond

b/c the = sign is now optional.

@paykroyd

Good feedback. I had sort of forgotten that I had made some changes to my environment just to get it all working together. I'll try to address those problems.

I'm still testing the %debug stuff. I found that with %pdb on it would jump to the right file, but the location in that file was not correct. I'm still working on that.

Based on a note in ipython.el, I also modified the completion code to send the entire line as well the current item being completed and switched it to use get_ipython().complete(). That works nicely, it can now complete things import statements. I still need to make that change for the xemacs completion function.

@fperez
Owner

Awesome! It looks like we're finally getting out of the rut we've been in with emacs support! Many thanks to you (and @defunkt) for pitching in, welcome to the team :)

paykroyd added some commits
@paykroyd paykroyd Gets ipython working with python-mode with little config.
* sets the default value for py-shell-name to the ipython-command
* fixes color scheme setting to set a default value
cbf9083
@paykroyd paykroyd Changes color config detection based on feed back from @fperez. 32a579d
@paykroyd

@fperez, ok I think that I have the config problem fixed. I tested my changes with a vanilla install. One of the problems was that python-mode makes these configuration variables buffer-local. So setting them once isn't always effective.

Here is the complete .emacs configuration that I used to test:

(add-to-list 'load-path "~/.emacs.d/python-mode")
(add-to-list 'load-path "~/.emacs.d/ipython/docs/emacs")
(require 'ipython)

I think the color issues are sorted as well and I incorporated the change that you suggested for detecting whether a color option had been set.

There are still some issues to be sorted with pdb, I think, and with some making completion smoother, but maybe that's a different pull request?

Let me know if you have any issues.

@paykroyd

Oh I forgot to mention, to test manually configuring the parameters to ipython, see the documentation change. Use setq-default instead of setq.

@deftpunk

paykroyd: Given that python-mode makes some variables buffer-local, is it even necessary for ipython.el to do

(require 'python-mode) 

anymore? Is it possible to do

(setq-default somevar somevalue)

on all of those variables directly in ipython.el - this might help with python.el compatibility.

@paykroyd

deftpunk: I think it still needs to require it because ipython.el modifies the py-mode-map as well as a few other variables defined in python-mode.el. I'm pretty green to elisp though, is there a better way?

@jenshnielsen

Seems to work fine for me.

One bug: It used to be possible to complete from a buffer and not only the terminal inside emacs. When I do that now
I get a Buffer *.py has no process error

One other long standing bug that I noticed is that one cannot complete stuff that contains a - because ship-chars-backwards does not skip it. If one changes l.394 to this it works.
(beg (save-excursion (skip-chars-backward "a-z0-9A-Z_./\-" (point-at-bol))
The slash is to escape the - in the regex

@fperez
Owner

Hi folks, I'll try to finish the merge for this one soon, but I wanted to point your attention to #22, which also deals with emacs-related issues. If you have a chance to look at that code and you find anything worth doing with it (presuming the author is willing to relicense it as BSD for ipython), that would be great. And if everything there has already been done, we can simply close that issue.

@fperez fperez referenced this pull request from a commit
@fperez fperez Add '-' to escaped characters for emacs completion.
After feedback from @jenshnielsen in #1015.
9eb1305
@fperez fperez merged commit 32a579d into ipython:master
@fperez
Owner

Guys, I went ahead and merged it because it's already a big improvement over the previous situation. Oddly, tab completion isn't working at all for me. But because everything else is already so much better, I'd rather merge it now. If you guys think you know what could be the problem, we can open another PR for a new fix. I tried with the absolutely minimal .emacs file as noted above by @paykroyd, and still completion doesn't work even for something like 'import'.

But in any case, thanks a lot for the work, things are already in much better shape than before!

@takluyver
Owner

Emacs users: can you have a look at the docs about ipython.el, and update anything that needs it? In particular, I hope the issue with Python 2.2 has been solved now. ;-)

The relevant section is here: https://github.com/ipython/ipython/blob/master/docs/source/config/editors.txt#L40

@paykroyd

@fperez, just to clarify, you were trying to do an import in the ipython shell, right? Were there any messages in the Messages buffer after it failed? If you start to do an import and then type in M-x ipython-complete does it work?

Thanks.

@fperez
Owner

@paykroyd: yes, this is trying to complete by typing 'imp', which should unambigously complete to 'import'.

I tried both ways, with and with M-x ipython-complete, and both cases led to the same message in the *Messages* buffer:

Can't find completion for "imp" based on line imp [2 times]

HTH

@paykroyd

@fperez, that's odd. I was hoping for an error message. This is the line of code that runs in ipython:

print(';'.join(get_ipython().complete('%s', '%s')[1])) #PYTHON-MODE SILENT\n

Where the first string is the current word ("imp") and the next is the whole line of code. Does that API call make sense? Are you on emacs or xemacs? I just found a bug in the xemacs hanlding.

@fperez
Owner
@fperez fperez referenced this pull request from a commit
Commit has since been removed from the repository and is no longer available.
@andreas-roehler

Hi all,
seems something to do still at the python-mode.el side.
Linked a bug-report to this

https://bugs.launchpad.net/python-mode/+bug/934022

Cheers,
Andreas

@andreas-roehler

Hi,

if these kind of things happen, please alert us at
http://launchpad.net/python-mode
Let's prevent the evil at it's source :)

Cheers,
Andreas

@ipstone

hello, is this matter resolved? the ipython.el is indeed outdated, I tried multiple times in the past, and gave up for a while. just tried it again today on 2012-12-11, it froze my emacs 24.2 on windows ... it's a pity, maybe I'll go with vim then...

@tkf

My impression is that ipython.el is not maintained. Both python.el and python-mode.el support IPython now. So I guess there is no need for ipython.el now. If you are using Emacs 24.2, you have the new python.el which supports ipython and ipdb.

@takluyver
Owner
@mattvonrocketstein mattvonrocketstein referenced this pull request from a commit in mattvonrocketstein/ipython
@fperez fperez Add '-' to escaped characters for emacs completion.
After feedback from @jenshnielsen in #1015.
e7650fd
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Commits on Nov 19, 2011
  1. @paykroyd
  2. @paykroyd

    Ensures that ipython-complete is called.

    paykroyd authored
    python-mode defines a key-binding for the tab key for its own
    completion function (py-shell-complete) if that one wins, then
    py-shell-complete will hang emacs as it tries to communicate
    with the python shell process which does not exist. This uses
    def-advice to instead call ipython-complete when py-shell-complete
    is called.
Commits on Nov 20, 2011
  1. @paykroyd

    Switches to use a better completion API and send the whole line.

    paykroyd authored
    * sends the whole line of code to get better results
    * not updated for xemacs yet
Commits on Nov 22, 2011
  1. @paykroyd

    Gets ipython working with python-mode with little config.

    paykroyd authored
    * sets the default value for py-shell-name to the ipython-command
    * fixes color scheme setting to set a default value
  2. @paykroyd
This page is out of date. Refresh to see the latest.
Showing with 41 additions and 26 deletions.
  1. +41 −26 docs/emacs/ipython.el
View
67 docs/emacs/ipython.el
@@ -17,7 +17,8 @@
;; can be used to convert bits of a ipython session into something that can be
;; used for doctests. To install, put this file somewhere in your emacs
;; `load-path' [1] and add the following line to your ~/.emacs file (the first
-;; line only needed if the default (``"ipython"``) is wrong)::
+;; line only needed if the default (``"ipython"``) is wrong or ipython is not
+;; in your `exec-path')::
;;
;; (setq ipython-command "/SOME-PATH/ipython")
;; (require 'ipython)
@@ -36,7 +37,7 @@
;; always in ``pylab`` mode with hardcoded light-background colors, you can
;; use::
;;
-;; (setq py-python-command-args '("--pylab" "--colors=LightBG"))
+;; (setq-default py-python-command-args '("--pylab" "--colors=LightBG"))
;;
;;
;; NOTE: This mode is currently somewhat alpha and although I hope that it
@@ -132,6 +133,12 @@
(require 'shell)
(require 'executable)
(require 'ansi-color)
+;; XXX load python-mode, so that we can screw around with its variables
+;; this has the disadvantage that python-mode is loaded even if no
+;; python-file is ever edited etc. but it means that `py-shell' works
+;; without loading a python-file first. Obviously screwing around with
+;; python-mode's variables like this is a mess, but well.
+(require 'python-mode)
(defcustom ipython-command "ipython"
"*Shell command used to start ipython."
@@ -164,12 +171,8 @@ the second for a 'normal' command, and the third for a multiline command.")
(if (not (executable-find ipython-command))
(message (format "Can't find executable %s - ipython.el *NOT* activated!!!"
ipython-command))
- ;; XXX load python-mode, so that we can screw around with its variables
- ;; this has the disadvantage that python-mode is loaded even if no
- ;; python-file is ever edited etc. but it means that `py-shell' works
- ;; without loading a python-file first. Obviously screwing around with
- ;; python-mode's variables like this is a mess, but well.
- (require 'python-mode)
+ ;; change the default value of py-shell-name to ipython
+ (setq-default py-shell-name ipython-command)
;; turn on ansi colors for ipython and activate completion
(defun ipython-shell-hook ()
;; the following is to synchronize dir-changes
@@ -218,21 +221,24 @@ the second for a 'normal' command, and the third for a multiline command.")
py-shell-input-prompt-2-regexp "^ [.][.][.]+: *" )
;; select a suitable color-scheme
(unless (delq nil
- (mapcar (lambda (x) (eq (string-match "^--colors=*" x) 0))
+ (mapcar (lambda (x) (eq (string-match "^--colors*" x) 0))
py-python-command-args))
- (push (format "--colors=%s"
- (cond
- ((eq frame-background-mode 'dark)
- "Linux")
- ((eq frame-background-mode 'light)
- "LightBG")
- (t ; default (backg-mode isn't always set by XEmacs)
- "LightBG")))
- py-python-command-args))
- (unless (equal ipython-backup-of-py-python-command py-python-command)
- (setq ipython-backup-of-py-python-command py-python-command))
- (setq py-python-command ipython-command))
-
+ (setq-default py-python-command-args
+ (cons (format "--colors=%s"
+ (cond
+ ((eq frame-background-mode 'dark)
+ "Linux")
+ ((eq frame-background-mode 'light)
+ "LightBG")
+ (t ; default (backg-mode isn't always set by XEmacs)
+ "LightBG")))
+ py-python-command-args)))
+ (when (boundp 'py-python-command)
+ (unless (equal ipython-backup-of-py-python-command py-python-command)
+ (setq ipython-backup-of-py-python-command py-python-command)))
+ (setq py-python-command ipython-command)
+ (when (boundp 'py-shell-name)
+ (setq py-shell-name ipython-command)))
;; MODIFY py-shell so that it loads the editing history
(defadvice py-shell (around py-shell-with-history)
@@ -312,7 +318,7 @@ gets converted to:
(replace-match "" t nil)))))
(defvar ipython-completion-command-string
- "print(';'.join(get_ipython().Completer.all_completions('%s'))) #PYTHON-MODE SILENT\n"
+ "print(';'.join(get_ipython().complete('%s', '%s')[1])) #PYTHON-MODE SILENT\n"
"The string send to ipython to query for all possible completions")
@@ -388,6 +394,7 @@ in the current *Python* session."
(beg (save-excursion (skip-chars-backward "a-z0-9A-Z_./" (point-at-bol))
(point)))
(end (point))
+ (line (buffer-substring-no-properties (point-at-bol) end))
(pattern (buffer-substring-no-properties beg end))
(completions nil)
(completion-table nil)
@@ -399,7 +406,7 @@ in the current *Python* session."
(setq ugly-return (concat ugly-return string))
"")))))
(process-send-string python-process
- (format ipython-completion-command-string pattern))
+ (format ipython-completion-command-string pattern line))
(accept-process-output python-process)
(setq completions
(split-string (substring ugly-return 0 (position ?\n ugly-return)) sep))
@@ -409,10 +416,10 @@ in the current *Python* session."
(setq completion (try-completion pattern completion-table))
(cond ((eq completion t))
((null completion)
- (message "Can't find completion for \"%s\"" pattern)
+ (message "Can't find completion for \"%s\" based on line %s" pattern line)
(ding))
((not (string= pattern completion))
- (delete-region beg end)
+ (delete-region (- end (length pattern)) end)
(insert completion))
(t
(message "Making completion list...")
@@ -421,6 +428,14 @@ in the current *Python* session."
(message "Making completion list...%s" "done")))))
)
+;;; if python-mode's keybinding for the tab key wins then py-shell-complete is called
+;;; instead of ipython-complete which result in hanging emacs since there is no shell
+;;; process for python-mode to communicate with
+(defadvice py-shell-complete
+ (around avoid-py-shell-complete activate)
+ (ipython-complete))
+
+
;;; autoindent support: patch sent in by Jin Liu <m.liu.jin@gmail.com>,
;;; originally written by doxgen@newsmth.net
;;; Minor modifications by fperez for xemacs compatibility.
Something went wrong with that request. Please try again.