imenu is way too slow

[asmeurer]

For a reasonably sized Python file (for example, a single class with 100 methods, where the file is about 1500 lines long), it takes 3 seconds to run python-imenu-prev-index-position with emacs -Q (according to elp), and a full imenu takes over 15 seconds with my complete configuration (clearly there's something else going on in my complete configuration, but this is a good place to start).

Here's the profiling results for my complete profile

python-info-current-defun                      260         16.116206999  0.0619854115
python-nav-end-of-defun                        264         9.3584680000  0.0354487424
python-imenu-prev-index-position               131         8.1096539999  0.0619057557
python-nav-end-of-statement                    40950       6.6539469999  0.0001624895
python-nav-beginning-of-defun                  15462       6.4952520000  0.0004200783
python-nav--beginning-of-defun                 15462       6.4294110000  0.0004158201
python-syntax-context-type                     63184       3.6582260000  5.789...e-05
python-util-forward-comment                    41214       2.2133439999  5.370...e-05
python-info-line-ends-backslash-p              41222       2.1830099999  5.295...e-05
python-info-looking-at-beginning-of-defun      46916       1.3367139999  2.849...e-05
python-nav-beginning-of-statement              272         0.0217740000  8.005...e-05
python-syntax-stringify                        16          0.000931      5.81875e-05
python-nav-backward-block                      6           0.000856      0.0001426666
python-nav-forward-block                       6           0.000839      0.0001398333

[fgallina]

Fixed in revno 112328 in Emacs bzr trunk.

[asmeurer]

How long does that take to get updated here?

[fgallina]

I tried to merge this quickly, but unfortunately the merge was not straightforward and I'm not in the mood to fight it.

I'm seriously thinking into dropping github to mirror python.el as there's a git mirror[0] for Emacs source repository available which should suit most people needs. It's updated frequently (so frequently that the change is already reflected there).

Latest from emacs-24 branch: http://repo.or.cz/w/emacs.git/blob_plain/refs/heads/emacs-24:/lisp/progmodes/python.el
Latest trunk version can be found: http://repo.or.cz/w/emacs.git/blob_plain/HEAD:/lisp/progmodes/python.el

[0] http://repo.or.cz/w/emacs.git

[asmeurer]

If you do that, where would issues and pull requests go? To the main emacs?

[fgallina]

I don't care much about pull requests, after all, everything's a patch.

I'd prefer if we use just one bug tracker: GNU Emacs' one. I'll be defining all this stuff in the next days and update the Readme accordingly.

[asmeurer]

I'm not a fan of the new behavior. It only shows the top level functions and classes by default, which isn't helpful to me. I want to quickly jump to a function definition using ido imenu, without having to type the name of the class it sits in first.

[tkf]

FYI emacs-jedi has imenu support http://tkf.github.io/emacs-jedi/#jedi:install-imenu for a while. But it is still experimental part of emacs-jedi since I am not regularly using it and it has no test. In theory it should be able to generate better imenu index because it can tell you that both f = lambda... and def f()... define function, etc.

[asmeurer]

Thanks. For reference, I had to do sudo emacs-jedi/env/bin/python jedi/setup.py install (for some reason, it needed sudo), having already done make requirements, and then add (setq jedi:install-imenu t). There seem to be some bugs with it, though. Should I report them to jedi or emacs-jedi?

[tkf]

Yes, emacs-jedi's tracker is better place to discuss the topic.

[fgallina]

The python.el present at the emacs trunk[0] does provide faster imenu implementations:

• python-imenu-create-index* (creates a nested tree)
• python-imenu-create-flat-index** (provides a flat version of imenu, ido friendly).
• http://bzr.savannah.gnu.org/lh/emacs/trunk/annotate/head:/lisp/progmodes/python.el#L3139
  ** http://bzr.savannah.gnu.org/lh/emacs/trunk/annotate/head:/lisp/progmodes/python.el#L3152

You can grab it from: http://bzr.savannah.gnu.org/lh/emacs/trunk/download/head:/python.el-20091113204419-o5vbwnq5f7feedwu-3008/python.el

[asmeurer]

How do I set imenu to use the flat version? I put (setq imenu-create-index-function 'python-imenu-create-flat-index) in my .emacs, but it still uses python-imenu-create-index.

[fgallina]

@asmeurer:

(add-hook 'python-mode-hook 
          (lambda () 
            (set (make-local-variable 'imenu-create-index-function)
                 #'python-imenu-create-index)))

[asmeurer]

With that I get Symbol's definition is void: python-imenu-create-index.