Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

slow in large files #34

Closed
davidhalter opened this Issue · 37 comments

7 participants

Dave Halter Gilson Filho Edmond Burnett Wladston Viana Ferreira Filho Edd Barrett Hugo Oliveira Florian Bruhin
Dave Halter
Owner

When I install the plugin and open a file in python + - 400 lines, when it gets slow when I start typing. And when I type the point, it takes about 2 seconds to appear oa listing.

Remembering that this was a project in Django.

also in: davidhalter/jedi-vim#20

from @gilsondev (in another issue)

Dave Halter davidhalter referenced this issue in davidhalter/jedi-vim
Closed

jedi-vim slow in large files. #20

Dave Halter
Owner

If you want a quick solution for the typing issue, just disable: let g:jedi#show_function_definition = 0 (in jedi-vim).
The completion is slow in certain places, I really don't like it in certain places. I'm working on solutions.

If you can give me examples of slow code I would really be interested. (I haven't seen to much, beside my Jedi library, which is awfully recursive).

Gilson Filho

It happened in the cases that I was encoding a query with the Django ORM.

Happens is that as it has several methods (taking the slow typing), it takes about 1 second to display the list.

Another situation that makes it slow is charging the docstring of some class and method being entered. It has an option to disable for a while, until you see the reason for this slowness?

Dave Halter
Owner

Huge files should be much faster now, at least for the function definition display. Completion may still be a problem, but I'm working on it (It remains in the dev branch for now, because I don't know how stable it is, but you may try it of course).

Basically you can disable:

let g:jedi#show_function_definition = 0
let g:jedi#popup_on_dot = 0

for the time being. This way, nothing starts automatically. Your vim will only be slow if you explicitly start the completion.

Dave Halter
Owner

Awesome! The last few changes really improved this situation a lot! I'm typing/completing in a 14k file (wx._core) and it's just really really fast!

@gilsondev Could you please checkout the current revision (dev branch) and tell me if it's still slow?

Gilson Filho
Dave Halter
Owner

@gilsondev Could please send me a "slow" file? Doesn't have to be public, you also send me an email.

Otherwise I need to know the structure of this file. This means how big is your file, how big are your classes, functions, where you are trying to use completion, etc.

Edmond Burnett

I'm getting 1-2 second pauses when in insert mode (Vim) and moving over the isupper(), lower(), and upper() Python string methods. The file does not have to be large. See example code:

https://gist.github.com/4383325

Dave Halter
Owner

@edburnett What do you mean by "moving over"? I cannot reproduce the slowness.

Edmond Burnett

By 'moving over' I mean using the arrow keys while in insert mode to navigate through the line. When the cursor arrives at the parentheses of those methods then it pauses/slows down.

I know navigating while in insert mode is not considered proper usage of Vim but it sounded similar to this bug so I thought they might be related.

Dave Halter
Owner

I know now what you mean (and there is a certain lag sometimes [not on every move, though]). I'll try to fix it, but you have to give me some time. I know where it comes from and it's not that easy to fix :-)

Edmond Burnett

No worries, thanks for the great plugin. :)

Wladston Viana Ferreira Filho

I'm also affected by the slowness in many situations. Even if the file I'm editing is small, if the function comes from an external lib, it usually will slow down while on insert mode, right after I type in the "(".

doing the "let g:jedi#show_function_definition = 0" for now.

Btw, that's a great plugin, congratz!

Dave Halter davidhalter referenced this issue in davidhalter/jedi-vim
Closed

slowdown of typing when inside x[] #95

Edd Barrett

If i type "sys." for the first time, I have to wait about 5 seconds. This is on a 2GHz machine.

Have you considered some kind of pre-compiled index of the common modules? os, sys, ...

Maybe such an index could be built on first run.

Dave Halter
Owner

@vext01 You're on a Windows machine?

This index already exists. But that version (feature) is not available with pip, yet. Just try to use the master branch of github (but it's not as stable as I'd expect it to be).

Edd Barrett
Dave Halter
Owner

@vext01 Is it faster on dev head?

Edd Barrett
Hugo Oliveira

Hello,

i can confirm @vext01 problem too, @davidhalter. The problem is worse if you are closing the file and opening other vim (quit vim and reopening) session...probably some missing cache feature around some packages.
open a file test.py:

import numpy as np
a = np. here it takes a good 9s on my laptop for options to appear and vim usage goes 100% cpu
a = np.zeros( here it show the dialog stuff

if we close the file and open again, +9 seconds to reload the numpy methods and etc.

Same for os, system, scipy, etc...

my laptop is a i3-350m, and sometimes take a very long time to load all the imports.

Maybe a favorite option to cache import classes and etc...or better an auto-adjust base on usage...or faster loadings :P

PS: tested on both master and dev.

Dave Halter
Owner

@ocehugo @vext01 This issue is now my main work again.

Dave Halter
Owner

@gilsondev Could you please check the latest master branch (jedi version 0.6) and check if it's still slow?

@ocehugo: The following should now be pretty fast: Loading VIM, completing numpy (as in your examples) closing VIM and then opening/completing again. The second time it should be fast, something like 1.6 seconds on my 4 year old notebook.

@vext01: sys loads on my 4 year old notebook very fast (< 0.5s). But this is probably only true after the second time reopening VIM.

Gilson Filho
Dave Halter
Owner

@gilsondev Sure, no problem!

Dave Halter
Owner

From what I've heard, this doesn't seem to be a problem anymore. Jedi seems to be a lot faster now. If anybody doesn't agree, I can still reopen. But I'll close for now.

Dave Halter davidhalter closed this
Wladston Viana Ferreira Filho

It's indeed way faster now, but the problem still occurs for me when editing source files with over 3k+ lines.

Dave Halter
Owner

@wladston Oh, that might be true. Could you send me one? Because that seems awfully unpythonic :-). I still have room for a few optimizations, but I don't really have a "slow" file at hand, which is being edited by humans and not just computer generated (real life use case).

Hugo Oliveira

@davidhalter, thankz for that. The autocomplete loading are pretty fast now.

Actually some of my problems was inside the filesystem container. After i start a preload/cache service of some python/vim files the performance improves a lot (Including vim startup with a lot of bundles).

Wladston Viana Ferreira Filho

I think this would be a nice test case :

https://github.com/django/django/blob/master/django/forms/fields.py

it has over 1k size, it should make typing slow if using the dot popup and the function definition helper.

Dave Halter
Owner

@wladston Well that's the thing, this file (and others of this size) is pretty fast. I don't have any lag problems. Do you have problems in there?

If so: What operating system, what hardware?

Wladston Viana Ferreira Filho

Indeed, I tried out with the django sources and it worked very well. I'm on a Macbook air / Mac mini, I don't think hardware is the issue ... I'll try again with source files from work and will see if I can produce a test case.

Dave Halter
Owner

@wladston Look out for either: Long files or long sections without breaking functions/classes. That's what I suspect.

Wladston Viana Ferreira Filho

We have classes with over 1k lines, definition of lists with over 200 lines, source files with over 3k lines.... many includes, some includes have sources with over 2/3k lines... I wasn't able to produce a similar test file that slows down jedi, but it is still slow to the point of being unusable with these sources...

I noticed that the greatest slow downs happen when I type dot after typing an included object. It's fast with the builtins.

Hugo Oliveira

@wladson, did you see if your HD I/O are runing up to 100% when loading this files!? I say that because 50% of my problem with jedi auto loading was related to I/O issues over large files and a lot of small ones( and of course an already know little bug with my file system). Maybe you should do a test with preload of the request files via a init.d service or before editing the sources.

Example:
my_packages ::: array of paths for preloading

my_packages=("$HOME/.vim" "/usr/lib64/python2.7/site-packages/numpy" "yourprojectfolder" "yoursourcepy")
for i in ${my_packages[@]}; do
echo "Preloading ${i}"
find ${i} -xdev -type f -exec cat '{}' \; > /dev/null
done

As long you dont clean the cache, the files will remains on ram. Try this before casting vim & your source file...after that if typing "." for methods is still slow, well, probably it's jedi-vim fault :P.

PS: It's important to preload all packages that are imported in ur python files to make sure that I/O it's not bounding your performance.
PS2: you can put this tiny hack on a startup script or use preload from sf.net.

Wladston Viana Ferreira Filho

@ocehugo I ended up quitting my job at the company (I dislike working with huge files), so currently I don't have access to those files to test anymore ...

Anyways, thanks a lot, your post will be a good guidance for other programmers that might run into this problem.

Dave Halter davidhalter referenced this issue in davidhalter/jedi-vim
Open

jedi-vim causes lag in insert mode #217

Gilson Filho
Florian Bruhin

With http://git.the-compiler.org/qutebrowser/tree/qutebrowser/config/configdata.py?id=de2f26760711655ad2c6e8faddbec702b5da3874 vim gets unusably slow, especially when typing inside some of the description strings. This is only 1000 lines though, and a rather fast machine (i7, SSD).

Gilson Filho
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.