Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Speedup startup by beeing lazy. #11965

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

Carreau
Copy link
Member

@Carreau Carreau commented Nov 25, 2019

A significant chunk of the startup time is spent loading pygments
lexers.

This make it lazy which slightly improve the hot and cold startup time.

time ipython -c exit() cold goes from ~0.7 to ~0.65 seconds on my work
laptop I expect much more on old or slower machines.

A significant chunk of the startup time is spent loading pygments
lexers.

This make it lazy which slightly improve the hot and cold startup time.

time `ipython -c exit()` cold goes from ~0.7 to ~0.65 seconds on my work
laptop I expect much more on old or slower machines.
@Carreau
Copy link
Member Author

Carreau commented Nov 26, 2019

I realized that on my machine about 1/2 the startup time comes from the setuptools's installed ipython's script that loads pkg_resources, and then loads entry points.

@takluyver you are our installer expert; any idea ?

@takluyver
Copy link
Member

Since a couple of years ago, pygments should lazy-load pkg_resources only if you try to list all lexers/formatters/styles/filters:
https://bitbucket.org/birkenfeld/pygments-main/pull-requests/690/delay-loading-pkg_resources-until-it-is/diff

If we can avoid triggering that kind of listing, we shouldn't need an extra lazy-loading layer on top of it.

As for the script: if you install with pip (and not the -e flag), you should get a script that doesn't use pkg_resources. pip creates the simpler scripts with distlib, rather than setuptools. For this reason, I tend to create my own developer installs by doing pip install ., then manually replacing the installed folder in site-packages with a symlink. Or I use Flit, of course. ;-)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants