entry_points/console_scripts is too slow

[gwk]

BPO	33902
Nosy	@warsaw, @ericvw, @gwk

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2018-06-19.20:00:40.510>
created_at = <Date 2018-06-19.14:34:17.805>
labels = []
title = 'entry_points/console_scripts is too slow'
updated_at = <Date 2018-06-19.20:07:46.611>
user = 'https://github.com/gwk'

bugs.python.org fields:

activity = <Date 2018-06-19.20:07:46.611>
actor = 'gwk'
assignee = 'none'
closed = True
closed_date = <Date 2018-06-19.20:00:40.510>
closer = 'barry'
components = []
creation = <Date 2018-06-19.14:34:17.805>
creator = 'gwk'
dependencies = []
files = []
hgrepos = []
issue_num = 33902
keywords = []
message_count = 5.0
messages = ['319971', '319989', '319994', '319995', '319996']
nosy_count = 3.0
nosy_names = ['barry', 'ericvw', 'gwk']
pr_nums = []
priority = 'normal'
resolution = 'third party'
stage = 'resolved'
status = 'closed'
superseder = None
type = None
url = 'https://bugs.python.org/issue33902'
versions = []

[gwk]

On my newish macOS laptop using Python 3.6 or 3.7, a no-op script takes 3 times as long to invoke using the entry_points machinery as it does to invoke directly. Here are some exemplary times (best times after several tries).

$ time python3.6 entrypoint.py 

real 0m0.047s
user 0m0.033s
sys 0m0.010s

$ time python3.6 /Library/Frameworks/Python.framework/Versions/3.6/bin/entrypoint

real 0m0.153s
user 0m0.129s
sys 0m0.020s

In this example, entrypoint.py consists of def main(): pass; the second script is that generated by pip3 install .... I have confirmed that the -e flag to pip is not the culprit.

I look at a cprofile trace of the invocation and I do not see an obvious single culprit, but I'm not an expert at reading such traces either.

I know that there has been some previous debate about "how slow is slow", whether to try to improve import times, etc. I'm not trying to fan any flames here, but from a practical perspective, this discrepancy is showing up in developer tools that I'm writing, and is driving me to abandon entry_points in favor of less standard, less cross-platform install hackery in my setup.py scripts. An extra 0.1s per invocation is visibly detrimental to shell scripts calling out to installed python programs.

[warsaw]

This really isn't enough information to know exactly what's going on in your case, but there are things that are known to be slow for CLI startups. Probably the biggest contributor is pkg_resources. Of course, that's a third party library that's popular for handling entry points, so it's not really a Python problem. There are some other third party libraries on PyPI that may be more performant for you. Entry point handling is something that I'd like to try to address in 3.8 in a manner similar to importlib.resources.

[gwk]

Thanks Barry. My question then is, what relationship does cpython have with pip, setuptools, distutils and pkg_resources? Since pip comes bundled with Python now it seems a little bit closer than "3rd party". Thanks!

[warsaw]

"It's complicated" :) But technically speaking they are all separate projects, so while there is synergy, CPython itself *imports* some of those to provide various tasks, but it doesn't lead their development. You'll find a lot of the same players in all those projects though.

This might be a good resource: https://www.pypa.io/en/latest/

I hope you don't mind, but I am going to close this issue since it's not directly related to Python itself. If you feel otherwise and/or can provide a reproducible test case, please do reopen it.

Thanks for your contribution to Python!

[gwk]

OK, thanks. I agree that this is best pursued with the developers of the relevant modules. I appreciate your quick and detailed responses!