Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Poor performance of pip.installed when given a list of packages #47689
Description of Issue/Question
When using pip.installed to manage the presence of a list of packages, the performance is quite poor, with each package in the list adding ~1-2s to the runtime of the state.
This is due to the re-execution of "pip freeze" for each package, which is an expensive operation. It could potentially be avoided by running "pip freeze" once and storing the output for re-use, refreshing it only as required.
Start from a CentOS7 system. Install pip and salt 2017.7.
Note - the issue is also present in 2018.3 and dev.
Set up a pip state to install the current python pip packages:
Steps to Reproduce Issue
Apply the state - note that "pip freeze" is called once for each package.
Run time is about 26s:
Example patch that shows an improvement is included below, as well as debug level logs. With that change, run time is about 1.2s:
Proof of concept patch
This proof of concept patch loads the full pip list only once when checking if packages are installed.
It does not use the
Debug level logs
after applying the above change: