Skip recreating the virtualenv if it already exists

[edmorley]

Hi!

I noticed looking at the build log, that even with a cached build directory, time was wasted trying to recreate the virtualenv.

For example from this log, there is:

$ python2.7 -mvirtualenv --no-site-packages --no-download /home/docs/checkouts/readthedocs.org/user_builds/treeherder/envs/latest 

New python executable in /home/docs/checkouts/readthedocs.org/user_builds/treeherder/envs/latest/bin/python2.7
Also creating executable in /home/docs/checkouts/readthedocs.org/user_builds/treeherder/envs/latest/bin/python
Installing setuptools, pip, wheel...done.

 Command time: 2s Return: 0 

Whilst it's only a few seconds, it would still be good to avoid recreating the virtualenv if not necessary (if needed a python/pip version check could be added to ensure they get upgraded if needed).

(Reducing the end-to-end time will hopefully avoid the problems we're seeing with jobs overlapping and hitting #2901)

Many thanks :-)

[safwanrahman]

There are some problem about removing the existing package that are not in your requirements.
Like, foo is installed and later removed from the requirements. But if the virtualenv does not recreate, foo will be in the virtualenv. In this way, if a broken package come to the vritualenv, it will be hard to remove that!

[edmorley]

Oh I thought that the installed packages were already being re-used (otherwise why is there the clear cache feature?), and so this issue was about stopping the virtualenv being reinstalled over the top of an existing dirty virtualenv. However from your comment it seems that the virtualenv isn't ever re-used at all?

[stsewd]

The cache is saved on another directory https://github.com/rtfd/readthedocs.org/blob/f22289a30b909a61c6d498d6faf4e90b0824b4ed/readthedocs/projects/models.py#L468-L473

[stsewd]

I think this feature is not needed if the cache directory is outside the virtual environment, right?

[stsewd]

Closing this as I think is not needed because we already cached the packages installed in the virtual environment and we can't reuse the virtual environment.