Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement periodic update feature #1841

Merged
merged 9 commits into from
Jun 21, 2020
Merged

Conversation

gaborbernat
Copy link
Contributor

@gaborbernat gaborbernat commented May 28, 2020

  • print out seeded versions in report
  • periodically check in a background out of band process if the embed
    wheels are out of date
  • promote app data at root level, with three implementations - via disk
    folder, via temp folder and via not available
  • introduce a common wheelhouse to not replicate wheels accross python
    versions, decreasing size of the app data folder
  • allow manually upgrading the embeded wheels by using the
    --upgrade-embed-wheels flag, in this case no virtual environment is
    created
  • upgrade setuptools to 47.1.0

Resolves #1821.
Resolves #1840.

@codecov
Copy link

codecov bot commented Jun 4, 2020

Codecov Report

Merging #1841 into master will increase coverage by 4.99%.
The diff coverage is 93.12%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1841      +/-   ##
==========================================
+ Coverage   88.84%   93.84%   +4.99%     
==========================================
  Files          82       86       +4     
  Lines        3946     4159     +213     
==========================================
+ Hits         3506     3903     +397     
+ Misses        440      256     -184     
Flag Coverage Δ
#tests 93.84% <93.12%> (+4.99%) ⬆️
Impacted Files Coverage Δ
src/virtualenv/config/cli/parser.py 95.60% <ø> (-0.05%) ⬇️
src/virtualenv/create/pyenv_cfg.py 97.87% <ø> (ø)
src/virtualenv/run/plugin/discovery.py 100.00% <ø> (ø)
src/virtualenv/run/session.py 98.27% <ø> (ø)
...ualenv/seed/embed/via_app_data/pip_install/copy.py 100.00% <ø> (ø)
...env/seed/embed/via_app_data/pip_install/symlink.py 93.18% <ø> (ø)
src/virtualenv/util/error.py 100.00% <ø> (ø)
...virtualenv/seed/embed/via_app_data/via_app_data.py 79.54% <79.54%> (ø)
src/virtualenv/app_data/na.py 81.08% <81.08%> (ø)
src/virtualenv/app_data/via_disk_folder.py 84.54% <84.54%> (ø)
... and 40 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f99353c...36ba4c5. Read the comment docs.

@gaborbernat gaborbernat force-pushed the bundle branch 2 times, most recently from c543ad2 to 243b2ec Compare June 6, 2020 12:53
setup.py Show resolved Hide resolved
gaborbernat and others added 5 commits June 13, 2020 08:18
- print out seeded versions in report
- periodically check in a background out of band process if the embed
  wheels are out of date
- promote app data at root level, with three implementations - via disk
  folder, via temp folder and via not available
- introduce a common wheelhouse to not replicate wheels accross python
  versions, decreasing size of the app data folder
- allow manually upgrading the embeded wheels by using the
  --upgrade-embed-wheels flag, in this case no virtual environment is
  created
- upgrade setuptools to 47.1.0

Signed-off-by: Bernat Gabor <bgabor8@bloomberg.net>

Signed-off-by: Bernat Gabor <bgabor8@bloomberg.net>
Signed-off-by: Bernat Gabor <bgabor8@bloomberg.net>
Signed-off-by: Bernat Gabor <bgabor8@bloomberg.net>
Signed-off-by: Bernat Gabor <bgabor8@bloomberg.net>
Co-authored-by: Pradyun Gedam <pradyunsg@gmail.com>
Signed-off-by: Bernat Gabor <bgabor8@bloomberg.net>
Signed-off-by: Bernat Gabor <bgabor8@bloomberg.net>
Signed-off-by: Bernat Gabor <bgabor8@bloomberg.net>
Signed-off-by: Bernat Gabor <bgabor8@bloomberg.net>
@gaborbernat gaborbernat force-pushed the bundle branch 4 times, most recently from e6246e1 to e83f776 Compare June 20, 2020 16:27
Signed-off-by: Bernat Gabor <bgabor8@bloomberg.net>
@gaborbernat
Copy link
Contributor Author

This is now ready to go. I expect to merge it over the weekend and ship it on Monday morning.

@gaborbernat gaborbernat merged commit 0cd009b into pypa:master Jun 21, 2020
@gaborbernat gaborbernat deleted the bundle branch June 21, 2020 07:29
@dstufft
Copy link
Member

dstufft commented Jun 22, 2020

If I'm reading this correctly, this will download the wheel but hold onto it for 28 days or so before using it? Is that right?

@gaborbernat
Copy link
Contributor Author

If it's downloaded automatically hold onto it until it becomes 28 days old release (one hour after download). You can always make it use right away by manually invoking the upgrade process via the flag.

@dstufft
Copy link
Member

dstufft commented Jun 24, 2020 via email

@gaborbernat
Copy link
Contributor Author

Would it be useful if the simple index included the release date to use that instead of downloading and holding onto it?

It would be useful, though I'd guess we'd still hold onto it, but we'd not need to go to the PyPi API to fetch it. That being said we'd likely need to keep around the current implementation until the simple index is able to provide this information. Alternatively, it would also work if we could extract this information from the wheel/sdist itself somehow.

@asottile
Copy link
Contributor

I'm seeing breaking (presumably) due to this change. virtualenv is no longer reproducible and I've only just noticed setuptools>=50 despite not changing my virtualenv version.

I think this should be reverted back to when virtualenv was reproducible by default, we're not making the situation better for edge-breaking and in may ways it's more confusing now that it's delayed by several weeks.

@gaborbernat
Copy link
Contributor Author

This topic has been over discussed, and both sides made their point. There has been a majority desire both on pypa and general public side of some form of auto upgrade, which has been implemented here. There are configurations available to achieve the old behavior, but we don't plan to revert this feature. (For what it's worth I was also against this feature personally, but I consider it a decent middle ground over the old always update behavior).

@pypa pypa locked and limited conversation to collaborators Sep 15, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Plug-in for bundled wheels Ensure bundled pip does not get too out of date
5 participants