upgrade fixes

[jimfulton]

We often check for new buildout and setuptools versions before deciding that we're not able to upgrade. This is wasteful.

• We should decide sooner that we can't upgrade and not check for new versions if we can't.

• We should not be willing to upgrade any eggs if the existing versions are outside the eggs directory. This should fix Buildout infinite subprocess recursion #312.

• We should consider buildout and all of its dependencies (buildout, setuptool and all of the new and future (wheel) setuptools dependencies.

[leorochael]

What if, during a buildout run, we only check dependencies of buildout itself and bail if they don't match?

We can tell the user to run buidout bootstrap to fix this situation and let bootstrap deal with making sure the right versions of buildout and dependencies are set.

This way we never upgrade nor restart buildout during its run. If the user wants to upgrade buildout or dependencies she can run buildout bootstrap again with unpinned versions.

This means bootstrap would have to read the buildout configuration, to download the correct versions of any pinned dependencies.

[jimfulton]

@leorochael so if someone updates a buildout dependency and it's not pinned, buildout should fail? I don't like that idea. :) It also doesn't really solve a problem afaict. After all, that upgrade could fail, just as current ones do sometimes and the user is just as stuck as they are now.

The new docs encourage people to not use a local buildout (and explain the tradeoff involved). What's emerging is an explicit way to control automatic upgrade:

• If you don't want auto-upgrade, use a non-local buildout. This is what I do, mainly because I'm too lazy to bootstrap my projects. (I'm veeeery lazy.) I only feel upgrading breakage when I read about it in the news. :)

• If you want the latest of everything, or want to pin everything, use a local buildout and let buildout make it so.

[mdeguzis]

Would this fix situations where something requires setuptools in a high version?

Error: There is a version conflict. 
We already have: setuptools 0.9.8 
but zc.buildout 2.9.2 requires 'setuptools>=8.0'

Seems like when the latest setuptools is used, it often breaks things. I guess I can periodically review the pinned setuptools version if I am going to use newest = true in my buildout.cfg.

[jimfulton]

@mdeguzis I don't understand what scenario you're describing.

[mdeguzis]

If I have newest = true and do not pin setuptools, eggs / extensions like Mr. Developer fail the deployment since they request other versions it seems:

Log: https://gist.github.com/mdeguzis/2452391c0fbc6f4717268f41224f7f83
buildout.cfg: https://gist.github.com/mdeguzis/abf18dc9dd1c65d6d7ba411c85187cbb

[jimfulton]

Ah, thanks for reminding me. :)

An issue with extensions like mr.developer, is that they're loaded before upgrading. Upgrading conforms buildout to its requirements, but extensions are loaded before it has a chance to do that.

I suspect we should load extensions after upgrading.

[jimfulton]

Note that currently, when doing a local bootstrap (from a virtualenv or local buildout), the paths to buildout and its dependencies come from the original locations, which means that buildouts bootstrapped this way would never be upgraded. This suggests to me that bootstrapping should make sure it installs the buildout script using buildout-local eggs.