add --prefer-lowest and --prefer-stable to update command #3450

Merged
merged 3 commits into from Dec 14, 2014

Projects

None yet

6 participants

@nicolas-grekas
Contributor

Run composer update --prefer-lowest-stable and get all the deps installed to their lowest (pretended) supported version

Fixes #3440
Fixes #1902

@stof
Contributor
stof commented Nov 21, 2014

@naderman there is a protected method selectNewestPackages in the DefaultPolicy class, but it looks unused. What is its goal ?

@naderman
Member

@stof used to be a helper method for selectPreferedPackages, I'll delete it as it's no longer used.

@naderman
Member
@nicolas-grekas
Contributor

Please tell me if I should do anything before merge

@stof
Contributor
stof commented Nov 21, 2014

hmm, adding an integration test covering this case to ensure it works ?

@naderman
Member

Only thing I wonder is if we should have a regular prefer lowest without prefer stable? /cc @Seldaek @stof

@stof stof and 2 others commented on an outdated diff Nov 21, 2014
src/Composer/Installer.php
@@ -697,10 +698,14 @@ private function createPolicy()
// old lock file without prefer stable will return null
// so in this case we use the composer.json info
if (null === $preferStable) {
- $preferStable = $this->package->getPreferStable();
+ if (getenv('COMPOSER_PREFER_LOWEST_STABLE')) {
+ $preferStable = $preferLowest = true;
@stof
stof Nov 21, 2014 Contributor

are you sure about $preferStable being true ? this would prefer a 2.4.0 over a 2.4.0-beta release, while 2.4.0 is lowest.
I think it should rather force $preferStable to false, so that only the version ordering is used to find the selected version

@naderman
naderman Nov 21, 2014 Member

Yeah that's what I meant, currently it's called LOWEST_STABLE so that makes sense, but maybe we should support both LOWEST and LOWEST_STABLE?

@nicolas-grekas
nicolas-grekas Nov 21, 2014 Contributor

In fact, I tried without prefer-stable, and I went back in -rc1 or -alpha state, which is certainly not what I'll ever want to test. So, if the only use case of this is testing the lowest version you pretend to support, I'm not sure that anyone will go up to supporting alpha nor even beta versions.

@nicolas-grekas
Contributor

I just added a small unit test. For the integration test, there is none for other ENV vars...

@naderman
Member

well other env vars have a config equivalent, this does not?

@naderman
Member

This should probably also somehow use the new getComposerEnv() to avoid accidentally influencing test behavior with environment variables set in the process. https://github.com/composer/composer/blob/master/src/Composer/Config.php#L306

@nicolas-grekas
Contributor

Rebased to use config->getComposerEnv().
There is no corresponding config option because I followed comments on #3440,
please tell me if I should update.

@naderman
Member

Yes I didn't mean to ask for one, just meant that's why there probably aren't env tests for the others?

@nicolas-grekas
Contributor

I added a line in the travis-ci matrix so that composer itself is tested with its lowest deps.
That made me fix the composer.json.

@nicolas-grekas nicolas-grekas changed the title from add COMPOSER_PREFER_LOWEST_STABLE to add --prefer-lowest-stable to update command Nov 22, 2014
@nicolas-grekas
Contributor

In fact, while trying to correctly test composer with the new flag, I've been hit by the same problem as in #3448 So, I replaced the env var by an option on the update command.
To make tests pass with the new line in the .travis file, I also had to remove phpunit from require-dev, because in fact, your tests relies on an implicit version of phpunit's deps.
Tests are green again now.

@naderman
Member

Err I understood all of that except why you had to remove phpunit from require-dev?

@nicolas-grekas
Contributor

It's because tests do not pass: phpunit itself is broken when setuped with --prefer-lowest-stable

@naderman
Member

Hm I see, but shouldn't we find another solution to that then than remove the dependency on phpunit which we kind of need to test locally?

@naderman
Member

Ok I see what I misunderstood. By functional test earlier we did not mean that you should test composer itself with this feature. While that may be useful that should at least go into a separate PR. I meant a test like one of the tests in https://github.com/composer/composer/tree/master/tests/Composer/Test/Fixtures/installer

@nicolas-grekas
Contributor

I know you didn't mean that :) I updated the composer.json but forced phpunit to has recent enough deps, tests are green now. I still have to write a functional test, thanks for the link.

@naderman
Member

I'd still really prefer if you could move all the travis/phpunit changes into a separate PR.

@nicolas-grekas
Contributor

This one is now ready: tests added, and travis modifications moved to #3476

@naderman
Member

Alright perfect, still leaves the question from earlier of whether this should always be stable versions? I would think if you specify non-stable lower versions as deps then --prefer-lowest should kind of show you that those are a bad idea? So I'd say if we just do --prefer-lowest you can combine that with --prefer-stable if you want to just use the lowest stable version, no need to force it to be stable with --prefer-lowest?

@nicolas-grekas
Contributor

IMHO, there is not other use case: --prefer-stable only would mean getting the latest stable version.
But stating this (stable or dev) should stay the prerogative of the author of the composer.json.
On the other way, not having prefer-stable but having prefer-lowest has no use case, since pre-stable versions are not "stable" on purpose: they are released with no guarantee whatsoever.
Who would want to test against them?

@naderman
Member
naderman commented Dec 1, 2014

The whole point of having a --prefer-lowest is to test against the lowest versions of dependencies you specified. If you specified a dependency on a dev version or an alpha release but your project does not actually work with them, then that means your dependencies are broken. I'm pretty sure that was the point of introducing --prefer-lowest so it seems counter productive to have it somehow not test the lowest depedencies but only stable dependencies.

@naderman naderman and 1 other commented on an outdated diff Dec 1, 2014
src/Composer/Installer.php
@@ -697,10 +699,14 @@ private function createPolicy()
// old lock file without prefer stable will return null
// so in this case we use the composer.json info
if (null === $preferStable) {
- $preferStable = $this->package->getPreferStable();
+ if ($this->preferLowestStable) {
@naderman
naderman Dec 1, 2014 Member

This code doesn't actually get run for current lock file formats?

@nicolas-grekas
nicolas-grekas Dec 1, 2014 Contributor

Not sure to understand what you mean: this option is only for the "update" mode, thus composer.lock should not be used, if that's what you meant

@nicolas-grekas
Contributor

So here is the scenario as I can imagine it:
at some point, you start with a new package, that relies on an other one that has no stable version released. Still you want to test your new package under lowest deps conditions. Thus, what you want to get is the oldest stable for all stable-released deps, and the latest-released version for the still-dev deps.
But once this other dep is released as stable, nobody will ever want to test against pre-stable releases.

Note that while writing this, I see that it's not what I implemented: currently, when no stable version is released, then the first-ever dev version is loaded, which looks wrong.

Can you describe the scenario you're thinking about that would require separating lowest and stable options?

@naderman
Member
naderman commented Dec 1, 2014

I think that there should simply be one option: --prefer-lowest, which tests against the lowest version your package claims to work against. Otherwise your users may end up using those unstable versions expecting them to work. The whole purpose of this feature is to make you aware of problems with the lowest versions. If you think your package does not work with a certain unstable version of another package, then don't add it as a dependency to your composer.json? What I don't see is how it's reasonable to somehow only do this for stable packages. I have no problem with adding that option too, and in that case I would do that by combining ---prefer-lowest and --prefer-stable, but if you want to verify the correctness of dependencies, it shouldn't ignore unstable versions (e.g. RC releases).

@Seldaek
Member
Seldaek commented Dec 1, 2014

Looks ok to me but indeed the patch seen as a whole is a bit messy now between preferLowest, preferLowestStable and preferStable.

I think the stable notion has to go entirely. It should just take the earliest release that matches your requirement, otherwise a dev-master requirement for example would match nothing with --prefer-stable? Sounds wrong.

So +1 for what @naderman said and please also make sure all the param names/properties/getters/... are just called preferStable or equivalent :)

@nicolas-grekas
Contributor

Ok, I see your point, let me try something

@nicolas-grekas
Contributor

So, spending one more hour on this, I still don't see any use case for specifying "prefer-lowest" but not "prefer-stable" together. In fact, and because I fail to imagine a use case, I fear people will start using "prefer-lowest XOR prefer-stable", and you'll get issues and increase support on your side.

Still, I added a commit that splits --prefer-lowest and --prefer-stable so everybody is happy :)

Really, after spending time on it, I see a flaw in your reasoning @naderman that may explain our respective pov: when someone sets @dev to a requirement, it never meant that the very first commit in the git history of that package is supported - but the exact opposite: only the very latest commit is supported.
This is very different from a "~2.3" which states explicitly that "2.3.0 and more" is supported.

Did you checkout my branch and run it against e.g. composer and/or symfony? If not, then you really should. That's the experimental argument that made me understand and have my current way of thinking.

I'd be happy to remove this last commit if you come to the same conclusion as mine. Or you can merge as is now if you prefer.

@dbu
Contributor
dbu commented Dec 2, 2014

if my only dependency is "@dev" or something equally funny, then i can
install with any version. if i use some library with "
@dev", but my
project asks for "0.1.*@dev" i will end up with some old version (assume
there was never a stable 0.1 release for example.

@docteurklein docteurklein referenced this pull request in phpspec/phpspec Dec 2, 2014
Closed

Refactor/bump php symfony version #571

@nicolas-grekas nicolas-grekas changed the title from add --prefer-lowest-stable to update command to add --prefer-lowest and --prefer-stable to update command Dec 13, 2014
@Seldaek
Member
Seldaek commented Dec 13, 2014

@nicolas-grekas could you give an example of a package where it didn't do the right thing for you? Because even with @dev it wouldn't install the oldest commit since it has no knowledge of all commits, if a library has: 1.0, 1.1, 1.2.x-dev (master) as versions, the lowest, even with @dev, is 1.0.

@nicolas-grekas
Contributor

Of course: any Symfony component with dependencies.
Here is for DependencyInjection when adding --prefer-stable after a first run with only --prefer-lowest:

> composer update --prefer-lowest --prefer-stable
Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Updating symfony/expression-language (v2.4.0-BETA1 => v2.4.0)
    Checking out 54ee1629680bf4313412cef284c6b19fd64c7a29
@Seldaek
Member
Seldaek commented Dec 13, 2014

Yes that makes sense, you use the first beta available as that is what a ~2.4 requirement specifies (any 2.4..). It may not be what everyone wants but strictly speaking that's what prefer-lowest should do IMO. If you use it you get what you ask for. But it probably makes sense to combine with --prefer-stable to most people :)

@nicolas-grekas
Contributor

Cool, then we can merge as is?

@Seldaek Seldaek and 1 other commented on an outdated diff Dec 13, 2014
src/Composer/Installer.php
@@ -700,10 +703,11 @@ private function createPolicy()
// old lock file without prefer stable will return null
@Seldaek
Seldaek Dec 13, 2014 Member

Would you mind adding prefer-lowest support for the lock file? I understand it's probably not a common use case or none at all but for completeness and to avoid bad surprises it should be persisted to the lock. See a few lines above..

@nicolas-grekas
nicolas-grekas Dec 13, 2014 Contributor

Hum, not sure to understand... Can you drive me a bit more please?

@Seldaek
Seldaek Dec 13, 2014 Member

Sure, basically the same as https://github.com/composer/composer/pull/3101/files + nicolas-grekas@4bd748b which fixed some mistakes :)

@Seldaek
Seldaek Dec 13, 2014 Member

Except preferLowest isn't on the root $package so you'll have to pass it in to the locker directly I guess.

@Seldaek
Member
Seldaek commented Dec 13, 2014

Yes except for the lock support sorry I missed that last time :)

@dbu
Contributor
dbu commented Dec 13, 2014

Yes that makes sense, you use the first beta available as that is what a ~2.4 requirement specifies (any 2.4..). It may not be what everyone wants but strictly speaking that's what prefer-lowest should do IMO. If you use it you get what you ask for. But it probably makes sense to combine with --prefer-stable to most people :)

If my bundle claims to work with 2.4.0-alpha1 or later, by using ~2.4 then it should be tested with that. so i notice that i should have restricted that to whatever version i am actually compatible with.

@Seldaek
Member
Seldaek commented Dec 13, 2014

@dbu I agree on principle but in effect nobody is likely to use your bundle with 2.4.0-alpha1 once a 2.4.0 stable is out, but ~2.4 will allow the alpha still, unless you do ~2.4-stable. Under normal conditions composer will take the stable if it's there anyway so I would say for "compat testing" --prefer-stable --prefer-lowest sounds reasonable (you'll catch problems but not waste time debugging issues occuring only with alpha/beta versions of your deps).

@dbu
Contributor
dbu commented Dec 13, 2014

okay. did not want to argue against the combination of --prefer-stable and --prefer-lowest, only that --prefer-lowest alone should install an alpha or beta.

@Seldaek
Member
Seldaek commented Dec 13, 2014

Ah ok yes for sure :)

@nicolas-grekas
Contributor

prefer-lowest is now persisted in composer.lock

@alcohol
Member
alcohol commented Dec 13, 2014

Can you be sure to update doc/03-cli.md and doc/04-schema.md?

@nicolas-grekas
Contributor

CLI doc updated (but there is no schema change for composer.json)

@Seldaek
Member
Seldaek commented Dec 14, 2014

Great, looks very complete now, thanks @nicolas-grekas!

@Seldaek Seldaek merged commit 901fd83 into composer:master Dec 14, 2014

1 check passed

continuous-integration/travis-ci The Travis CI build passed
Details
@nicolas-grekas nicolas-grekas deleted the nicolas-grekas:prefer-lowest-stable branch Dec 14, 2014
@fabpot fabpot added a commit to symfony/symfony that referenced this pull request Dec 15, 2014
@fabpot fabpot minor #12542 Test components using their lowest possible deps (nicola…
…s-grekas)

This PR was squashed before being merged into the 2.3 branch (closes #12542).

Discussion
----------

Test components using their lowest possible deps

| Q             | A
| ------------- | ---
| Bug fix?      | no
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | -
| License       | MIT
| Doc PR        | -

Once composer/composer#3450 is merged, we'll see if our deps are correct, and ensure that is kept over time by testing each component with its lowest possible deps.

Commits
-------

25fef27 Test components using their lowest possible deps
206ebc7
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment