Can not require package, even with update-with-all-dependencies

[schmunk42]

My composer.json:

{
    "name": "neoacevedo/openbiblio2",
    "description": "Sistema automatizado de gestión bibliotecaria y OPAC.",
    "keywords": ["yii2", "opac", "openbiblio2", "online library project"],
    "homepage": "https://www.neoacevedo.co/openbiblio2",
    "type": "project",
    "license": "GPL-3.0-or-later",
    "support": {
        "issues": "https://github.com/neoacevedo/openbiblio2/issues",
        "forum": "https://www.neoacevedo.co/support",
        "source": "https://github.com/neoacevedo/openbiblio2"
    },
    "minimum-stability": "stable",
    "require": {
        "php": ">=7.0",
        "yiisoft/yii2": "~2.0.6",
        "yiisoft/yii2-bootstrap": "~2.0.0",
        "yiisoft/yii2-swiftmailer": "~2.0.0",
        "johnitvn/yii2-rbac-plus": "^1.0",
        "kartik-v/yii2-widget-sidenav": "*",
        "yiisoft/yii2-jui": "^2.0",
        "kartik-v/yii2-mpdf": "dev-master",
        "mpdf/mpdf": "^7.0",
        "kartik-v/yii2-export": "@dev"
    },
    "require-dev": {
        "yiisoft/yii2-debug": "~2.0.0",
        "yiisoft/yii2-gii": "~2.0.0",
        "yiisoft/yii2-faker": "~2.0.0",

        "codeception/base": "^2.2.3",
        "codeception/verify": "~0.3.1"
    },
    "config": {
        "process-timeout": 1800,
        "fxp-asset":{
            "installer-paths": {
                "npm-asset-library": "vendor/npm",
                "bower-asset-library": "vendor/bower"
            }
        }
    }
}

Output of composer diagnose:

Checking composer.json: WARNING
require.kartik-v/yii2-widget-sidenav : unbound version constraints (*) should be avoided
require.kartik-v/yii2-mpdf : unbound version constraints (dev-master) should be avoided
require.kartik-v/yii2-export : unbound version constraints (@dev) should be avoided
Checking platform settings: OK
Checking git settings: OK
Checking http connectivity to packagist: OK
Checking https connectivity to packagist: OK
Checking github.com oauth access: OK
Checking disk free space: OK
Checking pubkeys: 
Tags Public Key Fingerprint: 57815BA2 7E54DC31 7ECC7CC5 573090D0  87719BA6 8F3BB723 4E5D42D0 84A14642
Dev Public Key Fingerprint: 4AC45767 E5EC2265 2F0C1167 CBBB8A2B  0C708369 153E328C AD90147D AFE50952
OK
Checking composer version: OK
Composer version: 1.6.3
PHP version: 7.1.9
PHP binary path: /usr/local/bin/php

When I run this command:

composer require yiisoft/yii2-apidoc:^2.1 --update-with-all-dependencies -vvv

I get the following output:

Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Installation request for yiisoft/yii2-apidoc ^2.1 -> satisfiable by yiisoft/yii2-apidoc[2.1.0].
    - yiisoft/yii2-apidoc 2.1.0 requires cebe/markdown ~1.0.0 | ~1.1.0 -> satisfiable by cebe/markdown[1.0.0, 1.0.0-rc, 1.0.1, 1.0.2, 1.0.3, 1.0.x-dev, 1.1.0, 1.1.1, 1.1.2] but these conflict with your requirements or minimum-stability.
  Problem 2
    - yiisoft/yii2 2.0.15.1 requires cebe/markdown ~1.0.0 | ~1.1.0 -> satisfiable by cebe/markdown[1.0.0, 1.0.0-rc, 1.0.1, 1.0.2, 1.0.3, 1.0.x-dev, 1.1.0, 1.1.1, 1.1.2] but these conflict with your requirements or minimum-stability.
    - yiisoft/yii2 2.0.15.1 requires cebe/markdown ~1.0.0 | ~1.1.0 -> satisfiable by cebe/markdown[1.0.0, 1.0.0-rc, 1.0.1, 1.0.2, 1.0.3, 1.0.x-dev, 1.1.0, 1.1.1, 1.1.2] but these conflict with your requirements or minimum-stability.
    - yiisoft/yii2 2.0.15.1 requires cebe/markdown ~1.0.0 | ~1.1.0 -> satisfiable by cebe/markdown[1.0.0, 1.0.0-rc, 1.0.1, 1.0.2, 1.0.3, 1.0.x-dev, 1.1.0, 1.1.1, 1.1.2] but these conflict with your requirements or minimum-stability.
    - Installation request for yiisoft/yii2 (locked at 2.0.15.1, required as ~2.0.6) -> satisfiable by yiisoft/yii2[2.0.15.1].


Installation failed, reverting ./composer.json to its original content.

And I expected this to happen:

An update with all dependencies

Workaround:

  composer require yiisoft/yii2-apidoc:^2.1 --no-update
  composer update

Related issues:

• install error yiisoft/yii2-apidoc#123
• Allow a given package and its dependencies (including siblings) to be updated. #6661 (comment)
• Updating a package that itself has an updated dependency is failing #5362
• Update whitelisting does not correctly whitelist dependencies of new packages to be installed #5438

Background:

When running a full update the package cebe/markdown is updated:

Installs: scrivo/highlight.php:v8.9.1, mikevanriel/text-to-latex:1.0.1, cebe/markdown-latex:1.1.4, cebe/js-search:0.9.3, nikic/php-parser:v1.4.1, phpdocumentor/reflection:3.0.1, bower-asset/jquery:3.2.1, bower-asset/yii2-pjax:2.0.7.1, bower-asset/punycode:v1.3.2, bower-asset/inputmask:3.3.11, bower-asset/bootstrap:v3.3.7, yiisoft/yii2-apidoc:2.1.0, bower-asset/typeahead.js:v0.11.1, bower-asset/jquery-ui:1.12.1, bower-asset/bootstrap3-dialog:v1.35.4
Updates: cebe/markdown:1.0.3, phpdocumentor/reflection-docblock:2.0.5
Removals: webmozart/assert, phpdocumentor/type-resolver, phpdocumentor/reflection-common
  - Removing webmozart/assert (1.3.0)
  - Removing phpdocumentor/type-resolver (0.4.0)
  - Removing phpdocumentor/reflection-common (1.0.1)
  - Installing scrivo/highlight.php (v8.9.1): Loading from cache
 Extracting archive    REASON: yiisoft/yii2-apidoc 2.1.0 requires scrivo/highlight.php ~8.0 -> satisfiable by scrivo/highlight.php[v8.0, v8.1, v8.2, v8.3, v8.4, v8.5, v8.6, v8.7, v8.8.0, v8.9.1].

  - Installing mikevanriel/text-to-latex (1.0.1): Loading from cache
 Extracting archive    REASON: cebe/markdown-latex 1.1.4 requires mikevanriel/text-to-latex ~1.0.0 -> satisfiable by mikevanriel/text-to-latex[1.0.0, 1.0.1].

  - Updating cebe/markdown (1.1.2 => 1.0.3): Loading from cache Extracting archive
    REASON: cebe/markdown-latex 1.1.4 requires cebe/markdown ~1.0.0 -> satisfiable by cebe/markdown[1.0.0, 1.0.1, 1.0.2, 1.0.3].

[...]

  - Updating phpdocumentor/reflection-docblock (4.3.0 => 2.0.5): Loading from cache Extracting archive
    REASON: phpdocumentor/reflection 3.0.1 requires phpdocumentor/reflection-docblock ~2.0 -> satisfiable by phpdocumentor/reflection-docblock[2.0.0, 2.0.1, 2.0.2, 2.0.3, 2.0.4, 2.0.5].

[...]

Writing lock file
Generating autoload files

But it's a dependency of

root@0df0de4e0f4c:/app# composer why cebe/markdown
cebe/markdown-latex  1.1.4     requires  cebe/markdown (~1.0.0)           
yiisoft/yii2         2.0.13.1  requires  cebe/markdown (~1.0.0 | ~1.1.0)  
yiisoft/yii2-apidoc  2.1.0     requires  cebe/markdown (~1.0.0 | ~1.1.0)  

So it's a dependency of a root dependency, maybe that's the reason why it is not touched.
According to help:

--update-with-all-dependencies  Allows all inherited dependencies to be updated, including those that are root requirements.

That should actually be --update-with-root-dependencies (?) or is there also an option to allow a full update?

[Seldaek]

I can't say that I see what causes the issue, seems to be some weird edge case in the solver when doing a partial update.. --update-with-all-dependencies means all dependencies of the package you are requiring, not all your dependencies. There is no way to do a full update from require, run a composer update if you want a full update.

[schmunk42]

--update-with-all-dependencies means all dependencies of the package you are requiring

isn't that --update-with-dependencies?

not all your dependencies.

...of the package I am requiring and the ones listed in my projects composer.json, but not their deps, right?

There is no way to do a full update from require, run a composer update if you want a full update.

An issue from the UX point of view is that there are cases where you can not require a package directly, but the option update-with-all-dependencies means suggests it should be possible (at least to me).

[Seldaek]

--update-with-dependencies only updates dependencies of the package you are requiring that are not also required by your root package, so as to avoid updating more than needed.

...of the package I am requiring and the ones listed in my projects composer.json, but not their deps, right?

No it means what I said, only all dependencies of the package you are requiring, including those that you may have required in root that the pcakage also requires.

I understand that it's a little confusing, but I am not sure that yet another flag would reduce confusion..

[schmunk42]

No it means what I said, only all dependencies of the package you are requiring, including those that you may have required in root that the pcakage also requires.

I meant that 😉 it's a little confusing 😄

I can't say that I see what causes the issue, seems to be some weird edge case in the solver when doing a partial update

To my understanding it's like so:

• requiring yiisoft/yii2-apidoc requires cebe/markdown (~1.0.0 | ~1.1.0)
• cebe/markdown is not a root dependency, but a dep of yiisoft/yii2 locked at a specific version
• therefore it can be updated with any of the current options

I don't think that's a very uncommon edge case, but advising to use

composer require yiisoft/yii2-apidoc:^2.1 --no-update
composer update

would be OK to me (that's what I've expected from "all").

Feel free to close it or mark as docs.

CC: @cebe

[Seldaek]

Well the odd thing is the --no-update + composer update should not have to be done.. It should just update fine as part of the require with --update-with-dependencies as you are not requiring cebe/markdown yourself. Seems like a bug but I can't imagine where.

[Seldaek]

Fixed by #7832

[Seldaek]

Would appreciate if you can confirm the fix works with latest snapshot build btw :)

[schmunk42]

Does not work for me :(

root@f683d4892b35:/repo/composer-issue-7261# composer -V
Composer version 1.9-dev (81de5f82aa1418cd566ca2b6ee79f35642070be5) 2019-01-28 16:57:31

 composer require yiisoft/yii2-apidoc:^2.1 --update-with-all-dependencies
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Conclusion: don't install yiisoft/yii2-apidoc 2.1.1
    - Conclusion: remove cebe/markdown 1.1.2
    - Installation request for yiisoft/yii2-apidoc ^2.1 -> satisfiable by yiisoft/yii2-apidoc[2.1.0, 2.1.1].
    - Conclusion: don't install cebe/markdown 1.1.2
    - yiisoft/yii2-apidoc 2.1.0 requires cebe/markdown-latex ~1.0 -> satisfiable by cebe/markdown-latex[1.0.0, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4].
    - cebe/markdown-latex 1.0.0 requires cebe/markdown ~1.0.0 -> satisfiable by cebe/markdown[1.0.0, 1.0.1, 1.0.2, 1.0.3].
    - cebe/markdown-latex 1.1.0 requires cebe/markdown ~1.0.0 -> satisfiable by cebe/markdown[1.0.0, 1.0.1, 1.0.2, 1.0.3].
    - cebe/markdown-latex 1.1.1 requires cebe/markdown ~1.0.0 -> satisfiable by cebe/markdown[1.0.0, 1.0.1, 1.0.2, 1.0.3].
    - cebe/markdown-latex 1.1.2 requires cebe/markdown ~1.0.0 -> satisfiable by cebe/markdown[1.0.0, 1.0.1, 1.0.2, 1.0.3].
    - cebe/markdown-latex 1.1.3 requires cebe/markdown ~1.0.0 -> satisfiable by cebe/markdown[1.0.0, 1.0.1, 1.0.2, 1.0.3].
    - cebe/markdown-latex 1.1.4 requires cebe/markdown ~1.0.0 -> satisfiable by cebe/markdown[1.0.0, 1.0.1, 1.0.2, 1.0.3].
    - Can only install one of: cebe/markdown[1.0.0, 1.1.2].
    - Can only install one of: cebe/markdown[1.0.1, 1.1.2].
    - Can only install one of: cebe/markdown[1.0.2, 1.1.2].
    - Can only install one of: cebe/markdown[1.0.3, 1.1.2].
    - Installation request for cebe/markdown (locked at 1.1.2) -> satisfiable by cebe/markdown[1.1.2].


Installation failed, reverting ./composer.json to its original content.

[Seldaek]

Soo.. cebe/markdown is required by your root package as ^1.1.2 or similar right? And you have what version of yiisoft/yii2-apidoc installed when you run that command?

I can try to repro locally tomorrow see if I can fix it. Any repro details you have would be useful.

[schmunk42]

I just created a composer.json from the initial posting. That's actually from one of the linked issues AFAIR.
I have nothing installed (also no .lock) file.
Then I tried to run the composer require command.

[Seldaek]

Huh.. But if you have nothing installed yet and no lock file I don't quite see where it got that Installation request for cebe/markdown (locked at 1.1.2) from :|

[schmunk42]

Sorry, my fault. I first ran an update then a require.

You should be able to reproduce with this:

{
  "name": "neoacevedo/openbiblio2",
  "description": "Sistema automatizado de gestión bibliotecaria y OPAC.",
  "keywords": ["yii2", "opac", "openbiblio2", "online library project"],
  "homepage": "https://www.neoacevedo.co/openbiblio2",
  "type": "project",
  "license": "GPL-3.0-or-later",
  "support": {
    "issues": "https://github.com/neoacevedo/openbiblio2/issues",
    "forum": "https://www.neoacevedo.co/support",
    "source": "https://github.com/neoacevedo/openbiblio2"
  },
  "minimum-stability": "stable",
  "repositories": [
    {
      "type": "composer",
      "url": "https://asset-packagist.org"
    }
  ],
  "require": {
    "php": ">=7.0",
    "yiisoft/yii2": "~2.0.6",
    "yiisoft/yii2-bootstrap": "~2.0.0",
    "yiisoft/yii2-swiftmailer": "~2.0.0",
    "johnitvn/yii2-rbac-plus": "^1.0",
    "kartik-v/yii2-widget-sidenav": "*",
    "yiisoft/yii2-jui": "^2.0",
    "kartik-v/yii2-mpdf": "dev-master",
    "mpdf/mpdf": "^7.0",
    "kartik-v/yii2-export": "@dev"
  },
  "require-dev": {
    "yiisoft/yii2-debug": "~2.0.0",
    "yiisoft/yii2-gii": "~2.0.0",
    "yiisoft/yii2-faker": "~2.0.0",
    "codeception/base": "^2.2.3",
    "codeception/verify": "~0.3.1"
  },
  "config": {
    "process-timeout": 1800,
    "fxp-asset":{
      "installer-paths": {
        "npm-asset-library": "vendor/npm",
        "bower-asset-library": "vendor/bower"
      }
    }
  }
}

[Seldaek]

ok @schmunk42 I could repro and it seems the issue is that the --update-with[-all]-dependencies only works for packages that are already installed. Otherwise we don't have their dependencies yet.. So really it makes sense for update but for require it's not so usable.

This fixes it after a require --no-update:

composer update --with-all-dependencies yiisoft/yii2-apidoc phpdocumentor/reflection-docblock cebe/markdown

As these two packages need to be downgraded to fit the yii2-apidoc deps:

  - Uninstalling webmozart/assert (1.4.0)
  - Uninstalling phpdocumentor/type-resolver (0.4.0)
  - Uninstalling phpdocumentor/reflection-common (1.0.1)
  - Updating cebe/markdown (1.1.2) to cebe/markdown (1.0.3)
  - Installing scrivo/highlight.php (v8.9.1)
  - Updating phpdocumentor/reflection-docblock (4.3.0) to phpdocumentor/reflection-docblock (2.0.5)
  - Installing nikic/php-parser (v1.4.1)
  - Installing phpdocumentor/reflection (3.0.1)
  - Installing mikevanriel/text-to-latex (1.0.1)
  - Installing cebe/markdown-latex (1.1.4)
  - Installing cebe/js-search (0.9.3)
  - Installing yiisoft/yii2-apidoc (2.1.1)

This might be fixable in 2.0 after refactorings in the Installer class, but I am not sure to be honest, because for a package that is not installed yet, which versions would we take its dependencies from? It's not that simple. Leaving open for now.

[Seldaek]

Actually.. duplicate of #5438 (just saw now you referenced that issue) so closing. #5438 (comment) explains why this is complex.