Confusion as to versions when adding more specific new requires to existing set?

[jdforrester]

[Initially reported at https://phabricator.wikimedia.org/T296208]

MW core has in require-dev "symfony/yaml": "~3.4|~5.1",

WikiLambda has "symfony/yaml": "5.3.6" in require.

~5.1 would be satisified by 5.3.6

$ composer update --verbose
> init: Wikimedia\Composer\Merge\V2\MergePlugin->onInit
  [merge-plugin] Loading composer.local.json...
  [merge-plugin] Loading extensions/AntiSpoof/composer.json...
  [merge-plugin] Adding wikimedia/equivset
  [merge-plugin] Loading extensions/Flow/composer.json...
  [merge-plugin] Adding pimple/pimple
  [merge-plugin] Loading extensions/OAuth/composer.json...
  [merge-plugin] Prepending git repository
  [merge-plugin] Adding firebase/php-jwt
  [merge-plugin] Adding league/oauth2-server
  [merge-plugin] Loading extensions/OATHAuth/composer.json...
  [merge-plugin] Adding christian-riesen/base32
  [merge-plugin] Adding jakobo/hotp-php
  [merge-plugin] Loading extensions/TemplateStyles/composer.json...
  [merge-plugin] Adding wikimedia/css-sanitizer
  [merge-plugin] Loading extensions/WebAuthn/composer.json...
  [merge-plugin] Adding web-auth/webauthn-lib
  [merge-plugin] Loading extensions/WikiLambda/composer.json...
  [merge-plugin] Adding opis/json-schema
  [merge-plugin] Adding symfony/yaml
> pre-update-cmd: Wikimedia\Composer\Merge\V2\MergePlugin->onInstallUpdateOrDump
> pre-update-cmd: ComposerHookHandler::onPreUpdate
Loading composer repositories with package information
Updating dependencies                                               
Dependency resolution completed in 0.040 seconds
Analyzed 2256 packages to resolve dependencies
Analyzed 68587 rules to resolve dependencies
Dependency resolution completed in 0.000 seconds
Unable to find a compatible set of packages based on your non-dev requirements alone.
Your requirements can be resolved successfully when require-dev packages are present.
You may need to move packages from require-dev or some of their dependencies to require.

  Problem 1
    - Root composer.json requires symfony/yaml 5.3.6, found symfony/yaml[v5.3.11] but it does not match the constraint.

5.3.6 would satisfy the requirements, but perhaps the plugin doesn't want to downgrade?

[reedy]

For a minimum replication case based on MW would be something like...

composer.json

{
	"require": {
		"wikimedia/composer-merge-plugin": "2.0.1"
	},
	"require-dev": {
		"symfony/yaml": "~3.4|~5.1"
	},
	"extra": {
		"merge-plugin": {
			"include": [
				"composer.local.json"
			],
			"merge-dev": false
		}
	}
}

composer.local.json

{
        "extra": {
                "merge-plugin": {
                        "include": [
                                "composer-extra.json"
                        ]
                }
        }
}

composer-extra.json

{
	"require": {
		"symfony/yaml": "5.3.6"
	}
}

I wonder if this is trivially makeable into a failing test...

[reedy]

For a minimum replication case based on MW would be something like...

composer.json

{
	"require": {
		"wikimedia/composer-merge-plugin": "2.0.1"
	},
	"require-dev": {
		"symfony/yaml": "~3.4|~5.1"
	},
	"extra": {
		"merge-plugin": {
			"include": [
				"composer.local.json"
			],
			"merge-dev": false
		}
	}
}

If this is swapped for the below (ie symfony/yaml is promoted from require-dev to require) it works fine...

{
	"require": {
		"wikimedia/composer-merge-plugin": "2.0.1",
		"symfony/yaml": "~3.4|~5.1"
	},
	"require-dev": {
	},
	"extra": {
		"merge-plugin": {
			"include": [
				"composer.local.json"
			],
			"merge-dev": false
		}
	}
}

With it in require-dev, setting merge-dev: true doens't help...

So it seem this is something weird happening wrt the handling of the require-dev stuff eventually, when the requirement (elsewhere in the tree) is in require...

[reedy]

The log for that looks more like

$ composer update --verbose
> init: Wikimedia\Composer\Merge\V2\MergePlugin->onInit
  [merge-plugin] Loading composer.local.json...
  [merge-plugin] Loading extensions/AntiSpoof/composer.json...
  [merge-plugin] Adding wikimedia/equivset
  [merge-plugin] Loading extensions/Flow/composer.json...
  [merge-plugin] Adding pimple/pimple
  [merge-plugin] Loading extensions/OAuth/composer.json...
  [merge-plugin] Prepending git repository
  [merge-plugin] Adding firebase/php-jwt
  [merge-plugin] Adding league/oauth2-server
  [merge-plugin] Loading extensions/OATHAuth/composer.json...
  [merge-plugin] Adding christian-riesen/base32
  [merge-plugin] Adding jakobo/hotp-php
  [merge-plugin] Loading extensions/TemplateStyles/composer.json...
  [merge-plugin] Adding wikimedia/css-sanitizer
  [merge-plugin] Loading extensions/WebAuthn/composer.json...
  [merge-plugin] Adding web-auth/webauthn-lib
  [merge-plugin] Loading extensions/WikiLambda/composer.json...
  [merge-plugin] Adding opis/json-schema
  [merge-plugin] Merging symfony/yaml
> pre-update-cmd: Wikimedia\Composer\Merge\V2\MergePlugin->onInstallUpdateOrDump
> pre-update-cmd: ComposerHookHandler::onPreUpdate
Loading composer repositories with package information
Updating dependencies                                               
Dependency resolution completed in 0.031 seconds
Analyzed 2182 packages to resolve dependencies
Analyzed 48235 rules to resolve dependencies
Nothing to modify in lock file
Dependency resolution completed in 0.000 seconds
Installing dependencies from lock file (including require-dev)
Package operations: 0 installs, 0 updates, 0 removals
Package phpunit/php-token-stream is abandoned, you should avoid using it. No replacement was suggested.
Generating optimized autoload files
> pre-autoload-dump: Wikimedia\Composer\Merge\V2\MergePlugin->onInstallUpdateOrDump
> post-autoload-dump: PackageVersions\Installer->dumpVersionsClass
composer/package-versions-deprecated: Generating version class...
composer/package-versions-deprecated: ...done generating version class
55 packages you are using are looking for funding.
Use the `composer fund` command to find out more!
> post-update-cmd: Wikimedia\Composer\Merge\V2\MergePlugin->onPostInstallOrUpdate
> post-update-cmd: ComposerVendorHtaccessCreator::onEvent