Increase performance on Update

[francoispluchino]

Currently, for the install, the plugin recovers only package definitions accepted by the range version. However, for the update, the plugin retrieves all definitions of packages, if there is no constraint in the root Composer package.

I did a proof of concept to drastically reduce the number of import of package definitions for the update:

Filter the tags that are greater than or equal to the installed, or defined by the range version in the root Composer package. In this way, the update only fetches the latest version for each dependency (or greater than the installed version).

However, there is a problem that can happen when a dependency uses a new version lower than the installed version: instead of removing the installed version and install the old version, Composer will throw an exception stating that the version does not exist.

Of course, 3 solutions can work around the problem:

• delete the vendor directory, do the update
• disable temporarily the feature in the extra section
• or add the dependency in the root Composer package with the required old version, do the update, and remove the dependency in the root Composer package

In view of the small ergonomic problem, I wonder if I can add this feature by default. I can also add an option (disabled by default) to enable / disable this feature.

What do you think?

[francoispluchino]

@cebe, @hason, @samdark What do you think?

[samdark]

Well, update performance is indeed a problem. If I'd have to choose I'll go for such solution even if there are issues.

[francoispluchino]

This case is rare, but it can happen. What bothers me, is that I have no way to add this information directly in the error message. Of course I can put this warning in the documentation.

With the option in the extra section, There are 3 solutions to resolve the problem, knowing that delete the vendor directory, is a is the quickest and easiest solution.

Do I activate this feature by default?

[francoispluchino]

Example with bootstrap:

{
    "require": {
        "bower-asset/bootstrap": "3.2.*"
    }
}

Install:

$ composer install --prefer-dist -v
Loading composer repositories with package information
Updating dependencies (including require-dev)
Reading bower.json of bower-asset/bootstrap (v3.2.0)
Importing tag v3.2.0 (3.2.0.0)
Reading bower.json of bower-asset/jquery (2.1.1)
Importing tag 2.1.1 (2.1.1.0)
Reading bower.json of bower-asset/jquery (2.1.0)
Importing tag 2.1.0 (2.1.0.0)
Reading bower.json of bower-asset/jquery (2.0.3)
Importing tag 2.0.3 (2.0.3.0)
Reading bower.json of bower-asset/jquery (2.0.2)
Importing tag 2.0.2 (2.0.2.0)
Reading bower.json of bower-asset/jquery (2.0.1)
Importing tag 2.0.1 (2.0.1.0)
Reading bower.json of bower-asset/jquery (2.0.0)
Importing tag 2.0.0 (2.0.0.0)
Reading bower.json of bower-asset/jquery (1.11.1)
Importing tag 1.11.1 (1.11.1.0)
Reading bower.json of bower-asset/jquery (1.11.0)
Importing tag 1.11.0 (1.11.0.0)
Reading bower.json of bower-asset/jquery (1.10.2)
Importing tag 1.10.2 (1.10.2.0)
Reading bower.json of bower-asset/jquery (1.10.1)
Importing tag 1.10.1 (1.10.1.0)
Reading bower.json of bower-asset/jquery (1.10.0)
Importing tag 1.10.0 (1.10.0.0)
Reading bower.json of bower-asset/jquery (1.9.1)
Importing tag 1.9.1 (1.9.1.0)
Reading bower.json of bower-asset/jquery (1.9.0)
Importing tag 1.9.0 (1.9.0.0)
  - Installing bower-asset/jquery (2.1.1)
    Loading from cache
    Extracting archive

  - Installing bower-asset/bootstrap (v3.2.0)
    Loading from cache
    Extracting archive

Writing lock file
Generating autoload files

Update:

$ composer update --prefer-dist -v
Loading composer repositories with package information
Updating dependencies (including require-dev)
Reading bower.json of bower-asset/jquery (2.1.1)
Importing tag 2.1.1 (2.1.1.0)
Reading bower.json of bower-asset/bootstrap (v3.2.0)
Importing tag v3.2.0 (3.2.0.0)
Nothing to install or update
Generating autoload files

Update without optimization:

{
    "require": {
        "bower-asset/bootstrap": "3.2.*"
    },
    "extra": {
        "asset-optimize-with-installed-packages": false
    }
}

$ composer update --prefer-dist -v
Loading composer repositories with package information
Updating dependencies (including require-dev)
Reading bower.json of bower-asset/jquery (2.1.1)
Importing tag 2.1.1 (2.1.1.0)
Reading bower.json of bower-asset/jquery (2.1.0)
Importing tag 2.1.0 (2.1.0.0)
Reading bower.json of bower-asset/jquery (2.0.3)
Importing tag 2.0.3 (2.0.3.0)
Reading bower.json of bower-asset/jquery (2.0.2)
Importing tag 2.0.2 (2.0.2.0)
Reading bower.json of bower-asset/jquery (2.0.1)
Importing tag 2.0.1 (2.0.1.0)
Reading bower.json of bower-asset/jquery (2.0.0)
Importing tag 2.0.0 (2.0.0.0)
Reading bower.json of bower-asset/jquery (1.11.1)
Importing tag 1.11.1 (1.11.1.0)
Reading bower.json of bower-asset/jquery (1.11.0)
Importing tag 1.11.0 (1.11.0.0)
Reading bower.json of bower-asset/jquery (1.10.2)
Importing tag 1.10.2 (1.10.2.0)
Reading bower.json of bower-asset/jquery (1.10.1)
Importing tag 1.10.1 (1.10.1.0)
Reading bower.json of bower-asset/jquery (1.10.0)
Importing tag 1.10.0 (1.10.0.0)
Reading bower.json of bower-asset/jquery (1.9.1)
Importing tag 1.9.1 (1.9.1.0)
Reading bower.json of bower-asset/jquery (1.9.0)
Importing tag 1.9.0 (1.9.0.0)
Reading bower.json of bower-asset/jquery (1.8.3-patch1)
Importing tag 1.8.3-patch1 (1.8.3.0-patch1)
Reading bower.json of bower-asset/jquery (1.8.3)
Importing tag 1.8.3 (1.8.3.0)
Reading bower.json of bower-asset/jquery (1.8.2)
Importing tag 1.8.2 (1.8.2.0)
Reading bower.json of bower-asset/jquery (1.8.1)
Importing tag 1.8.1 (1.8.1.0)
Reading bower.json of bower-asset/jquery (1.8.0)
Importing tag 1.8.0 (1.8.0.0)
Reading bower.json of bower-asset/jquery (1.7.2)
Importing tag 1.7.2 (1.7.2.0)
Reading bower.json of bower-asset/jquery (1.7.1)
Importing tag 1.7.1 (1.7.1.0)
Reading bower.json of bower-asset/jquery (1.7.0)
Importing tag 1.7.0 (1.7.0.0)
Reading bower.json of bower-asset/jquery (1.7)
Importing tag 1.7 (1.7.0.0)
Reading bower.json of bower-asset/jquery (1.6.4)
Importing tag 1.6.4 (1.6.4.0)
Reading bower.json of bower-asset/jquery (1.6.3)
Importing tag 1.6.3 (1.6.3.0)
Reading bower.json of bower-asset/jquery (1.6.2)
Importing tag 1.6.2 (1.6.2.0)
Reading bower.json of bower-asset/jquery (1.6.1)
Importing tag 1.6.1 (1.6.1.0)
Reading bower.json of bower-asset/jquery (1.6.0)
Importing tag 1.6.0 (1.6.0.0)
Reading bower.json of bower-asset/jquery (1.6)
Importing tag 1.6 (1.6.0.0)
Reading bower.json of bower-asset/jquery (1.5.2)
Importing tag 1.5.2 (1.5.2.0)
Reading bower.json of bower-asset/jquery (1.5.1)
Importing tag 1.5.1 (1.5.1.0)
Reading bower.json of bower-asset/jquery (1.5.0)
Importing tag 1.5.0 (1.5.0.0)
Reading bower.json of bower-asset/jquery (1.5)
Importing tag 1.5 (1.5.0.0)
Reading bower.json of bower-asset/jquery (1.4.4)
Importing tag 1.4.4 (1.4.4.0)
Reading bower.json of bower-asset/jquery (1.4.3)
Importing tag 1.4.3 (1.4.3.0)
Reading bower.json of bower-asset/jquery (1.4.2)
Importing tag 1.4.2 (1.4.2.0)
Reading bower.json of bower-asset/jquery (1.4.1)
Importing tag 1.4.1 (1.4.1.0)
Reading bower.json of bower-asset/jquery (1.4.0)
Importing tag 1.4.0 (1.4.0.0)
Reading bower.json of bower-asset/jquery (1.4)
Importing tag 1.4 (1.4.0.0)
Reading bower.json of bower-asset/jquery (1.3.2)
Importing tag 1.3.2 (1.3.2.0)
Reading bower.json of bower-asset/jquery (1.3.1)
Importing tag 1.3.1 (1.3.1.0)
Reading bower.json of bower-asset/jquery (1.3.0)
Importing tag 1.3.0 (1.3.0.0)
Reading bower.json of bower-asset/jquery (1.3)
Importing tag 1.3 (1.3.0.0)
Reading bower.json of bower-asset/jquery (1.2.6)
Importing tag 1.2.6 (1.2.6.0)
Reading bower.json of bower-asset/jquery (1.2.5)
Importing tag 1.2.5 (1.2.5.0)
Reading bower.json of bower-asset/jquery (1.2.4)
Importing tag 1.2.4 (1.2.4.0)
Reading bower.json of bower-asset/jquery (1.2.3)
Importing tag 1.2.3 (1.2.3.0)
Reading bower.json of bower-asset/jquery (1.2.2)
Importing tag 1.2.2 (1.2.2.0)
Reading bower.json of bower-asset/jquery (1.2.1)
Importing tag 1.2.1 (1.2.1.0)
Reading bower.json of bower-asset/jquery (1.2)
Importing tag 1.2 (1.2.0.0)
Reading bower.json of bower-asset/jquery (1.1.4)
Importing tag 1.1.4 (1.1.4.0)
Reading bower.json of bower-asset/jquery (1.1.3.1)
Importing tag 1.1.3.1 (1.1.3.1)
Reading bower.json of bower-asset/jquery (1.1.3)
Importing tag 1.1.3 (1.1.3.0)
Reading bower.json of bower-asset/jquery (1.1.2)
Importing tag 1.1.2 (1.1.2.0)
Reading bower.json of bower-asset/jquery (1.1.1)
Importing tag 1.1.1 (1.1.1.0)
Reading bower.json of bower-asset/jquery (1.1)
Importing tag 1.1 (1.1.0.0)
Reading bower.json of bower-asset/jquery (1.0.4)
Importing tag 1.0.4 (1.0.4.0)
Reading bower.json of bower-asset/jquery (1.0.3)
Importing tag 1.0.3 (1.0.3.0)
Reading bower.json of bower-asset/jquery (1.0.2)
Importing tag 1.0.2 (1.0.2.0)
Reading bower.json of bower-asset/jquery (1.0.1)
Importing tag 1.0.1 (1.0.1.0)
Reading bower.json of bower-asset/jquery (1.0)
Importing tag 1.0 (1.0.0.0)
Reading bower.json of bower-asset/bootstrap (v3.2.0)
Importing tag v3.2.0 (3.2.0.0)
Nothing to install or update
Writing lock file
Generating autoload files

[nsams]

Imho as long-term solution for the performance issue the problem of loading that many packages should be solved in composer itself.

I tried to do that some time ago but without success as I didn't understand the concept of this "whitelist" where all package versions plus their dependencies are added to.

[samdark]

Looks OK. Will it handle

{
    "require": {
        "bower-asset/bootstrap": "~3.2.0"
    }
}

?

[francoispluchino]

@samdark Of course!

@nsams From what I understood, the principle of Solver SAT, is that it requires to know the entirety of the dependency tree.

On a short-term solution, I think it's better to do it in the plugin, and remove this optimization thereafter.

[cebe]

Speaking for Yii only, and if it is disabled by default, we can enable the feature in the application template that people will download. However this is worse than enabling it in the plugin by default because it will be well visible in the docs.

In general I think downgrading a package will most likely work inside of the root package to fix a specific version and not because another package will require a lower version. As downgrading in the root package will still work and when the docs state this somehow well visible, we should enable it by default.

[francoispluchino]

@cebe Sorry, I did not understand everything, do you want to enable this feature by default? This is the case currently.

[cebe]

I gave some arguments for and against but I'd vote for enabling by default and highlight in documentation.

[francoispluchino]

Ok, that's what I'm doing (doc and FAQs).

[francoispluchino]

Added by fca1f3a.

Doc and FAQs: bf491ad