install should have a --prune flag

[yehosef]

If I remove dependencies from my composer.json file, the only way (I know of) to actually get those removed from my vendor directory is to do a composer update. This will also update my versions to the latest matching version, which is not necessarily what I was trying to do.

Eg, If I have two packages

AAA/aaa ~1
BBB/bbb ~2

and I am currently at version 1.0 of AAA/aaa and they released 1.1, but I'm not looking to upgrade because I need to run my tests, etc...
when I delete BBB/bbb and want to remove it from my vendor (because it something like a geoip database that's 50MB) - I'll automatically be upgraded to 1.1, which I don't want.

What I want is to be able to run a composer install --prune which will go through the dependencies in composer.json and install the specific versions mentioned in composer.lock (if they exist), install appropriate versions if they don't exist, and remove any irrelevant packages from vendor and composer.lock that do not match a dependency in composer.json

[alcohol]

How about you just run composer update bbb/bbb ?

[yehosef]

The problem is that it's something that I want to be automatic and not requiring human interaction.

Here's the flow where I encountered this problem: we have a githook to do a composer install on checkout. I want updates to be an intentional process and the composer.lock is added to to the repo so the install will just install what is needed based on the composer.lock.

If I have different dependencies on different branches switching branches will ensure I have what is listed in the composer.lock but it won't delete extra things. I don't want each developer to have to run a compose update of the delete packages and don't want to complicate the process to create a log of things that should get deleted and manage that myself - I think the package manager should do it.

In the dev environment it causes extra files to sit around - and might cause problems by leaving files in the autoinclude path that I'm not expecting. Eg, lets say I'm migrating from code AAA to CCC - but I still have some references to AAA in my code. The testing server wouldn't detect the failure it should because the code would still be in the vendor directory and loaded. It's also bad because in my live rollout code I'll still be rsyncing the unused files.

There may be workarounds - but they are not elegant and I really shouldn't have to do it. I think what I'm suggesting is a reasonable feature of a good package management system. The .lock files is a great feature of composer and there should just be someway to enforce it.

[alcohol]

Good luck with that.

[yehosef]

For those that are desperate, I posted a workaround http://stackoverflow.com/questions/26930816/how-to-remove-unused-dependencies-from-composer

[alcohol]

I think the main problem is that there is no way to fully automate this.

Install behaves differently depending on the availability of a .lock file or not.

Update updates everything unless you specifically pass in a whitelist of packages.

Regarding cleanup:

• Update should take care of that (afaik).
• Install does not (afaik), but then again I don't see why it should. Install is not update, it should just install (either from scratch of from .lock file).

I think what you're asking for, is a check in the install flow, that determines if there is a vendor directory already or not, and if so, cleans it up while installing specific versions.

I'm not sure how it could properly do that though, other than just removing everything and installing only what is required (which could make the install procedure quite lengthy unnecessarily).

I know there is also an installed.json in the vendor directory (vendor/composer/installed.json). I would imagine this is (or could be) used for something the above describes.

[yehosef]

What about this.

After the current flow for a composer install finishes, a --prune option
would build a list of top level directories currently in vendor
(/vendor//). You would loop through those directories and
any directory that is not in the list of packages from the composer.lock
you would delete.

What do you think - do you see any issues with this approach?

[alcohol]

Honestly I am not sure. I never encountered this use-case myself yet (something similar, but not the same, and composer update was sufficient for that scenario). I suspect it will be rather complex to get this done, and there isn't enough demand to warrant the work. Almost everything can already be done by simply using one of the following commands;

composer install
composer update
composer remove

[alcohol]

I'm also not sure what the whole issue is.
I tried to replicate it but was not successful.

• I cloned composer twice
• Ran composer install on both clones
• Removed a dependency from the first clone
• Ran composer update on the first clone
• Copied the resulting composer.json and composer.lock to the second clone.

It cleaned up exactly as expected:

$ cp ../composer/composer.lock .
cp: overwrite ‘./composer.lock’? y
$ cp ../composer/composer.json .
cp: overwrite ‘./composer.json’? y
$ composer install
Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
  - Removing symfony/console (v2.6.4)
Generating autoload files

So what does it not clean up according to you? Did I not reproduce the scenario properly? If you have a way for me to reproduce the scenario step by step that would be immensely useful.

It looks to me like a git hook that runs composer install would be sufficient.

[yehosef]

ok - pretty strange. I was doing this yesterday and it wasn't working like this. I guess in this universe everything works as requested.

looks like composer install does it. Closing here, I'll leave open in the other universe because it's still broken there.

[alcohol]

Not sure I understand what you mean by that...

Does it work now? Does it behave differently than before? Did this resolve your issue?

I'm confused :-(

[yehosef]

Sorry - I'm also confused. But it's working now as far as I can tell. I'm moving between branches with different dependencies and it seems to be updating and deleting properly. I'm not sure what was happening before (hence the reference to parallel universes).

[alcohol]

Well, should you run into a situation that does not behave as expected, feel free to reopen or reply to this thread again.

[yehosef]

Back - there is a problem. I was changing the major dependencies and that is updating properly (which I don't think was happening for me yesterday.. not sure.)

But what's still not working, I just wasn't paying attention, is that when I have sub-dependencies and I remove the package, they are not removed.

Here's the flow to see the problem:
composer require elasticsearch/elasticsearch
this will install elasticsearch and several dependencies
composer require kohana/cache
also.
composer remove elasticsearch/elasticsearch

at this point all the dependencies from the elasticsearch are still there, but I don't want them. Moreover, the composer.lock also has the references. So a composer install won't help. And I don't want to do a composer update because I don't want to update the packages I am using. I could manually do a composer remove on all the packages, but that's not nice - I didn't even ask for those packages. I installed elasticsearch and I now have to figure out which package are using what's left and which I can delete.

So this is the use case and I don't know how this can't be happening to everyone all the time that is removing packages. Perhaps people are comfortable updating or specify more specific version - but I think I should be able to remove a package and the dependencies with it without upgrading my other packages.

The implementation is a little more complicated than I originally suggested, but not terribly so. I'll post it in another comment.

[yehosef]

It's really not complicated.. There are two arrays - the requirements in the composer.json and the packages in composer.lock. Inside each package in the composer.lock, there are the dependencies for that.

You have a function to find all the dependencies of a package that are listed in the composed.lock. the pseudo-code looks like

function getDependencies($package, $tree) {
    $dependencies = [$package];
    $sub_packages = $tree[$package]['require'];
    foreach ($sub_packages as $sub_package) {
        $dependencies[] = $sub_package;
        $dependencies = array_merge($dependencies, getDependencies($sub_package,$tree)
    }
    return $dependencies;
}

The top look using the composer.json for the packages and then you then find all the require dependencies from there.

Anything that is no longer required will not make it in the final list. If you write this back to the composer.lock and do a composer install, it seems it will delete the extra packages for you. The whole problem is that the composer.lock contains references that are not needed.

let me know if that makes sense.

[alcohol]

If you run composer remove as follows it behaves as expected:

rob@emac019 /srv/git/test $ composer remove elasticsearch/elasticsearch --update-with-dependencies
Loading composer repositories with package information
Updating dependencies (including require-dev)
  - Removing elasticsearch/elasticsearch (v1.3.2)
  - Removing monolog/monolog (1.12.0)
  - Removing pimple/pimple (v2.1.1)
  - Removing guzzle/http (v3.9.2)
  - Removing guzzle/parser (v3.9.2)
  - Removing guzzle/stream (v3.9.2)
  - Removing guzzle/common (v3.9.2)
  - Removing symfony/event-dispatcher (v2.6.4)
  - Removing psr/log (1.0.0)
Writing lock file
Generating autoload files

[yehosef]

ok then - seems like that would work. sorry for the trouble and thanks for the lesson.

[alcohol]

/cc @Seldaek any idea why this isn't default behaviour by the way?

Edit: the above example, specifically the --update-with-dependencies option flag.

[stof]

@alcohol whitelisting dependencies used to be the default for partial updates (composer remove was not yet implemented at this time), and it caused confusion by allowing to update more packages than what was asked. This is why it is now done only for an explicit choice.

Install does not (afaik), but then again I don't see why it should. Install is not update, it should just install (either from scratch of from .lock file).

Regarding this old comment for you, there is nothing like "install from scratch". If there is no lock file, composer install does exactly the same than composer update.

[charlieman]

I'm having this problem myself. In this case, we have /vendor ignored in git but commit composer.json and composer.lock. When a developer removes a dependency using composer remove sure it cleans the package from their /vendor directory but the rest still have the old packages in /vendor

The only workaround I've found is to delete the whole /vendor directory and run composer install again. Not a big problem since packages are cached but it would be nice if there was an autoremove command like apt (Debian) and dnf (Fedora) have to remove dependencies that are no longer required.

[stof]

@charlieman you don't need to remove the vendor folder. You only have to run composer install whenever you checkout a branch with an updated composer.lock.

install is about installing what is required by the lock file, and this includes taking care to remove extra stuff.