Specify/document how package selection switches (should) work together

[mpdude]

In order to write some tests for #180, I'd like to get an overview of the various package-selection-specific switches and how they go together (if at all).

In particular, we've got:

• require-all
• require
• require-dependencies
• require-dev-dependencies
• minimum-stability
• Packages can be supplied on the command line

Additionally, the way replace is used (some bugs fixed recently) might come into play.

Update

Here's how I think that the various switches should go together.

At all stages, packages not matching the minimum-stability will be discarded and their dependencies will not be followed.

Step 1 – Initial package selection

require-all	No package filter provided	With package filter
true	Select all packages from all repos.	From all repos, select packages by name (given as filter).
false	Select packages according to require list and version constraints.	Take only those version constraints from the require list that are given in the filter.

Step 2 – Add/follow dependencies

• If require-dependencies and require-dev-dependencies is not set, nothing to do.
• Otherwise, for all (dev) dependencies for all selected packages:
  • Add all packages that match (dev) dependency and version constraint to selection list.
  • Recursively process newly added packages.

Caveats

• For "partial" (filtered) updates, merge found/processed packages into existing packages.json. Also make sure we don't lose package versions not present in the update.
• Special handling for "Composer" type repositories with providers might apply.
• Alias packages can be ignored.
• Special handling for provides/replace?

[mpdude]

@shochdoerfer What's the use case for the package filter (command line args) you implemented?

Do you use it with require-all and instead of a require section in the config file?

Do you see any other useful way how it can be used?

[shochdoerfer]

Without a package filter satis will build the packages for every known repo (as defined in the satis config file). This can lead to only waiting times when you deal with a lot of repos and a lot versions (as we do). When you pass a package or a list of packages to the build command only these packages get built (all repos still get queried but it is a lot faster in this case). This way when our Jenkins instance gets triggered for a specific project we can let satis build the packages for just that project which in our case is pretty fast. This was the main intention of the patch.

Concerning the use of require-all or require in the config file I am not able to comment on that one. Maybe @akimsko can help out. If I recall correctly he contributed a patch on that matter.

[mpdude]

If I got you right, every time your Jenkins runs for a particular project it also runs Satis for this project's dependencies/packages.

Do all these Jenkins runs share a single satis.json config file? Does that have a set of require deps or is require-all set to true?

[mpdude]

Oh. I have been looking at it from the package-selection perspective only. What I missed was that the existing packages.json would be merged into the results in a later step: https://github.com/composer/satis/blob/9b4918168378a9c311a55be67ede20a786e3625e/src/Composer/Satis/Command/BuildCommand.php#L159-165.

[shochdoerfer]

Imaging we have two packages mycompany/package1 and mycompany/package2. Both reside in different git repos and are hooked into our Jenkins instance. Whenever I make a commit in either repo Jekins will run and do the usual stuff (phpcs, phpunit, ...). After the particular Jenkins job run successfully we need to let satis re-index the repo so that the latest changes are reflected in the satis repo (e.g. a new version of the package got released). In the past satis would run for every project on every commit which was very inefficient. Now with the package filtering in place satis will only build the package that needs to be built.

We do only use one satis.json config file. I do not see that it would be possible to use multiple satis config files, one per project. That would probably mess things up I guess when it comes to generating the packages.json file.

[shochdoerfer]

Correct, you need to take the generation of the package.json file into account. Someone recently lined out that the code seems to be too complex and could be simplified which might be true. Could not find the time so far to dig into that. The PR evolved over time as I got more and more insights into the inner workings of satis ;)

[mpdude]

Yes, I missed the fact that with the package filter enabled results will be merged into the existing packages.json.

WIthout that merge, it would only output the packages you used for the filter, which in turn would not make sense if you change the filter on every run. So I thought that you would either use require-all + a filter (all relevant packages upon every invocation), or a requires section. Things start to make more sense now.

[shochdoerfer]

Good to hear and great to see that you tackle the "testing problem". In case of other questions concerning this PR feel free to ping me.

[mpdude]

Here is how I think the filter currently behaves:

• With require-all: true:
  • Include all packages from Composer-type repos that use providers
  • For other repos, include the packages named by the filter arguments; skip Alias packages and packages of too low stability
• With require-all: false, pick packages from requires also named in the filter. Additionally, follow their dependencies/devDependencies.

Packages that are present in packages.json but were not processed will be kept as-is.

[mpdude]

I updated the initial comment with an overview of how I think the switched should work together. I would really appreciate feedback especially from @naderman, @Seldaek and @stof to see if this is what you would expect as well.

Seems the above description is not exactly what is currently happening for require-all: true and/or Composer type repos with providers; but that might be a bug as well.