[Feature] Add possibility to run one plugin several times in one stage

[SimonHeimberg]

Use case:

• Run some scripts, run phpunit, run some shell scripts
• run mess detector with different settings for src/ and tests/
• run several composer commands (can be done with shell, but must find composer binary manually)
• run phpunit with different environment settings (maybe better create a separate project for this?)

Suggestion for implementation: allow a space and some id after the plugin name:

setup:
  composer install: {}
  composer myCommand:
    action: "myCustomCommand"
tests:
  php_mess_detector:
    path: 'src'
    allowed_warnings: 0
  php_mess_detector tests:
    path: 'tests'
    allowed_warnings: -1

This will be simple for some plugins. But it may be hard for some collecting output (phpunit, ...), maybe forbid those.

[corpsee]

Thank you for the idea @SimonHeimberg! It will be very hard to implement... Most likely it will not be done soon)

[SimonHeimberg]

Calling one plugin several times in one stage is very simple. In fact, it can already be done now for internal plugins:

tests:
    composer:
       action: validate
    Composer:
        action: outdated
    composer_:
        action: install --dry-run --no-suggests
    composer _:
        ...

This is good enough for me. I use it to run several composer commands, and to run shell several times, without stopping the next shell command from running, but still return failure. (plugins/shell.md does not mention that shell commands stop on first failure, but this is how it works.)

A first step in the desired direction is done in this branch:
https://github.com/SimonHeimberg/php-censor/commits/multiCallInStage
It works like this:

tests:
    shell:
       - echo 1; false
    shell - e2:
        - echo 2

I do not see a good way how to prevent unsupported plugins to run several times (or at least warn about). How should this plugins be recognized? A manual created list? Implement some interface? Parse its source?

Plugins working without any problems (in 2017):

• when only creating log output (and pass/fail in summary):
  Plugins: composer, shell, env, security_checker, any notifications, ...
• when creating errors:
  Plugins: security_checker, php_cs_fixer, ...
  Plugins creating wrong results (in 2017):
• when creating data for the plot. Only the number of the last instance is plotted. Building the sum instead of overwriting would not be very hard.
  The list of errors is correct.
  Plugins: PhpCodeSniffer, PhpCpd, PhpParallelLint, ...
• when creating separate data to display. Only the data of the last instance is kept. At least harder to keep all data, and unsure if helpful.
  (The list of errors is correct.)
  Plugins: PhpUnit, Codeception, PhpLoc, PhpTalLint, ...

[corpsee]

Implemented in #417.