Add auto_up option to exclude service from docker-compose up by default

[acran]

This an updated version of #3047, now complete with added test cases.

This adds the new config option auto_up to services. When this is explicitly set to false the service will not be started when executing docker-compose up without service names (unless it is pulled in as a dependency of another service).

This allows to define optional services in a docker-compose.yml which are only started when listed explicitly on the command line.

Resolves #6742, #1896 and many more reports requesting this feature or similar.

Use case

For one example use case of the many listed in #1896 and linked issues consider the following docker-compose.yml:

version: '3.8'
services:
  app:
    image: php
    depends_on:
      - db

  db:
    image: mysql

  phpmyadmin:
    auto_up: false
    image: phpmyadmin/phpmyadmin
    depends_on:
      - db

  mailhog:
    auto_up: false
    image: mailhog/mailhog

This may be an example of a prototypical web app running on PHP and using a MySQL database. Most of the time one only wants to run the app and db service but for debugging/development purposes two additional, optional services are defined here:

• phpmyadmin: simple web interface to access and manipulate the raw database
• mailhog: a mocked mail transfer agent to be able to test and debug e-mails sent by the application

This now allows to start the main services with a simple docker-compose up and the optional services by listing them:

# start only app and db
# without the auto_up option this would also start phpmyadmin and mailhog
docker-compose up

docker-compose up phpmyadmin # starts phpmyadmin, as well as db if necessary
docker-compose up mailhog # starts only the mailhog service

Implementation details

This feature is realized by slightly changing get_services in the Project module. If no service names are passed via the command line it falls back to all services in the docker-compose.yml - which are now filtered for services which have auto_up explicitly set to false.

When auto_up is set to trueor is omitted the services are included in the default set as before. Additionally this change only affects the docker-compose up and docker-compose create commands. Consider this docker-compose.yml:

version: '3.8'
services:
  service1:
    image: alpine
    command: echo "I'm service 1"

  service2:
    auto_up: false
    image: alpine
    command: echo "I'm service 2"

  service3:
    image: alpine
    depends_on:
      - service1
      - service2
    command: echo "I'm service 3"

  service4:
    auto_up: false
    image: alpine
    depends_on:
      - service2
    command: echo "I'm service 4"

With this docker-compose behaves as follows:

docker-compose up service1 # starts only service1
docker-compose up service2 # starts only service2

docker-compose up
# starts service1, service2, service3:
# service4 is skipped because of auto_up: false
# service2 is pulled in as dependency of service3

docker-compose up service4 # starts service4 and service2 as its dependency

docker-compose stop # stops all (4) services

docker-compose down # stops and removes all (4) services
# note that --remove-orphans is not needed here

docker-compose logs # shows logs of all 4 services

Alternatives

Currently the only way to achive this is by splitting the services into multiple docker-compose.yml files and pass them via the --file parameter:

# our example from above...
docker-compose up phpmyadmin

# ..,would be instead
docker-compose -f docker-compose.tools.yml up phpmyadmin

# but since phpmyadmin depends_on db we actually need multiple files
docker-compose -f docker-compose.yml -f docker-compose.tools.yml up phpmyadmin

As can be seen this is quite cumbersome and especially when using docker-compose for local development this a huge inconvenience as discussed in #1896 extensively and is confirmed by the amount of feature request for many years.

This PR now adds the simple option auto_up to the schema from version 2.0 up.

Documentation

The documentation for this option can be found - and later merged from - here: acran/docker.github.io@03ce9ed

References

This or a similar feature was requested in: #6742 (most recent discussion), #1896 (main discussion), #3047 (previous PR), #697, #912, #1294, #1439, #1547, #2803, #3027, #3289

also related: #1661, #1988, #2773, #3684, #4096, #4650, #6676

[acran]

Although I now also added test cases (which just all passed green 😄) and spend some extra time checking all code paths this change is affecting for any unintended side effects, I appreciate any feedback on this PR.
I'll gladly incorporate any changes required to make this PR ready for merge!

[ulyssessouza]

Hello @acran .
Thank you for the contribution, but as this is a change on the specification, the change has to be discussed and validated in https://github.com/compose-spec/compose-spec
Also, changing the already released schemas are not allowed. That's like changing the past.

I'm very sorry, but I cannot integrate this change before it. I invite you to create an issue on https://github.com/compose-spec/compose-spec to discuss it for the next version.

[acran]

Hi @ulyssessouza

Thank you for your feedback. I wasn't aware of the dedicated repository for for the compose file specification. Since this feature definitely requires a schema change I now created an issue there to discuss the proposed change: compose-spec/compose-spec#87

[remort]

What's happened with this PR? Still can't see it functioning in the modern versions of docker-compose.

[acran]

@remort:
This PR was ultimately superseded by the service profiles feature introduced with docker-compose v1.28.0: instead of setting auto_up: false as proposed here you can assign profiles:

services:
  # ...
  service2:
    profiles: ["cli-only"]
    image: alpine
    command: echo "I'm service 2"

You can find the related discussions in compose-spec/compose-spec#97, compose-spec/compose-spec#110 and #7930