[subaction/matrix] Generate multiple matrices when targets reference other targets as build contexts

[jeliebig]

Description

We'd like to use the matrix subaction to generate multiple matrices, since we have a couple of targets that depend on other targets and would like to avoid building the same target multiple times.

---

As an example, consider the following docker-bake.hcl (containing targets with Dockerfiles that can't be easily merged into a multi-stage Dockerfile):

group "default" {
    targets = ["apps", "special-app"]
}

target "base" {
    context = "basedir"
    tags = ["base:latest"]
}

target "apps" {
    name = "app_${version}"
    context = "appdir/${version}"
    matrix =  {
        version = ["1-0-0", "2-0-0", "2-1-5"]
    }
    tags = ["app:${version}"]
    contexts = {
        base = "target:base"
    }
}

target "special-app" {
    context = "appdir/special"
    tags = ["special-app:latest"]
    contexts = {
        app = "target:app_2-0-0"
    }
}

Now we commit and push a change to basedir/Dockerfile, which triggers a GitHub workflow that builds these targets on multiple runners using the matrix subaction and pushes them to our registry.

Currently the subaction generates the following matrix:

[{"target": "base"}, {"target": "app_1-0-0"}, {"target": "app_2-0-0"}, {"target": "app_2-1-5"}, {"target": "special-app"}]

This generates 5 workflow jobs running in parallel, but most of them spend time building dependent targets first, which other jobs are already building. This results in a lot of duplicate work which could be avoided.

Ideally the subaction would generate multiple matrices like this:

[
    [{"target": "base"}],
    [{"target": "app_1-0-0"}, {"target": "app_2-0-0"}, {"target": "app_2-1-5"}],
    [{"target": "special-app"}]
]

This way we could define multiple jobs that depend on the previous ones and supply them with one matrix each.

Snippet of an example workflow using multiple matrices

jobs:
  generate:
    runs-on: ubuntu-latest
    outputs:
      jobs_amount: ${{ steps.generate.outputs.jobs_amount }}
      matrices: ${{ steps.generate.outputs.matrices }}
    steps:
      - uses: actions/checkout@v6
      - id: generate
        uses: docker/bake-action/subaction/matrix@multiple-matrices

  stage_1:
    needs:
      - generate
    if: ${{ fromJson(needs.generate.outputs.jobs_amount) >= 1 }}
    strategy:
      matrix:
        include: ${{ toJson(fromJson(needs.generate.outputs.matrices)[0]) }}
    steps:
      # ...

  stage_2:
    needs:
      - generate
      - stage_1
    if: ${{ fromJson(needs.generate.outputs.jobs_amount) >= 2 }}
    strategy:
      matrix:
        include: ${{ toJson(fromJson(needs.generate.outputs.matrices)[1]) }}
    steps:
      # ...

  stage_3:
    needs:
      - generate
      - stage_2
    if: ${{ fromJson(needs.generate.outputs.jobs_amount) >= 3 }}
    strategy:
      matrix:
        include: ${{ toJson(fromJson(needs.generate.outputs.matrices)[2]) }}
    steps:
      # ...

---

We imagine that this usecase is common enough to be supported here and would like to contribute that feature, since we already made it for our own workflows.

Would you be interested in this feature?