Preparation work for Supporting matrix context in output keys

[ajaykn]

Please note that this is not supported yet, and Server side changes are still in-progess

we will release a changelog when supported end to end.

Currently outputs for matrix jobs are represented by a single output and only the last successful matrix job is used to populate the output. This prevents users from being able to target output from a specific job and can lead to unpredictable results since the matrix jobs don't run in a guaranteed order.

So as part of the improvements, we want to support
Allow the matrix context to be used to define the output variable name. This allows users to make the output completely deterministic if they choose to include all dimensions of the matrix in their key.

outputs:
  ${{ matrix.os }}_${{ matrix.arch }}_result: ${{ steps.step1.outputs.test }}

Example using all dimensions in output key:

jobs:
  job1:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        os: ['linux', 'windows','macos']
        arch: ['x86','x64']
    outputs:
      ${{ matrix.os }}_${{ matrix.arch }}_result: ${{ steps.step1.outputs.test }}
    steps:
      - id: step1
        run: "test=${{ matrix.os }} - ${{ matrix.arch }}" >> $GITHUB_OUTPUT
  job2:
    runs-on: ubuntu-latest
    needs: job1
    steps:
      - run: echo ${{ needs.job1.outputs["linux_x86_result"] }}

Please note that: Server side changes are still in-progess.

[0x53A]

hallelujah, this is great news! Thank you.

Please note that: Server side changes are still in-progess

is there a public issue we can watch?

[srosell-ut]

Hey @ajaykn 🖖 !
this is just what we were looking for but although it seems to be merged and deployed in the release v2.303.0 it seems to not work as the example shows. We've tried and we get the following error:

The workflow is not valid. .github/workflows/test.yaml (Line: 16, Col: 7): Unrecognized named-value: 'matrix'. Located at position 1 within expression: matrix.os .github/workflows/test.yaml (Line: 24, Col: 14): Unexpected symbol: '"linux_x86_result"'. Located at position 20 within expression: needs.job1.outputs["linux_x86_result"]

We just copy/paste you example as it was not working in our use case to see if we where doing something wrong but we have the same error.

Are we missing something? 🤔 I guess it is still something missing due to the side-note:

Please note that: Server side changes are still in-progess

Thanks for the work 🙇

[SvenStaehs]

Love it, just started getting stuck on this issue. To my untrained eye, the changes look like they only affect job outputs, not workflow outputs (for reusable workflows where the workflows are called with a matrix), is that a separate change? Because trying the same syntax on those gives me Unexpected symbol: '"build-image-${{' trying to dereference in the workflow that needs this as an input.

Edit: I re-checked the example and noticed it uses a hard-coded string to resolve the output, not something like echo ${{ needs.job1.outputs["${{ matrix.os }}_${{ matrix.arch }}_result"] }}, which is what I need: My job2 runs on the same matrix as job1 and the respective second jobs are supposed to use the outputs of the respective first jobs that ran for the same matrix values... So maybe this implementation doesn't really cover my specific use-case completely?

[griseau]

Such a great improvement ! Do you have any ETA for this ? So that I know if I need to make a workaround or can just wait !
Thanks a lot

[caio-eiq]

@ajaykn is this already working on v2.303.0?
I am running into a similar error as mentioned by @srosell-uz

[ajaykn]

@srosell-uz , @caio-eiq

Please note that there are still few more changes from server side.
We will release a changelog / blog post once we support end to end.

@griseau
Currently under internal testing and will be released in another 2-3 weeks.

@SvenStaehs

To my untrained eye, the changes look like they only affect job outputs, not workflow outputs (for reusable workflows where the workflows are called with a matrix), is that a separate change?

We wil share more in blog post once the feature is completely baked.
To give you initial idea, for reusable wf outputs, we were thinking to support via inputs context.

caller workflow

job1:
    strategy:
      matrix:
        os: ['linux', 'windows','macos']
        arch: ['x86','x64']
    uses: ./.github/workflows/called1.yml
    with:
      os: ${{ matrix.os }}
      arch: ${{ matrix.arch }} 
job2:
    runs-on: ubuntu-latest
    needs: job1
    steps:
      - run: echo ${{ needs.job1.outputs["linux_x86_result"] }}

called workflow

on:
  workflow_call:
    inputs:
      os:
        type: string
      arch:
        type: string
    outputs:
      ${{ inputs.os }}_${{ inputs.arch }}_result:
        value: ${{ jobs.job1.outputs.result}}

Edit: I re-checked the example and noticed it uses a hard-coded string to resolve the output, not something like echo ${{ needs.job1.outputs["${{ matrix.os }}_${{ matrix.arch }}_result"] }}, which is what I need: My job2 runs on the same matrix as job1 and the respective second jobs are supposed to use the outputs of the respective first jobs that ran for the same matrix values... So maybe this implementation doesn't really cover my specific use-case completely?

yes currently, we will not be supporting expressions inside an expression like ${{ needs.job1.outputs["${{ matrix.os }}_${{ matrix.arch }}_result"] }}

[ChristopherHX]

expressions inside an expression is not required to access composed output keys.

${{ needs.job1.outputs["${{ matrix.os }}_${{ matrix.arch }}_result"] }}

This can be rewritten to ${{ needs.job1.outputs[format('{0}_{1}_result', matrix.os, matrix.arch)] }}, only downside is that syntax is harder to write (and read) for workflow authors.

[brianjmurrell]

Could we get an update when the server-side bits have been completed and are live so that the change described here can work please?

(Note I am not asking for an ETA, just a note here when it's complete so that we know we can start using this change.)

[nicholasbergesen]

Hi all, unfortunately the team working on this feature has been laid off. I have no information on whether or not the work will be continued by another team or if the work has stopped completely. Its quite sad as it was very close to completion.

cc @mph4

[brianjmurrell]

This is a real pity as this is a pretty serious feature gap. Not being able to get matrix per intersection outputs seriously impacts the usability of the matrix feature.

[qc00]

There's no design documentation linked, so I am not sure what alternative syntax was considered, but templating in the field name seems to be a bit limiting because the user of the outputs will then have to enumerate all the needed field names somehow.

This will either lead to a hard-coded, manual expansion of the matrix combinations, or some convoluted code to reuse the matrix definition.

Why not allow nested objects so we can use the existing [object filter] feature?

For example:

outputs:
  my_output:
    key: ${{matrix.os}}
    nested:
      key: ${{matrix.arch}}
      value: ...

# Usage:
${{jobs.x.outputs.my_output.*.*}}
${{jobs.x.outputs.my_output.linux.*}}
${{jobs.x.outputs.my_output.*.x64}}

The last two use cases are almost impossible with the current syntax

[MatthiasValvekens]

This will either lead to a hard-coded, manual expansion of the matrix combinations, or some convoluted code to reuse the matrix definition.

Wouldn't that be solvable with strategy.job-index as a unique identifier?

[andrewstec]

Can we please get an update on this? I understand the layoffs have occurred. Is there a PM/team who still has this in the product roadmap or is this indefinitely scrapped?

[RafPe]

Having this feature would make dynamic CICD with github actions a 💥 If there is any help we as community could contribute please let us know!

[patrickmoore-nc]

Just investigated parameterising a few similar jobs using matrix and I am in total disbelief this functionality wasn't provided from the get-go. It's something that every user of the matrix feature would obviously want.

[Zermond]

Does it work in cloud runner? Because I got error just like srosell-ut

[RafPe]

It will not work until Github pushes backend changes to support this 😢

[cdata]

This change landed almost 6 months ago and there isn't any way to track progress on whatever other changes are blocking it. Can we get an update on the outlook for this feature? Or, perhaps, a reference to another issue to follow?

[RafPe]

I have taken a try to get the attention on Github by sharing a callout on LinkedIn platform! https://www.linkedin.com/posts/raftech_preparation-work-for-supporting-matrix-context-activity-7120734515211161600-12jD

Lets try to get their attention to this in every possible way. As automation/platform engineer this is a killer feature which would enable us to automate beyond what we got atm!

[elviskahoro]

We should loop in some of GitHub's developer advocates, especially given GitHub Universe starts tomorrow. Will try to get back with a list of folks

[shapirus]

is there a public issue we can watch?

I second this question.

[kishaningithub]

Given the importance of this i could not stop myself from posting this (meme below)

[zjplab]

Any new update so far on this topic?

[shinebayar-g]

No we're working on AI.

[RDhar]

While it's a bit niche, worth mentioning how Microsoft's other tool, Playwright (the E2E test runner w/ browser automation) handles the challenge of merging reports from multiple parallel tests.

Since they have native support for sharding (scaling parallel test execution by running tests on multiple machines simultaneously), they've shared a GH Actions example which leverages actions/upload-artifact and actions/download-artifact as follows:

jobs:
  run-tests:
    runs-on: ubuntu-latest
    strategy:
      fail-fast: false
      matrix:
        shardIndex: [1, 2, 3, 4]
        shardTotal: [4]

    steps:
    - name: Run Playwright tests
      run: npx playwright test --shard=${{ matrix.shardIndex }}/${{ matrix.shardTotal }}

    - name: Upload blob report to GitHub Actions Artifacts
      if: always()
      uses: actions/upload-artifact@v4
      with:
        name: blob-report-${{ matrix.shardIndex }}
        path: blob-report
        retention-days: 1

  merge-reports:
    # Merge reports after run-tests, even if some shards have failed
    if: always()
    needs: [run-tests]
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v4
    - uses: actions/setup-node@v4
      with:
        node-version: 18
    - name: Install dependencies
      run: npm ci

    - name: Download blob reports from GitHub Actions Artifacts
      uses: actions/download-artifact@v4
      with:
        path: all-blob-reports
        pattern: blob-report-*
        merge-multiple: true

    - name: Merge into HTML Report
      run: npx playwright merge-reports --reporter html ./all-blob-reports 

    - name: Upload HTML report
      uses: actions/upload-artifact@v4
      with:
        name: html-report--attempt-${{ github.run_attempt }}
        path: playwright-report
        retention-days: 14

While the added HTML-report related steps are specific for Playwright's use-case, the merge-multiple: true line is the crux here for pulling together the partial blob reports together, as required.

It's not exactly straightforward by a long-shot, but it is an "officially" recommended method for merging outputs together cohesively from prior jobs running matrix strategy.

[SvenStaehs]

well that's a bit off-topic. Uploading multiple files to a single artifact was originally possible and was only broken with release v4 of upload-artifacts. So what they do with merge-multiple: true in that extra job is more like a workaround for this new restriction. Good to know this exists, so thanks for sharing.

But what we wanted to see from this PR was not about artifacts but about outputs: A job that runs in matrix and produces output variables -- e.g. the digest of the docker image it just created -- is currently useless (unless in specific circumstances where you don't care which job's output you consume because they're all going to be the same). So in the same way the upload-artifacts action accepts the input name: blob-report-${{ matrix.shardIndex }}, we wanted to be able to define a job-level output with matrix value in its name:

outputs:
  ${{ matrix.os }}_${{ matrix.arch }}_result: ${{ steps.step1.outputs.test }}

It has been suggested to use artifacts as a workaround for this: Instead of using output variables, write the values into a file that is then exposed as artifact. The file will have the matrix values in its name so you can load that file specifically and use the values it defines (could be a .env file). So this isn't entirely unrelated, but only as a matter of workarounds for workaround 😉

[RDhar]

I'm with you, @SvenStaehs, this is an (inconvenient) workaround that barely serves the purpose, at least until GH Actions actually begins to support multiple outputs from matrix jobs like this PR suggests it should.

Between this and passing secrets between jobs, I'm hoping for a lot from the Actions team! 🤞