How to share matrix between jobs

[marten-seemann]

I’m trying to build a regression test suite that basically builds a client and a server binary from different release versions, and then runs them against each other.

builder:
  strategy:
    matrix:
      cfg:
        - { go: 1.13, commit: v1.0.0 }
        - { go: 1.14, commit: v1.2.0 }
  steps:
    // upload binary named "bin-${{ matrix.cfg.go }}-${{ matrix.cfg.commit }}"
runner:
  strategy:
    matrix:
      server:
        - { go: 1.13, commit: v1.0.0 }
        - { go: 1.14, commit: v1.2.0 }
      client:
        - { go: 1.13, commit: v1.0.0 }
        - { go: 1.14, commit: v1.2.0 }
  steps:
     - // download binary
     - // run each server against each client

As you can see, I have repeat the build matrix three times in my configuration, which is error prone when I remove / add versions there.

My first idea was to just use YAML anchors here, but it turns out that GitHub Actions doesn’t support those for whatever reason.

I also tried to define the matrix as an output in the builder job:

builder:
  outputs:
    matrix: ${{ toJson(matrix) }}

And then use it in the runner job:

runner:
  strategy:
    matrix:
      client: ${{ fromJson(needs.builder.outputs.matrix) }}
      server: ${{ fromJson(needs.builder.outputs.matrix) }}

This just results in a “A mapping was not expected” error message.

Any idea how I can solve this?

[weide-zhou]

Hi @marten-seemann,

Glad to see you in Github Community Forum!

To share matrix between jobs, please follow the usage of doc here , in your code the runner job cannot get the value from ${{ toJson(matrix) }} since format issue.

Please try below code for your case:

  job1:
    runs-on: ubuntu-latest
    outputs:
      matrix: ${{ steps.set-matrix.outputs.matrix }}
    steps:
    - id: set-matrix
      run: |
        echo "::set-output name=matrix::[{\"go\":\"1.13\",\"commit\":\"v1.0.0\"},{\"go\":\"1.14\",\"commit\":\"v1.2.0\"}]"
  builder:
    needs: job1
    runs-on: ubuntu-latest
    strategy:
      matrix: 
        cfg: 
          - ${{fromJson(needs.job1.outputs.matrix)}}
    steps:
    - run: |
        echo bin-${{ matrix.cfg.go }}-${{ matrix.cfg.commit }}
  runner:
    needs: [job1,builder]
    runs-on: ubuntu-latest
    strategy:
      matrix:
        server: 
          - ${{fromJson(needs.job1.outputs.matrix)}}
        client:
          - ${{fromJson(needs.job1.outputs.matrix)}}
    steps:
    - run: |
          echo ${{ matrix.server.go }}-${{ matrix.server.commit }}
          echo ${{ matrix.client.go }}-${{ matrix.client.commit }}

You can refer to my workflow for the details.

Thanks

[marten-seemann]

Hi @weide-zhou, thank you for your reply.

I was hoping to avoid the

echo "::set-output name=matrix::[{\"go\":\"1.13\",\"commit\":\"v1.0.0\"},{\"go\":\"1.14\",\"commit\":\"v1.2.0\"}]"

This code is not very readable, especially when I want to add 10 or so more configurations there.

[weide-zhou]

Hi @marten-seemann,

Since the job outputs must be string format(doc here), i’m afraid you have to pull all configurations in a line.

Thanks

[marten-seemann]

Hi @weide-zhou,

I’m wondering if it’s somehow possible to use the toJson function to feed a human-readable JSON object into the configuration file, something like this:

builder:
  strategy:
    matrix:
      cfg:
        - { go: 1.13, commit: v1.0.0 }
        - { go: 1.14, commit: v1.2.0 }
  outputs:
    matrix: ${{ steps.set-matrix.outputs.matrix }}
  steps:
    - id: set-matrix
      run: echo "::set-output name=matrix::${{ toJson(matrix.cfg) }}"
    // upload binary named "bin-${{ matrix.cfg.go }}-${{ matrix.cfg.commit }}"
runner:
  needs: [ builder ]
  strategy:
    matrix:
      server:
        - ${{ fromJson(needs.builder.outputs.matrix) }}
      client:
        - ${{ fromJson(needs.builder.outputs.matrix) }}
  steps:
     - // download binary
     - // run each server against each client

Unfortunately, this generates an error: Error when evaluating 'strategy' for job 'runner. (Line: xx, Col: xx): Error reading JObject from JsonReader. Path '', line 1, position 1.,(Line: xx, Col: xx): Error reading JObject from JsonReader. Path '', line 1, position 1.

And it looks like there’s no way to inspect the matrix output that was passed between the two jobs, so I’m having trouble debugging this error message.

[weide-zhou]

Hi @marten-seemann,

You cannot use toJson function transfer the job outputs, because:

1. format is not correct, set-output doesn’t accept format below, and miss \ to avoid escape characters.
   

2. The output is determined by the last check run if it’s a matrix job, cannot get accumulated value. Please check ticket below:
   [BUG] Jobs output should return a list for a matrix job

Hence, please put all configuration in a line with \.

Thanks

[marten-seemann]

Does that mean it’s not possible to have matrix jobs generated by a script? That seems like… a pretty basic thing for a CI, and I’d surprised to hear that it’s not possible on GitHub Actions.

I was thinking of putting the jobs in a separate file (as JSON), and read that file from disk. But that escaping requirement sounds like I can’t do that either (at least not without a script that would add the \s).

[marten-seemann]

I think I figured out a way to do this: I place the configuration in a separate file, matrix.json:

[
  { "go": "1.13", "commit": "v1.0.0" },
  { "go": "1.14", "commit": "v1.2.0" },
]

The config uses a separate matrix step to read this JSON file from disk:

matrix:
  runs-on: ubuntu-latest
  outputs:
    matrix: ${{ steps.set-matrix.outputs.matrix }}
  steps:
    - uses: actions/checkout@v2
    - id: set-matrix
      run: |
        TASKS=$(echo $(cat .github/workflows/matrix.json) | sed 's/ //g' )
        echo "::set-output name=matrix::$TASKS"
builder:
  needs: [ matrix ]
  strategy:
    matrix:
      cfg: ${{ fromJson(needs.matrix.outputs.matrix) }}
  steps:
    // upload binary named "bin-${{ matrix.cfg.go }}-${{ matrix.cfg.commit }}"
runner:
  needs: [ matrix, builder ]
  strategy:
    matrix:
      server: ${{ fromJson(needs.matrix.outputs.matrix) }}
      client: ${{ fromJson(needs.matrix.outputs.matrix) }}
  steps:
     - // download binary
     - // run each server against each client

The echo $(cat ...) remove all line breaks from the JSON file.
For some reason that I don’t fully understand it also seems necessary to remove all (?) spaces from the JSON string (sed 's/ //g').

Note that this solution also works if the matrix is dynamically generated. For example, it would be trivial now to write a script that outputs JSON, which is then used as a matrix in a later step.

[sigprof]

Actually this can be made to work, you just need a couple of minor adjustments to the code:

1. The matrix of your builder job needs to have only a single entry which contains the JS object data (represented in the YAML syntax inside the workflow). So you need to add an extra array around your data (or an object, which might be easier to represent in YAML in your case, because otherwise you would have an array of arrays there).
2. The set-output command supports only a single line of text, so you need to use a tool like jq to format the JSON data as a single line.
3. The only working way to get multiline input values (like the toJSON() output) into run is to pass them via environment variables.

builder:
  runs-on: ubuntu-latest
  strategy:
    matrix:
      cfg:
        - data:
          - { go: 1.13, commit: v1.0.0 }
          - { go: 1.14, commit: v1.2.0 }
  outputs:
    matrix: ${{ steps.set-matrix.outputs.matrix }}
  steps:
    - id: set-matrix
      env:
        CFG: ${{ toJSON(matrix.cfg) }}
      run: jq -r -n 'env.CFG | fromjson | @json "::set-output name=result::\(.data)"'
runner:

needs: [ builder ]

strategy:

matrix:

server: ${{ fromJson(needs.builder.outputs.matrix) }}

client: ${{ fromJson(needs.builder.outputs.matrix) }}

steps:

- // download binary

- // run each server against each client

At the moment such abuse of the matrix feature seems to be the only way to pass JSON data to actions while having it represented in the YAML form in the workflow file. (Of course you can wrap the data in a multiline string and pass it as an environment variable or an action input, but then you will lose any YAML syntax highlighting or other help from your editor, because it would treat that piece of data as a plain string.)

[SergiiShapoval]

https://vs-blogs.medium.com/dynamic-matrix-in-github-action-e516019cfdf5 (Approach 3) helped me

[cmoulliard]

It took me a while to figure the best approach and here is what I propose to todo to reuse a workflow where the configuration (YAML -> JSON) is parsed and saved as "matrix" key

1. Create first a reusable workflow exporting the output matrix
   https://github.com/ch007m/test-github-action/blob/main/.github/workflows/reusable-config.yml

on:
  workflow_call:
    # Map the workflow outputs to job outputs
    outputs:
      matrix:
        description: "Matrix of configurations"
        value: ${{ jobs.generate-json-matrix.outputs.matrix }}
...jobs:
  generate-json-matrix:
    name: Generate output
    runs-on: ubuntu-latest
    # Map the job outputs to step outputs
    outputs:
      matrix: ${{ steps.convert.outputs.matrix }}
    steps:
      - name: Checkout to repository
        uses: actions/checkout@v3

      - id: convert
        name: Convert config from YAML to JSON and pass it as value to matrix
        run: |
          echo "matrix=$(yq -P -o=json config.yaml | jq -c .)" >> $GITHUB_OUTPUT

2. From a workflow use it

https://github.com/ch007m/test-github-action/blob/main/.github/workflows/consume-config-matrix.yml

name: Call reusable workflow and use its outputs
on:
  workflow_dispatch:

jobs:
  job1:
    uses: ./.github/workflows/reusable-config.yml
  job2:
    runs-on: ubuntu-latest
    needs: job1
    strategy:
      matrix: ${{ fromJSON(needs.job1.outputs.matrix) }}
...
      - name: Log Java version
        if: always()
        run: |
          echo "Java version: ${{ matrix.java }}"

Result in action: https://github.com/ch007m/test-github-action/actions/runs/5833089159

Remark: There is one caveat as we cannot copy/paste the fields defined under the steps/jobs as yaml config file should include the matrix exploded

Wrong (see array declaration)

include:
- java: ["11", "17"] 
  builder-image= "quay.io/snowdrop/my_image:1.0"

Good (array removed)

- java: "11"  
  builder-image= "quay.io/snowdrop/my_image:1.0"
- java: "17" 
  builder-image= "quay.io/snowdrop/my_image:1.0"

Feel free to use this trick and to share it with others :-)

[helloint]

Thanks. I use this solution as base and added my requirement, which is to accept inputs.envs to narrow down the metrix

name: Read Matrix
on:
    workflow_call:
        inputs:
            envs:
                description: "Environments to filter (e.g., dev, qa1, or dev,qa1)"
                required: true
                type: string
        outputs:
            matrix:
                description: "Matrix of configurations"
                value: ${{ jobs.generate.outputs.matrix }}

jobs:
    generate:
        runs-on: ubuntu-latest
        outputs:
            matrix: ${{ steps.convert.outputs.matrix }}
        steps:
            - name: Checkout
              uses: actions/checkout@v4

            - name: Convert
              id: convert
              run: |
                # Convert the input envs to an array
                ENVS_ARRAY=$(echo "${{ inputs.envs }}" | tr ',' ' ')

                # Convert matrix.yml to JSON, filter the include array, and serialize to a single line
                MATRIX=$(yq -P -o=json .github/matrix.yml | jq --arg envs "$ENVS_ARRAY" -c '
                    .include |= map(select(.env == ($envs | split(" ")[])))
                ')

                # Output the filtered and serialized matrix
                echo "matrix=$MATRIX" >> $GITHUB_OUTPUT

xxx
jobs:
    matrix:
        uses: ./.github/workflows/read-matrix.yml
        with:
            envs: ${{ inputs.envs }}

    build:
        needs: [matrix]
        runs-on: ubuntu-latest
        strategy:
            matrix: ${{ fromJSON(needs.matrix.outputs.matrix) }}
        steps:
            - name: Build for ${{ matrix.env }}
            - xxxx

[tfoote]

Yaml anchors would be a standard way to do this, but they're unfortunately not supported in github's execution.

See thread https://github.com/orgs/community/discussions/4501 and issue actions/runner#1182