Can I generate two separate test results for two consecutive workflows?

[denil-ct]

I have two consecutive workflows where in the UI tests scheme and Unit tests scheme are run in parallel. Instead of having two separate reports I'm getting a single report. That is I'm getting the first report from the first workflow that completes, but then it gets overwritten by the second workflow when that completes. Is it how its supposed to work or am I missing anything?

[kishikawakatsumi]

Try to give a different title in the title parameter of each xcresulttool step.

If that doesn't fix the problem, I think it's my bug. I'm not familiar with GitHub Actions, so I'm not sure if I can fix it.

[denil-ct]

Thanks, let me try that

[denil-ct]

Didn't work, still got overwritten

[kishikawakatsumi]

Can you share your yml file and the result? Is it an open source project?

[denil-ct]

No its not an open source project. However, I will share maximum possible info

In unit_tests.yml

- name: Running test cases
        run: >-
          xcodebuild test
          -workspace $WORKSPACE
          -scheme $SCHEME
          -sdk iphonesimulator
          -destination 'platform=iOS Simulator,name=iPhone 11,OS=15.0'
          -skip-testing:UITests
          -resultBundlePath UnitTestResults
          
      - uses: kishikawakatsumi/xcresulttool@v1.3.1
        with:
          path: UnitTestResults.xcresult
          title: Unit Test Results
        if: success() || failure()

In ui_tests.yml

- name: Running test cases
        run: >-
          xcodebuild test
          -workspace $WORKSPACE
          -scheme $SCHEME
          -sdk iphonesimulator
          -destination 'platform=iOS Simulator,name=iPhone 11,OS=15.0'
          -skip-testing:UnitTests
          -resultBundlePath UITestResults
          
      - uses: kishikawakatsumi/xcresulttool@v1.3.1
        with:
          path: UITestResults.xcresult
          title: UI Test Results
        if: success() || failure()

The result would vary on which of these workflows finishes first. If ui_tests.yml completes the result will be associated to that workflow and we can see it properly, but after unit_tests.yml completes the ui_tests result gets overwritten by the new workflow and the result association still lies with the former one.

[kishikawakatsumi]

I would like to reproduce the problem. Are these two workflows executed as one action instead of being executed as independent actions? What kind of triggers are you setting?

[denil-ct]

Two separate actions
However both are triggered on pushing to the branch

[mikenachbaur-okta]

I'm encountering the same problem. Two different workflows run in parallel (unit tests and UI tests) and the results of one get overwritten by the other. It also seems interesting that the test results show up in the wrong workflow, which might be the problem.

My repo has 4 different workflows:

• Lint
• DocC Documentation
• Unit Tests
• UI Tests

Lint finishes first, Documentation second. When the Unit Tests complete, and the xcresulttool results are processed, the new job that xcresulttool creates ends up being attached to the Documentation workflow, instead of the Unit Test one. This might shed some light on the root cause of this problem.

[dan-dialpad]

This is also happening for our project.

When I kick off two workflows at similar times that both have xcresulttool stages only one result is processed if they end near the same time. If the work flows end at very different times both reports show up. Using unique resultBundlePath / path attributes and unique titles for the different workflows seems to have no effect.

Do you have any other suggestions to try?