qase-cypress v2: separate test runs are created for each file with tests

[gibiw]

The problem is that Cypress creates a separate reporter for each test file. As a result, after running all the tests in each single file, the run test is marked as completed. Instead, it should be completed after running tests in all files.

There is a workaround:

1. Create a test run using an HTTP request:

   curl --request POST \
        --url https://api.qase.io/v1/run/<PROJECT_CODE> \
        --header 'Token: <TOKEN>' \
        --header 'accept: application/json' \
        --header 'content-type: application/json' \
        --data '
   {
     "title": "<TEST_RUN_TITLE>"
   }
   '

2. Set the QASE_TESTOPS_RUN_ID environment variable to a test run ID

3. Set the QASE_TESTOPS_RUN_COMPLETE environment variable to False

4. Complete the run test using an HTTP request:

   curl --request POST \
        --url https://api.qase.io/v1/run/<PROJECT_CODE>/<TEST_RUN_ID>/complete \
        --header 'Token: <TOKEN>' \
        --header 'accept: application/json'

[NickVolynkin]

Workaround 2

Since cypress-qase-reporter@2.0.0-beta.3 the reporter can create one test run and share the run ID with other instances via ENV variable.

To update to this new version, run

npm install --save-dev cypress-qase-reporter@beta

There is still a problem that each reporter instance will make an API request to complete the test run. However, instances that are still running will still be able to report their result.

To make sure that reporting to a completed test run is enabled, go to Project Settings, Test Run tab, and enable the setting Allow to add results for cases in closed runs.

Implementation details

Quoting from qase-javascript-commons@v2.0.0-beta.9:

Now reporter handles Qase test runs in the following way:

1. The first instance of the reporter creates a Qase test run and stores the run ID
   in the ENV variable QASE_TESTOPS_RUN_ID.
2. Other instances of the reporter read this variable and report test results
   to the existing test run.

Nothing has changed in cases when there is a single instance of a reporter or
when it is using a test run, created with other tools, such as with an API request
or manually in the Qase app.

[LucasBenic]

Hi, I'm getting the error
invalid reporter '[object Object]'
TypeError: invalid reporter '[object Object]'

Do I need to modify my config file?

  reporter: 'cypress-qase-reporter',
  reporterOptions: {
    mode: "testops",
    screenshotFolder: 'screenshots',
    videoFolder: 'videos',
    apiToken: ".....", // Access the environment variable directly
    projectCode: 'TESTE',
    logging: true,
    basePath: 'https://api.qase.io/v1',
    sendScreenshot: true,
    sendVideo: true, // Add this line to enable sending video
    runComplete: true,
    video: true
  },
});

[cskmnrpt]

Hi @LucasBenic,

If you are using V2-beta of qase-cypress, please find the updated configuration format here 🔗

[NickVolynkin]

Hey, @LucasBenic! The configuration format has changed in v2, check it out: https://github.com/qase-tms/qase-javascript/tree/main/qase-cypress#configuration

You should have something like this:

  reporter: [
    ['list'],
    ['playwright-qase-reporter',
      {
        mode: 'testops',
        fallback: 'report',
        testops: {
          project: 'TESTE',
          run: {
            complete: true,
            title: 'Playwright tests at ' + new Date().toISOString(),
          },
          uploadAttachments: true,
          defect: false,
        },
      }],
  ],

[LucasBenic]

Guys, thank you so much for your support
I am using this config https://github.com/qase-tms/qase-javascript/tree/main/qase-cypress#configuration
I am using cypress-qase-reporter@beta

This beta version requires an ID in the test title, I recently removed all the IDs from my tests to make it easier to replicate to the rest of the project.
This version was supposed to solve the issue that the test run was setting as complete without all the specs finished, but I do not see that solved, or I am misconfiguring something.

[LucasBenic]

My main point about that is that I have an integration with Slack, so whenever the test is completed, all the team members receive a notification about the tests, currently I have hundreds of test cases divided by specs. The Slack notification is being triggered as soon as the first spec is finished, missing all the rest of the tests.

For now, since I was not able to solve this issue with the Beta version, I think the best solution is to create and complete the test run by HTTP request.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.