[v5] Cypress Test Launches do not complete successfully.

[drewrichardson2]

Describe the bug
None of my launches finish. They get stuck in "in progress" until they time out after a day. We are using cypress along with the report portalagent-js-cypress plugin.

To Reproduce
We have just setup a fresh report portal installation in Kubernetes. I have tried running single cypress tests, full suites, and through circle CI and have not had a single launch finish properly. I've used custom launch configurations and "out of the box" configurations.

Expected behavior
Launches should successfully finish.

Screenshots

[drewrichardson2]

In addition here's what I see when I run a single spec file and drill down:

The bottom child of the run shows complete:

As does the parent of that child:

However, it's at the spec level it doesn't complete:

[drewrichardson2]

As an additional update, it seems directly related to the automerge option. I turned it off and the run completed.

[sbarber51]

Unfortunately I am seeing the same thing. Report Portal is the only reporter that I am using, and I am not using any plugins or libraries that should interfere.

[pdobrosi]

I see the same issue with

• Report portal 5.3.2
• Cypress 4.1.0 also the same with latest 6.2.1
• Node JS 14.5.0

[BibinQA]

I also see the same issue. with autoMerge : true, the launches never complete eventhough the run has completed. if I turn off autoMerge, no issues

[tamasmagyar-bridge]

Can we have an update on this issue, please?

[akbarchandani]

Hi,

Can we have an update on this issue. I am currently doing a POC and auto-merge functionally with email notification is an essential feature for us.

Thanks

[schomper]

We have come across this issue also. It looks like the plugin counts all the files that matches your testFiles regex to find a total number of launches and then compares that the the amount of launches it has completed.

This means that your launches not completing could be due to you:

1. Not defining the name of your test files strictly enough, meaning it counts all the files instead of just the .feature for example.

2. Running a subset of your tests. An example might be testing a specific flag like "smoke test". While cypress only runs the 2 smoke test files, the plugin expects it to run the other test files also and so it never completes.

The 2nd point is really what needs to be looked at. Just because a user has 6 tests files doesn't mean the run will cover all 6 of the files, we need a smarter way of determining how many suites are being run for a specific test set.

I believe issues #62 and possibly #63 are also related to this issue.

[zsbutosi]

We had the same problem. And after digging into it, I found that the problem was with the following:
Our tests run in docker image, where the test execution started from entrypoint.
The problem was that when the tests finished, the docker stopped too.
But the report portal plugin did not sent the results yet, and did not finished the test launch either.
I solved the problem, by creating a script that runs the tests, and wait for 15 sec for the reporting to finish.
And I also modified the entrypoint of the docker image to start this script, instead of starting the tests directly.
This way the test launch is closed now, and all the test result is uploaded before the docker image stops.

[jitindoriya07]

Hey @zsbutosi can you please share us script which solved your case.

[zsbutosi]

@jitindoriya07 Here is my solution:

#!/bin/bash
echo "Test execution started!"
# Start the test execution
<command to start cypress test>
# Saving cypress exit code
exitCode=$?
# Wait for the reporting to report portal finish before stop the container
for i in `seq 1 15`;
do
  sleep 1
  echo "wait for test stopping - $i"
done
echo "Test execution finished!"
# Returning with cypress exit code
exit $exitCode

[piopi]

Found a workaround with autoMerge: true, the latest version of reportportal/client-javascript (it is a dependency of agent-js-cypress) added a timeout option. Add the following option in your cypress.json in the reporterOptions

"restClientConfig":{
          "timeout":300000
        }

and for people needing to wait for Cypress to finish sending the result before exiting (like in a Docker container) here is a bash script:

#!/bin/bash
waitall() { 
    echo "Sending results to Report Portal ..."
    for pid in "$@"; do
        name=$(ps -p $pid -o comm=)
        if [ $? -eq 0 ]
        then
            echo "Waiting ..."
            while ps -p $pid > /dev/null; do sleep 1; done;
        else
            echo "Cypress already done" 
        fi
    done
}
cypress run --project .
# Wait for Cypress to finish sending the results to reportPortal
waitall $(pidof Cypress)
echo "Done"

[BibinQA]

Thank you for that setting with timeout. But I am still seeing the issue with test not being marked complete. We run our test in CircleCI. Test results do get pushed to RP and are auto merged. But the status is in-progrees in RP even though the test have completed running. CircleCI timeout! and the pipeline fails.
Cypress: 8.0.0
Report Portal: 5.3.0
agent-js-cypress: 5.0.2

[brabenetz]

I had this problem with the 120 Demo Tests (created by Cypress if no tests exist).
Changing plugins/index.js to add the config solved that Problem:

I also had to increase the timeout (30s where far too low) in the ReporterOptions as described from @piopi :

restClientConfig: {
    // timoutconfig: https://github.com/reportportal/client-javascript#timeout-30000ms-on-axios-requests
    timeout: 300000, // (5m) default 30000 (30s)
},

Report-Portal should fix that, or at least update the documentation:

agent-js-cypress/README.md

Line 38 in cc0a5d8

module.exports = (on) => registerReportPortalPlugin(on);

Should be: module.exports = (on, config) => registerReportPortalPlugin(on, config); as seen in the example project:
https://github.com/reportportal/examples-js/blob/abc81c411c74556af5dce47c59ff4c166c65707c/example-cypress/cypress/plugins/index.js?plain=1#L19

Additional Info:

I set the reportOptions dynamically in plugins/index.js because only on Jenkins the results should be reported to the reportportal.

I also use the cypress-multi-reporters plugin

"reporter": "cypress-multi-reporters",
"reporterOptions": {
    "reporterEnabled": "cypress-mochawesome-reporter, mocha-junit-reporter, @reportportal/agent-js-cypress",
    "mochaJunitReporterReporterOptions": {
        "mochaFile": "target/surefire-reports/e2e-results-demo-[hash].xml"
    },
    "cypressMochawesomeReporterReporterOptions": {
        "reportDir": "target/site/e2e-tests-demo",
        "charts": true,
        "embeddedScreenshots": false,
        "inlineAssets": false,
        "reportPageTitle": "Cypress-Tests"
    },
    "reportportalAgentJsCypressReporterOptions": {
         ...
    }
},

[BibinQA]

The original issue of autoMerge=true still exists for me after trying all the above steps. @brabenetz do you have autoMerge set to true in your reporter options

[brabenetz]

yes with autoMerge=true.

[CassenH]

and for people needing to wait for Cypress to finish sending the result before exiting (like in a Docker container) here is a bash script:

#!/bin/bash
waitall() { 
    echo "Sending results to Report Portal ..."
    for pid in "$@"; do
        name=$(ps -p $pid -o comm=)
        if [ $? -eq 0 ]
        then
            echo "Waiting ..."
            while ps -p $pid > /dev/null; do sleep 1; done;
        else
            echo "Cypress already done" 
        fi
    done
}
cypress run --project .
# Wait for Cypress to finish sending the results to reportPortal
waitall $(pidof Cypress)
echo "Done"

@piopi I tried this but it seems that Cypress never finishes. Did you add anything else to ensure that the process ends? I believe this issue is caused by the point that schomper addressed:

Running a subset of your tests. An example might be testing a specific flag like "smoke test". While cypress only runs
the 2 smoke test files, the plugin expects it to run the other test files also and so it never completes.

[tvamev]

sch

Wow. This answer from @schomper is right! I had to pass individual file names instead of mentioning *.spec.js etc. Also if i pass all spec files from command line also, it does not work. I need to mention all spec files in the cypress json file. Then only it works without any issues.

[luizwurlitzer2]

i was facing the same error, what helped was maintain in the test directory only the specs which will be executed and in the command line:

npx cypress run --spec "./cypress/e2e/*.cy.js"

seems like the autoMerge count how many specs file is in the directory and keep waiting for all them finish.