Question: Synthetic tests in Datadog pass but CI report from the action fails due to a timeout

[wole-ada]

Hi there!
I'm currently using DataDog/synthetics-ci-github-action@v0.6.0 to run synthetics tests in a Github CI/CD workflow.
Some of the tests report a failure due to a timeout - what's strange is that the tests actual show as "Passed" in the Datadog dashboard.
wondering if there is a way to configure the result reporting timeout or if there is a limit we have to respect. Thanks in advance!

Failed run in CI due to result timeout

The same run in the Datadog dashboard

[peculiarism]

Hi @wole-ada, thank you for reaching out!

We have raised an engineering ticket which is in our backlog, and we will update this issue thread as soon as we have an update available.

Thanks,

[wole-ada]

Thanks @peculiarism! To add more info to the issue we're experiencing - after the test times out, clicking on the results URL leads to an error page that states that the CI results are not yet ready. It seems that the action receives a timeout event before the DD results are ready. I've linked a video to show the issue in context.
Hope that helps!

[wole-ada]

Hi @peculiarism! We've also tried extending the pollingTimeout config in datadog-ci.json to 10 minutes but to no avail.
It seems the reporting service may be taking some time to generate the results. Usually when the results link is clicked, there is a page with the test run results but instead we receive this page using this link: (https://app.datadoghq.com/synthetics/details/r8g-s72-fui/result/1888620009249587128?from_ci=true)

This is consistent with the other CI synthetic tests we run - it seems that although they pass in Datadog when viewed from the dashboard, the test reporter is not able to receive the results before the timeout

[yuta-hayashi]

@peculiarism
Hi, I have the same problem now.
Any progress on this?

[ruizb]

Hello, this problem is fixed in v0.9.1.

This new version bumps the default polling timeout from 2 minutes to 30 minutes. In addition, to make the action succeed in case of a test or batch timeout, you will have to provide a config file that has a "failOnTimeout": false property.

[JavaScriptBach]

After updating to 0.9.1, this action doesn't fail with timeout anymore, but it now take 10x longer to run. Are we running the synthetics sequentially now instead of in parallel?

[peculiarism]

Hi all,

With the release of Continuous Testing, tests running in CI will run sequentially.

In order to run tests in parallel, you will need to change your parallelization setting within the Continuous Testing Settings page. You can find a link to our documentation here