[Test] Add tag cloud visualization to dashboard in functional test for reporting #87600

tsullivan · 2021-01-07T01:11:21Z

Summary

Problem: A blocking issue came up that a Tag Cloud panel in a Dashboard shows blank in PDF and PNG reports. That problem should have been caught by automated testing when it was first introduced.

I looked into it and found there is a PNG test in the Dashboard x-pack functional app tests, but the dashboard used for the test is a custom one that doesn't have a tag cloud visualization.

This PR updates the test to include a tag cloud panel. Its saved object data can be found in the data.json.gz file that is changed in this PR.

It has changes to clarify how the comparePNGs function works and lowers the threshold of delta percentage to 85%

Checklist

Delete any items that are not applicable to this PR.

Unit or functional tests were updated or added to match the most common scenarios

elasticmachine · 2021-01-07T23:54:54Z

Pinging @elastic/kibana-app-services (Team:AppServices)

tsullivan · 2021-01-07T23:56:29Z

x-pack/test/functional/apps/dashboard/reporting/screenshots.ts

+        const url = await PageObjects.reporting.getReportURL(60000);
+        const res = await PageObjects.reporting.getResponse(url);
+
+        expect(res.status).to.equal(200);


Note

Unfortunately, all that the PDF tests are doing is verifying that the job completes successfully. That is ensured by the 200 response.

@elastic/kibana-app-services we need to think of a good way to validate the contents of a PDF file against a baseline. We have an implementation for PNG but not PDF.

tsullivan · 2021-01-14T05:28:55Z

@elasticmachine merge upstream

tsullivan · 2021-01-14T17:05:53Z

@elasticmachine merge upstream

rashmivkulkarni · 2021-01-15T18:50:40Z

x-pack/test/functional/apps/dashboard/reporting/screenshots.ts

          sessionReportPath,
          getBaselineReportPath(reportFileName, 'png'),
          config.get('screenshots.directory'),
          log
        );

-        expect(percentSimilar).to.be.lessThan(0.1);
+        expect(percentDiff).to.be.lessThan(0.09);


with value being (0.08) : I hit this error

at Assertion.assert (/Users/rashmikulkarni/Documents/kibana/packages/kbn-expect/expect.js:100:11) at Assertion.lessThan.Assertion.below (/Users/rashmikulkarni/Documents/kibana/packages/kbn-expect/expect.js:336:8) at Function.lessThan (/Users/rashmikulkarni/Documents/kibana/packages/kbn-expect/expect.js:531:15) at Context.<anonymous> (test/functional/apps/dashboard/reporting/screenshots.ts:132:35) at Object.apply (/Users/rashmikulkarni/Documents/kibana/packages/kbn-test/src/functional_test_runner/lib/mocha/wrap_function.js:84:16)

changing it to (0.09) - solves it.

kibanamachine · 2021-01-15T21:08:30Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: e388205

Metrics [docs]

✅ unchanged

History

💚 Build #98937 succeeded 55a1adc
💚 Build #98767 succeeded 3c0cc08
💚 Build #98135 succeeded 51f48ea
💚 Build #97741 succeeded a4c7403
💚 Build #97715 succeeded 57845d2

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

rashmivkulkarni

LGTM - post the changes - it is passing locally and on CI.

Thanks Tim!

…r reporting (elastic#87600) * add tag cloud to dashboard in kibana archive * add pdf snapshot testing to downloaded pdfs * update png baseline image * remove pdf snapshot testing * rename mislabeled variable * fix test comparison * remove unnecessary pdf utf8 checks * Update screenshots.ts Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

* master: (33 commits) [Security Solution][Case] Fix patch cases integration test with alerts (elastic#88311) [Security Solutions][Detection Engine] Removes duplicate API calls (elastic#88420) Fix log msg (elastic#88370) [Test] Add tag cloud visualization to dashboard in functional test for reporting (elastic#87600) removing kibana-core-ui from codeowners (elastic#88111) [Alerting] Migrate Event Log plugin to TS project references (elastic#81557) [Maps] fix zooming while drawing shape filter logs errors in console (elastic#88413) Porting fixes 1 (elastic#88477) [APM] Explicitly set environment for cross-service links (elastic#87481) chore(NA): remove mocha junit ci integrations (elastic#88129) [APM] Only display relevant sections for rum agent in service overview (elastic#88410) [Enterprise Search] Automatically mock shared logic files (elastic#88494) [APM] Disable Create custom link button on Transaction details page for read-only users [Docs] clean-up vega map reference documenation (elastic#88487) [Security Solution] Fix Timeline event details layout (elastic#88377) Change DELETE to POST for _bulk_delete to avoid incompatibility issues (elastic#87914) [Monitoring] Change cloud messaging on no data page (elastic#88375) [Uptime] clear ping state when PingList component in unmounted (elastic#88321) [APM] Consistent terminology for latency and throughput (elastic#88452) fix copy (elastic#88481) ...

kibanamachine · 2021-01-19T05:14:39Z