Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change default Parameters for screenshot testing to be more relaxed #1684

Open
TatuLund opened this issue Oct 2, 2023 · 0 comments
Open

Comments

@TatuLund
Copy link
Contributor

TatuLund commented Oct 2, 2023

The default Parameter's in screen shot testing are strict. This means that tests will be flaky.

For example I am using in my add-on testing the following settings

        // We do screenshot testing, adjust settings to ensure less flakiness
        Parameters.setScreenshotComparisonTolerance(0.05);
        Parameters.setScreenshotComparisonCursorDetection(true);
        testBench().resizeViewPortTo(800, 600);
        Parameters.setMaxScreenshotRetries(3);
        Parameters.setScreenshotRetryDelay(1000);

The default view port of app testing should be larger than 800 x 600 I use for add-ons. Point here is that we should ensure some fixed default size, since otherwise the view port size can wary.

More over I sometimes run tests also in Development mode, when dev mode gizmo needs to be hidden

        // Hide dev mode gizmo, it would interfere screenshot tests
        try {
            $("vaadin-dev-tools").first().setProperty("hidden", true);
        } catch (NotFoundException e) {

        }

Screenshot testing would be more approachable if the defaults would be something like this. Now developers need to consume time on trial and error to find these.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: 🅿️Parking lot - under consideration
Development

No branches or pull requests

1 participant