Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Testing: Upgrade puppeteer to v2.0 to fix the broken interactive mode #18205

Merged
merged 2 commits into from Oct 31, 2019

Conversation

@gziolo
Copy link
Member

gziolo commented Oct 31, 2019

Description

Reported by @diegohaz:

Does anyone know why I can’t run npm run test-e2e -- --puppeteer-interactive to see the browser? I always get a connect ECONNREFUSED 127.0.0.1:<PORT> error. The Chrome browser opens and closes really fast.

This PR resolves the issues by upgrading the puppeteer to the new major version. As usual, new major version produces some warnings during npm install but we can't do anything about it since external packages need to update their package.json entries.

How has this been tested?

npm run test-e2e -- --puppeteer-interactive

I confirmed it works with the latest version of puppeteer.

@gziolo gziolo force-pushed the fix/puppetteer-interactive branch from efed32d to e0ad9db Oct 31, 2019
@youknowriad

This comment has been minimized.

Copy link
Contributor

youknowriad commented Oct 31, 2019

We should use the new emulateMedia features https://github.com/GoogleChrome/puppeteer/releases to remove our "disable animations plugiin" and also the environment variable we use to disable animations. Instead we should just put the browser in "prefers-reduced-motion" mode.

@gziolo

This comment has been minimized.

Copy link
Member Author

gziolo commented Oct 31, 2019

We should use the new emulateMedia features https://github.com/GoogleChrome/puppeteer/releases to remove our "disable animations plugiin" and also the environment variable we use to disable animations. Instead we should just put the browser in "prefers-reduced-motion" mode.

Can you file a new issue? I don't want to block this PR as you can't debug code at the moment ...

@youknowriad

This comment has been minimized.

Copy link
Contributor

youknowriad commented Oct 31, 2019

Definitely not a blocker, just something I thought of when reading the changelog. I might give it a try after this PR get merged, it should be pretty easy.

@gziolo

This comment has been minimized.

Copy link
Member Author

gziolo commented Oct 31, 2019

Definitely not a blocker, just something I thought of when reading the changelog. I might give it a try after this PR get merged, it should be pretty easy.

You probably also know best how to tackle it. Ping me when you have PR ready, I can review pretty quick 👍

@gziolo

This comment has been minimized.

Copy link
Member Author

gziolo commented Oct 31, 2019

It looks like I upgraded axe and they introduced or updates some rules which cause some tests to fail. I will have to silence them for now, similar to other rules which Gutenberg violates. The rules are:

  • aria-hidden-focus
  • aria-input-field-name
@gziolo gziolo requested a review from talldan as a code owner Oct 31, 2019
@gziolo

This comment has been minimized.

Copy link
Member Author

gziolo commented Oct 31, 2019

It looks like I upgraded axe and they introduced or updates some rules which cause some tests to fail. I will have to silence them for now, similar to other rules which Gutenberg violates. The rules are:

  • aria-hidden-focus
  • aria-input-field-name

Done with 8de7f70, added to the list of rules to enable in #15452.

@ntwb
ntwb approved these changes Oct 31, 2019
Copy link
Member

ntwb left a comment

👍🏼

@gziolo gziolo merged commit ab59ea9 into master Oct 31, 2019
7 checks passed
7 checks passed
pull-request-automation
Details
Header rules - gutenberg-playground No header rules processed
Details
Pages changed - gutenberg-playground All files already uploaded
Details
Redirect rules - gutenberg-playground No redirect rules processed
Details
Mixed content - gutenberg-playground No mixed content detected
Details
Travis CI - Pull Request Build Passed
Details
netlify/gutenberg-playground/deploy-preview Deploy preview ready!
Details
@gziolo gziolo deleted the fix/puppetteer-interactive branch Oct 31, 2019
@afercia

This comment has been minimized.

Copy link
Contributor

afercia commented Oct 31, 2019

Thanks @gziolo! Regarding aria-hidden-focus please see my comment on #11631 (comment)

Regarding aria-input-field-name can you please point me at the log or the reference where it happens?

@gziolo

This comment has been minimized.

Copy link
Member Author

gziolo commented Oct 31, 2019

Thanks @gziolo! Regarding aria-hidden-focus please see my comment on #11631 (comment)

Regarding aria-input-field-name can you please point me at the log or the reference where it happens?

Sure thing, the good example is in https://travis-ci.com/WordPress/gutenberg/jobs/251398503#L1066"

FAIL packages/e2e-tests/specs/editor/blocks/table.test.js (19.372s)
  ● Table › displays a form for choosing the row and column count of the table
    expect(received).toPassAxeTests(expected)
    Expected page to pass Axe accessibility tests.
    Violations found:
    Rule: "aria-input-field-name" (ARIA input fields have an accessible name)
    Help: https://dequeuniversity.com/rules/axe/3.4/aria-input-field-name?application=axe-puppeteer
    Affected Nodes:
      tr:nth-child(1) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(1) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(1) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(1) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(1) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(2) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(2) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(2) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(2) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(2) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(3) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(3) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(3) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(3) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(3) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(4) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(4) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(4) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(4) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(4) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(5) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(5) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(5) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(5) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(5) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(6) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(6) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(6) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(6) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(6) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(7) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(7) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(7) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(7) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(7) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(8) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(8) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(8) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(8) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(8) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(9) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(9) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(9) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(9) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(9) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(10) > td:nth-child(1)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(10) > td:nth-child(2)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(10) > td:nth-child(3)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(10) > td:nth-child(4)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      tr:nth-child(10) > td:nth-child(5)
        Fix ANY of the following:
        - aria-label attribute does not exist or is empty
        - aria-labelledby attribute does not exist, references elements that do not exist or references elements that are empty
        - Element has no title attribute or the title attribute is empty
      178 | 	}
      179 | 
    > 180 | 	await expect( page ).toPassAxeTests( {
          | 	                     ^
      181 | 		// Temporary disabled rules to enable initial integration.
      182 | 		// See: https://github.com/WordPress/gutenberg/pull/15018.
      183 | 		disabledRules: [
      at toPassAxeTests (config/setup-test-framework.js:180:23)
          at runMicrotasks (<anonymous>)
      at Object.<anonymous> (config/setup-test-framework.js:218:2)
@afercia

This comment has been minimized.

Copy link
Contributor

afercia commented Oct 31, 2019

Ah! Thanks. That seems to relate to the table block: the editable within the table cells needs to be labelled. RIght now, there's nothing there to label the editables, just:
<div role="textbox" ....

Which, in ARIA terms, is equivalent to an input field with no label.

@youknowriad youknowriad added this to the Gutenberg 6.9 milestone Nov 11, 2019
CreativeDive added a commit to CreativeDive/gutenberg that referenced this pull request Nov 12, 2019
…WordPress#18205)

* Testing: Upgrade puppeteer to v2.0 to fix the broken interactive mode

* Testing: Disable new introduced rules in axe core which fail
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.