Fix intermittent transform e2e test failures

[ellatrix]

Description

Creating a new post for every this seems to resolve this. All other tests start from a new post as well. I restarted the build a few times and the transform e2e tests didn't fail. There's some other intermittent failures that should be resolved separately.

How has this been tested?

Screenshots

Types of changes

Checklist:

• My code is tested.
• My code follows the WordPress code style.
• My code follows the accessibility standards.
• My code has proper inline documentation.
• I've included developer documentation if appropriate.
• I've updated all React Native files affected by any refactorings/renamings in this PR. .

[gziolo]

It might slow down the execution of those tests. It needs to be confirmed whether the total execution time on Travis doesn't increase.

[ellatrix]

It will be slower for sure. But slower is better than failing all the time?

[ellatrix]

Most other tests start from a new post as well, that's nothing new.

[aduth]

I agree, it may be slower than it was previously, but this is a pattern we adopt in many other tests.

I would actually very much like to see an improvement to the implementation of createNewPost where we try to artificially reset the state of the editor to an initial state, rather than perform a full reload. But that can be considered a separate task.

[gziolo]

This one is quite unique, this test suite alone runs for 5 minutes on Travis.

[jorgefilipecosta]

I think these tests, proved useful and caught some bugs with transforms in the past. But now transforms are mostly stable and rarely change and given the complications, they are giving us I guess we can remove them and then include some specific tests for some transforms.

[jorgefilipecosta]

@ellatrix, would you prefer to update this PR to remove the tests instead, or I can submit a different PR that removes them?

[ellatrix]

Ok, let's remove them then.

[ellatrix]

@jorgefilipecosta I removed them in this PR.

[gziolo]

But now transforms are mostly stable and rarely change and given the complications, they are giving us I guess we can remove them and then include some specific tests for some transforms.

Yes, we definitely want to keep them but for a limited set of blocks 👍 Let's make sure we add a test suite in a follow-up PR. We should have at least text blocks (List, Quote, Paragraph), media blocks (Image, Gallery, Cover) and the Group block covered.

@jorgefilipecosta thinking about it a bit more, would it be possible to recreate this test suite by creating 3 sections (text, media, group) and listing fixtures for the source of blocks and run checks on the expected snapshots?

[aduth]

It's a bit surprising to me that it's necessary, given that the default implementation of click should be doing the scrolling already on our behalf:

https://github.com/puppeteer/puppeteer/blob/d17708ba1fc8411941bfd07f633aaf2fb5cb671e/lib/JSHandle.js#L278-L285

I wonder if it has to do with the fact that the element is in it's own scrollable region, not the page's default scroll area. If that's the case, seems like an area of improvement for Puppeteer to consider.

Edit: Related:

• puppeteer/puppeteer#2190 (comment)
• puppeteer/puppeteer#3347

[ellatrix]

Yeah, I stumbled upon the same code and the same issues. It's not exactly the same as element.scrollIntoView() will force the element to the top even if it's already in view.

[jorgefilipecosta]

LGTM 👍 I will try to follow up with some specific tests in the next few days.

[aduth]

Noting that I'm still observing an intermittent failure similar to what was intended to be addressed here.

FAIL packages/e2e-tests/specs/editor/blocks/quote.test.js (90.679s)
  ● Quote › can be created by converting a paragraph
    TimeoutError: waiting for selector ".block-editor-block-list__block[aria-label="Block: Quote"]" failed: timeout 30000ms exceeded
      at new WaitTask (../../node_modules/puppeteer/lib/DOMWorld.js:549:28)
      at DOMWorld._waitForSelectorOrXPath (../../node_modules/puppeteer/lib/DOMWorld.js:478:22)
      at DOMWorld.waitForSelector (../../node_modules/puppeteer/lib/DOMWorld.js:432:17)
      at Frame.waitForSelector (../../node_modules/puppeteer/lib/FrameManager.js:627:47)
      at Frame.<anonymous> (../../node_modules/puppeteer/lib/helper.js:112:23)
      at Page.waitForSelector (../../node_modules/puppeteer/lib/Page.js:1125:29)
      at page (../e2e-test-utils/build/@wordpress/e2e-test-utils/src/transform-block-to.js:18:8)
      at tryCatch (../../node_modules/regenerator-runtime/runtime.js:45:40)
      at Generator.invoke [as _invoke] (../../node_modules/regenerator-runtime/runtime.js:271:22)
      at Generator.prototype.<computed> [as next] (../../node_modules/regenerator-runtime/runtime.js:97:21)
        -- ASYNC --
      at Frame.<anonymous> (../../node_modules/puppeteer/lib/helper.js:111:15)
      at Page.waitForSelector (../../node_modules/puppeteer/lib/Page.js:1125:29)
      at page (../e2e-test-utils/build/@wordpress/e2e-test-utils/src/transform-block-to.js:18:8)
      at tryCatch (../../node_modules/regenerator-runtime/runtime.js:45:40)
      at Generator.invoke [as _invoke] (../../node_modules/regenerator-runtime/runtime.js:271:22)
      at Generator.prototype.<computed> [as next] (../../node_modules/regenerator-runtime/runtime.js:97:21)
      at asyncGeneratorStep (../../node_modules/@babel/runtime/helpers/asyncToGenerator.js:3:24)
      at _next (../../node_modules/@babel/runtime/helpers/asyncToGenerator.js:25:9)
          at runMicrotasks (<anonymous>)

At #18753, after a rebase to include revisions from this pull request.

[ellatrix]

@aduth Yeah that's odd. Not sure what is causing it. I'm also not immediately sure how to debug.