Race condition for interaction when using Playwright for e2e testing

[pcapel]

Environment

• Elixir version (elixir -v):

Erlang/OTP 25 [erts-13.1.4] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] [jit]

Elixir 1.14.3 (compiled with Erlang/OTP 25)

• Phoenix version (mix deps):

* phoenix 1.7.0-rc.3 (Hex package) (mix)
  locked at 1.7.0-rc.3 (phoenix) ae651c30

• Phoenix LiveView version (mix deps):

* phoenix_live_view 0.18.14 (Hex package) (mix)
  locked at 0.18.14 (phoenix_live_view) 4bee1502

• Operating system: macOS Ventura Version 13.0
• Browsers you attempted to reproduce this bug on (the more the merrier): N/A
• Does the problem persist after removing "assets/node_modules" and trying again? Yes/no: N/A

Actual behavior

Playwright failures due to behavior not triggering.

Expected behavior

All expected behavior should trigger during tests.

---

This issue is not strictly related to Phoenix or LiveView, but is an issue of integration with Playwright for end-to-end testing. I recognize that and am hoping to get feedback on the actual cause of the issue, as well as a potential path forward that is better than my current solution.

Let me expand a bit. I am trying to write an integration/e2e test for a feature involving a live component form within a modal. In the course of writing the test I ran into failures that were frustratingly disappearing during debug runs of the test (using npx playwright <testname> --debug). This was my first clue that the issue was a race condition.

To verify what I believe is occurring, I logged out the sent frames over the web-socket using:

  page.on("websocket", (ws) => {
    ws.on("framesent", (event) => console.log(JSON.parse(event.payload)));
  });

The outcome of this is that I can see passing tests when the event I expect is successfully triggered. The tests fail when that event is not present in the logged output.

This leads me to my hypothesis: there is a race condition between interaction of playwright and phoenix live view wiring up the click handlers. That is, Playwright is able to interact with the rendered page before the live view javascript can be connected to the click events.

I'm able to move past this issue using a set timeout, but that comes with all the normal issues that one expects.

I'm hoping someone might have a bit more insight into whether or not there is an existing solution here. I looked into various DOM/browser events, but I was unable to locate anything that related to triggers on event handlers. In my opinion, the ideal case here would be to await the correct handler being attached to the DOM element. So if that's currently possible and I just don't now how, that's excellent!

Or perhaps I'm completely wrong in my hypothesis! That would be extra exciting because I'd probably learn something nifty.

Regardless I wanted to bring some attention to this since I really like using both playwright and Phoenix.

I can also produce a minimal example of this if that's helpful. I wasn't sure which side made the most sense for this issue. If this is outside of scope for phoenix, then I can also raise an issue in the playwright project and see if they have ideas.

[josevalim]

Hi @pcapel , I recommend using ElixirForum for this, as others may have Playwright experience. We have limited time and we would rather focus on already reproducible and isolated bugs, as well as features.

[josevalim]

Btw, we emit certain events on live view page load, perhaps you could hook into those. But I have no actual idea.

[chrismccord]

See phx:page-loading-stop

[pcapel]

Thanks! I'll be sure to use the forum for this sort of thing in the future. Can't believe how fast y'all responded. I'm going to try and work out a solution using the phx:page-loading-stop and create a post about it in the forum.

[rickclare]

In case this helps others: I had a similar issue when using Playwright with LiveView - failing/flaky tests that only could be solved by slowing down the tests (e.g. by adding manual page.waitForTimeout(1000) between test steps)

The following solution worked for us:

After visiting a live-view page, wait for the CSS class .phx-connected to be present, before interacting with the page.
See https://hexdocs.pm/phoenix_live_view/bindings.html#loading-states-and-errors

For example (Playwright specific javascript code):

test("A citizen can register successfully", async ({ page }) => {
  await page.goto("/users/register")
  await page.waitForSelector("body > .phx-connected") // Ensure that the live-view is connected

  await page.getByLabel("Email").fill("rick@example.org")
  await page.getByRole("button", { name: "Create an account" }).click()

  await expect(page.getByText("Account created successfully")).toBeVisible()
})