Interactivity API: Fix data-wp-on-document flaky test #58008
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
cbravobernal
added
[Type] Bug
This is odd. Locators just give the Playwright instructions on how to find an element. There should be no difference between defining a new locator with the same instructions and reusing an existing one. cc @kevin940726 |
|
Judging from the error reported:
It seems like it's correctly getting the element but the text is still |
It's possible. This is a Preact app. The event listener is attached using a await visibilityButton.click(); // State is updated
await expect(counter).toHaveText("1"); // DOM is updated
// `useEffect` runs here and register the event listener.
await page.keyboard.press("ArrowDown");
await expect(updatedCounter).toHaveText("2"); |
|
Since it's not a real scenario or a real app, I think we can insert an arbitrary timeout in between to wait for the JS to hydrate. WDYT? Something along these lines: await visibilityButton.click();
await expect( counter ).toHaveText( '1' );
+ // Wait for the JS to attach the event listener. (Ref the issue)
+ await page.evaluate( () => new Promise( ( resolve ) => requestAnimationFrame( resolve ) ) );
await page.keyboard.press( 'ArrowDown' );
await expect( counter ).toHaveText( '2' ); |
|
Added in 17227fd to test it out. But, is adding delays a common way to solve flaky tests?
EDIT: Oh, ok, that's not the problem. The problem is that the |
No, but this is a bit of an edge case here unfortunately 😅. Usually this kind of problem only happens with poor hydration performance. If we expect the event listeners to only attach after some user action, then there is going to be a delay in between. If the user is really fast (as fast as Playwright) then it's going to result in poor UX. Given that it's not a real app but only a test case, I think it's fine to add some workaround with comments. In practice, some use advanced tricks to "record" the events before the hydration and "replay" them after the app hydrates (similar to the GA scripts). |
|
I think this is a better solution as it doesn't require an arbitrary delay and relays on Playwright's internal waiting mechanisms: 1c82bb0 @kevin940726 let me know what you think. |
|
Size Change: +2.39 kB (0%) Total Size: 1.7 MB
ℹ️ View Unchanged
|
|
Let's merge this to see if it solves the issue. |
|
The GB bot reopened the same issue, but this time it is for 
@c4rl0sbr4v0 can you apply the same approach to see if that solves the issue? |
luisherranz
removed
the
[Type] Bug
|
Warning: Type of PR label mismatch To merge this PR, it requires exactly 1 label indicating the type of PR. Other labels are optional and not being checked here.
Read more about Type labels in Gutenberg. Don't worry if you don't have the required permissions to add labels; the PR reviewer should be able to help with the task. |
kevin940726
reviewed
Jan 24, 2024
| // Add the element back. | ||
| await visibilityButton.click(); | ||
| await expect( counter ).toHaveText( '1' ); | ||
| await expect( counter ).toHaveText( '1' ); |
There was a problem hiding this comment.
Oh, true!
@c4rl0sbr4v0 can you fix this in the PR that you make to solve this?
There was a problem hiding this comment.
Done in #58181
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What?
Fixes #57983.
Why?
Nobody wants flaky tests.
How?
Defining a new location in playwright should wait for the locator to appear. (Which happens on the previously visibility button click.)