[BEAM-8977] Resolve test flakiness #10404

KevinGG · 2019-12-17T19:52:14Z

Removed test logic depending on execution of asynchronous tasks since
there is no control of them in a testing environment.
Replaced the dynamic plotting tests with tests directly/indirectly
invoking underlying logic of the asynchronous task.

Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

Choose reviewer(s) and mention them in a comment (R: @username).
Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

Post-Commit Tests Status (on master branch)

Lang	SDK	Apex	Dataflow	Gearpump	Samza	Spark
Go		---	---	---	---
Java
Python		---		---	---
XLang	---	---	---	---	---	---

Pre-Commit Tests Status (on master branch)

---	Java	Python	Go	Website
Non-portable
Portable	---		---	---

See .test-infra/jenkins/README for trigger phrase, status and link of all Jenkins jobs.

KevinGG · 2019-12-17T19:54:17Z

R: @tvalentyn
PTAL, thanks!

tvalentyn · 2019-12-17T21:43:52Z

sdks/python/apache_beam/runners/interactive/display/pcoll_visualization_test.py

  def test_dynamic_plotting_update_same_display(self,
-                                                mocked_display_facets):
+                                                mocked_display_dataframe,


Are we verifying that the same display is being updated? If so consider s/update/updates in test_dynamic_plotting_update_same_display, if not consider a name that communicates the expected behavior.

Yes, we are verifying that the same display is being updated.
Renaming the test to test_dynamic_plotting_updates_same_display.

SG - once you push the commit and tests pass we can merge this.

tvalentyn · 2019-12-18T18:05:27Z

Is the precommit failure related to the PR? If not could you please help triage it (file a bug and see if there is an obvious owner).

KevinGG · 2019-12-18T18:25:32Z

Is the precommit failure related to the PR? If not could you please help triage it (file a bug and see if there is an obvious owner).

Filed BEAM-8991.

And:
Run Python PreCommit
to see if it's just flaky.

KevinGG · 2019-12-18T18:25:54Z

Run Python PreCommit

tvalentyn · 2019-12-18T22:48:34Z

Looks like you hit BEAM-8974

tvalentyn · 2019-12-18T22:48:43Z

Run Python PreCommit

KevinGG · 2019-12-18T22:51:39Z

Looks like you hit BEAM-8974

Thanks for the heads up!

KevinGG · 2019-12-19T00:04:35Z

Yes, it fails at the error described in BEAM-8974.
I'll mark BEAM-8991 as a duplicate.

KevinGG · 2019-12-19T18:44:43Z

I'll do a rebase against the upstream/master once the issue is resolved and ping you whenever it's ready!
Thank you very much, Valentyn! Merry Christmas!

tvalentyn · 2019-12-19T19:33:29Z

Thanks! Since it's a flake, we can try running the tests again, and maybe it will not fail.
Happy holidays likewise :)

tvalentyn · 2019-12-19T19:33:32Z

Run Python PreCommit

tvalentyn · 2019-12-19T19:33:47Z

Run Python PreCommit

KevinGG · 2019-12-19T22:48:19Z

Run Python PreCommit

1. Removed test logic depending on execution of asynchronous tasks since there is no control of them in a testing environment. 2. Replaced the dynamic plotting tests with tests directly/indirectly invoking underlying logic of the asynchronous task.

KevinGG · 2019-12-20T18:56:07Z

Resolved merge conflicts and force pushed.

tvalentyn · 2019-12-20T20:50:58Z

Thanks, @KevinGG . Looks like something is not right with precommit checks since they have not triggered yet...

tvalentyn · 2019-12-20T20:51:07Z

Run Python Precommit

KevinGG · 2019-12-20T20:57:26Z

retest this please

KevinGG · 2019-12-20T21:07:20Z

Thanks, @KevinGG . Looks like something is not right with precommit checks since they have not triggered yet...

Thanks! Yeah, probably Jenkins is busy. If it's not executed by the EOD, I'll just close this PR and create a new one to see if it helps.

Tests triggered as soon as this comment is added, hah.

KevinGG · 2019-12-20T22:39:18Z

Run Python PreCommit

KevinGG · 2019-12-23T18:52:05Z

Run Python PreCommit

tvalentyn reviewed Dec 17, 2019

View reviewed changes

KevinGG force-pushed the BEAM-8977 branch from 1535ea9 to 5cdf423 Compare December 18, 2019 00:16

tvalentyn closed this Dec 19, 2019

tvalentyn reopened this Dec 19, 2019

[BEAM-8977] Resolve test flakiness

2375217

1. Removed test logic depending on execution of asynchronous tasks since there is no control of them in a testing environment. 2. Replaced the dynamic plotting tests with tests directly/indirectly invoking underlying logic of the asynchronous task.

KevinGG force-pushed the BEAM-8977 branch from 5cdf423 to 2375217 Compare December 20, 2019 18:55

tvalentyn merged commit 0028b79 into apache:master Jan 3, 2020

KevinGG deleted the BEAM-8977 branch January 6, 2020 18:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[BEAM-8977] Resolve test flakiness #10404

[BEAM-8977] Resolve test flakiness #10404

KevinGG commented Dec 17, 2019 •

edited

KevinGG commented Dec 17, 2019

tvalentyn Dec 17, 2019 •

edited

KevinGG Dec 17, 2019

tvalentyn Dec 17, 2019

KevinGG Dec 17, 2019

tvalentyn commented Dec 18, 2019

KevinGG commented Dec 18, 2019

KevinGG commented Dec 18, 2019

tvalentyn commented Dec 18, 2019

tvalentyn commented Dec 18, 2019

KevinGG commented Dec 18, 2019 •

edited

KevinGG commented Dec 19, 2019

KevinGG commented Dec 19, 2019

tvalentyn commented Dec 19, 2019

tvalentyn commented Dec 19, 2019

tvalentyn commented Dec 19, 2019

KevinGG commented Dec 19, 2019

KevinGG commented Dec 20, 2019

tvalentyn commented Dec 20, 2019

tvalentyn commented Dec 20, 2019

KevinGG commented Dec 20, 2019

KevinGG commented Dec 20, 2019 •

edited

KevinGG commented Dec 20, 2019

KevinGG commented Dec 23, 2019

[BEAM-8977] Resolve test flakiness #10404

[BEAM-8977] Resolve test flakiness #10404

Conversation

KevinGG commented Dec 17, 2019 • edited

Post-Commit Tests Status (on master branch)

Pre-Commit Tests Status (on master branch)

KevinGG commented Dec 17, 2019

tvalentyn Dec 17, 2019 • edited

Choose a reason for hiding this comment

KevinGG Dec 17, 2019

Choose a reason for hiding this comment

tvalentyn Dec 17, 2019

Choose a reason for hiding this comment

KevinGG Dec 17, 2019

Choose a reason for hiding this comment

tvalentyn commented Dec 18, 2019

KevinGG commented Dec 18, 2019

KevinGG commented Dec 18, 2019

tvalentyn commented Dec 18, 2019

tvalentyn commented Dec 18, 2019

KevinGG commented Dec 18, 2019 • edited

KevinGG commented Dec 19, 2019

KevinGG commented Dec 19, 2019

tvalentyn commented Dec 19, 2019

tvalentyn commented Dec 19, 2019

tvalentyn commented Dec 19, 2019

KevinGG commented Dec 19, 2019

KevinGG commented Dec 20, 2019

tvalentyn commented Dec 20, 2019

tvalentyn commented Dec 20, 2019

KevinGG commented Dec 20, 2019

KevinGG commented Dec 20, 2019 • edited

KevinGG commented Dec 20, 2019

KevinGG commented Dec 23, 2019

KevinGG commented Dec 17, 2019 •

edited

tvalentyn Dec 17, 2019 •

edited

KevinGG commented Dec 18, 2019 •

edited

KevinGG commented Dec 20, 2019 •

edited