Skip to content

[SPARK-9649] Fix flaky test MasterSuite again - disable REST #8084

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
andrewor14 wants to merge 1 commit into from
Closed

[SPARK-9649] Fix flaky test MasterSuite again - disable REST #8084

andrewor14 wants to merge 1 commit into from

Conversation

@andrewor14
Copy link
Contributor

@andrewor14 andrewor14 commented Aug 10, 2015

The REST server is not actually used in most tests and so we can disable it. It is a source of flakiness because it tries to bind to a specific port in vain. There was also some code that avoided the shuffle service in tests. This is actually not necessary because the shuffle service is already off by default.

@JoshRosen
Copy link
Contributor

JoshRosen commented Aug 10, 2015

Are there any tests where we need to explicitly re-enable this configuration or is this feature completely untested?

@andrewor14
Copy link
Contributor Author

andrewor14 commented Aug 10, 2015

It is tested in StandaloneRestSubmissionSuite but is mocked out there and doesn't go through the standalone Master.

@SparkQA
Copy link

SparkQA commented Aug 11, 2015

Test build #40343 timed out for PR 8084 at commit f86f2be after a configured wait of 175m.

@JoshRosen
Copy link
Contributor

JoshRosen commented Aug 11, 2015

Jenkins, retest this please.

@SparkQA
Copy link

SparkQA commented Aug 11, 2015

Test build #40411 timed out for PR 8084 at commit f86f2be after a configured wait of 175m.

@andrewor14
Copy link
Contributor Author

andrewor14 commented Aug 11, 2015

retest this please

@andrewor14
Copy link
Contributor Author

andrewor14 commented Aug 11, 2015

retest this please?

@SparkQA
Copy link

SparkQA commented Aug 11, 2015

Test build #40475 timed out for PR 8084 at commit f86f2be after a configured wait of 175m.

Do not enable REST server in tests
026f628 
There was some code that also avoided the shuffle service in tests.
This is actually not necessary because the shuffle service is
already off by default.
@andrewor14 andrewor14 force-pushed the fix-master-suite-again branch from f86f2be to 026f628 Compare August 11, 2015 21:27
@andrewor14
Copy link
Contributor Author

andrewor14 commented Aug 11, 2015

retest this please

@SparkQA
Copy link

SparkQA commented Aug 12, 2015

Test build #1456 has finished for PR 8084 at commit 026f628.

  • This patch fails Spark unit tests.
  • This patch does not merge cleanly.
  • This patch adds no public classes.

@andrewor14
Copy link
Contributor Author

andrewor14 commented Aug 12, 2015

retest this please

@SparkQA
Copy link

SparkQA commented Aug 12, 2015

Test build #40523 timed out for PR 8084 at commit 026f628 after a configured wait of 175m.

@JoshRosen
Copy link
Contributor

JoshRosen commented Aug 12, 2015

Jenkins, retest this please.

@SparkQA
Copy link

SparkQA commented Aug 12, 2015

Test build #1465 has finished for PR 8084 at commit 026f628.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@JoshRosen
Copy link
Contributor

JoshRosen commented Aug 12, 2015

Jenkins, retest this please.

@JoshRosen
Copy link
Contributor

JoshRosen commented Aug 12, 2015

Wow, this has been the unluckiest PR when it comes to flakiness :(

@SparkQA
Copy link

SparkQA commented Aug 12, 2015

Test build #1466 has finished for PR 8084 at commit 026f628.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@JoshRosen
Copy link
Contributor

JoshRosen commented Aug 12, 2015

LGTM, so I'm going to merge this to master and branch-1.5.

asfgit pushed a commit that referenced this pull request Aug 12, 2015
@JoshRosen
[SPARK-9649] Fix flaky test MasterSuite again - disable REST
0119edf 
The REST server is not actually used in most tests and so we can disable it. It is a source of flakiness because it tries to bind to a specific port in vain. There was also some code that avoided the shuffle service in tests. This is actually not necessary because the shuffle service is already off by default.

Author: Andrew Or <andrew@databricks.com>

Closes #8084 from andrewor14/fix-master-suite-again.

(cherry picked from commit ca8f70e)
Signed-off-by: Josh Rosen <joshrosen@databricks.com>
@asfgit asfgit closed this in ca8f70e Aug 12, 2015
@andrewor14 andrewor14 deleted the fix-master-suite-again branch August 12, 2015 04:36
@SparkQA
Copy link

SparkQA commented Aug 12, 2015

Test build #40575 has finished for PR 8084 at commit 026f628.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Aug 12, 2015

Test build #40580 has finished for PR 8084 at commit 026f628.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@JoshRosen
Copy link
Contributor

JoshRosen commented Aug 12, 2015

@andrewor14 do you think that this is also a candidate for backport to 1.4.x? I think that I've seen similar MasterSuite flakiness there as well.

@andrewor14
Copy link
Contributor Author

andrewor14 commented Aug 12, 2015

Yeah, I think it doesn't hurt.

CodingCat pushed a commit to CodingCat/spark that referenced this pull request Aug 17, 2015
@CodingCat
[SPARK-9649] Fix flaky test MasterSuite again - disable REST
7bbd4d5 
The REST server is not actually used in most tests and so we can disable it. It is a source of flakiness because it tries to bind to a specific port in vain. There was also some code that avoided the shuffle service in tests. This is actually not necessary because the shuffle service is already off by default.

Author: Andrew Or <andrew@databricks.com>

Closes apache#8084 from andrewor14/fix-master-suite-again.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@andrewor14 @JoshRosen @SparkQA