Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-7558] Demarcate tests in unit-tests.log (1.4) #6598

Closed
wants to merge 10 commits into from
Closed

[SPARK-7558] Demarcate tests in unit-tests.log (1.4) #6598

wants to merge 10 commits into from

Conversation

andrewor14
Copy link
Contributor

This includes the following commits:

original: 9eb222c
hotfix1: 8c99793
hotfix2: a4f2412
scalastyle check: 609c492

Original patch #6441
Branch-1.3 patch #6602

Andrew Or added 9 commits June 2, 2015 15:20
Introduce base abstract class for all test suites
75d361f
Add core tests as dependencies in all modules
8bc355d 
This is such that test suites in all modules can extend
SparkFunSuite.
Fix manual things that cannot be covered through automation
a932e8d
Fix log message
634a777
Fix tests
b4d93de 
In core/test, it turns out that selenium is bringing in guava 15,
but the rest of Spark we use guava 14, which is evicted. Since
we made catalyst/test depend on core/test, and hive/test depends
on catalyst/test, hive tests are getting guava 15, which results
in java.lang.IllegalAccessError because of a dependency conflict.

This is resolved by excluding guava 15 from selenium in core/test.
Fix tests?
eaa520e
Make all test suites extend SparkFunSuite instead of FunSuite
3d9bf04 
This is a big commit. In addition to just doing a search and
replace there is actually more work to do in ensuring that the
correct imports are introduced and the unused ones removed.

This step is largely automated. :)

Note that we exclude everything in the flume-sink module
because the dependencies do not work out there. A larger
comment will be added later to document why this is so.
Various whitespace changes (minor)
46d4361
[SPARK-7558] Guard against direct uses of FunSuite / FunSuiteLike
e217b78 
This is a follow-up patch to apache#6441.

Author: Andrew Or <andrew@databricks.com>

Closes apache#6510 from andrewor14/extends-funsuite-check and squashes the following commits:

6618b46 [Andrew Or] Exempt SparkSinkSuite from the FunSuite check
99d02ac [Andrew Or] Merge branch 'master' of github.com:apache/spark into extends-funsuite-check
48874dd [Andrew Or] Guard against direct uses of FunSuite / FunSuiteLike
@andrewor14 andrewor14 mentioned this pull request Jun 3, 2015
Closed
@SparkQA
Copy link

SparkQA commented Jun 3, 2015

Test build #34035 has finished for PR 6598 at commit e217b78.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@andrewor14
Copy link
Contributor Author

Unable to reproduce locally. Jenkins retest this please

@andrewor14
Copy link
Contributor Author

@liancheng In CliSuite, the test Commands using SerDe provided in --jars is failing, but I was not able to reproduce this locally. Do you know what's wrong?

@SparkQA
Copy link

SparkQA commented Jun 3, 2015

Test build #34054 timed out for PR 6598 at commit e217b78 after a configured wait of 150m.

@andrewor14
Copy link
Contributor Author

All scala tests actually passed. retest this please

@SparkQA
Copy link

SparkQA commented Jun 3, 2015

Test build #34073 timed out for PR 6598 at commit e217b78 after a configured wait of 150m.

@SparkQA
Copy link

SparkQA commented Jun 3, 2015

Test build #34074 timed out for PR 6598 at commit e217b78 after a configured wait of 150m.

@andrewor14
Copy link
Contributor Author

python streaming test is timing out. Jenkins retest this please

asfgit pushed a commit that referenced this pull request Jun 3, 2015
[SPARK-7558] Demarcate tests in unit-tests.log (1.3)
e5747ee 
This includes the following commits:

original: 9eb222c
hotfix1: 8c99793
hotfix2: a4f2412
scalastyle check: 609c492

---
Original patch #6441
Branch-1.4 patch #6598

Author: Andrew Or <andrew@databricks.com>

Closes #6602 from andrewor14/demarcate-tests-1.3 and squashes the following commits:

a75ff8f [Andrew Or] Fix hive-thrift server log4j problem
f782edd [Andrew Or] [SPARK-7558] Guard against direct uses of FunSuite / FunSuiteLike
2b7a4f4 [Andrew Or] Fix tests?
fec05c2 [Andrew Or] Fix tests
5342d50 [Andrew Or] Various whitespace changes (minor)
9af2756 [Andrew Or] Make all test suites extend SparkFunSuite instead of FunSuite
192a47c [Andrew Or] Fix log message
95ff5eb [Andrew Or] Add core tests as dependencies in all modules
8dffa0e [Andrew Or] Introduce base abstract class for all test suites
@SparkQA
Copy link

SparkQA commented Jun 3, 2015

Test build #34091 timed out for PR 6598 at commit e217b78 after a configured wait of 150m.

@andrewor14
Copy link
Contributor Author

Alright, one last time... retest this please

Merge branch 'branch-1.4' of github.com:apache/spark into demarcate-t…
4c3c566 
…ests-1.4

Conflicts:
	core/src/test/scala/org/apache/spark/deploy/worker/ui/LogPageSuite.scala
	core/src/test/scala/org/apache/spark/scheduler/SparkListenerWithClusterSuite.scala
@SparkQA
Copy link

SparkQA commented Jun 3, 2015

Test build #34108 timed out for PR 6598 at commit e217b78 after a configured wait of 150m.

@andrewor14
Copy link
Contributor Author

Tests keep timing out. Every time it times out at the very last stage, when python 3 tests are running. :(

@andrewor14
Copy link
Contributor Author

Alright, I just bumped up the test timeout. Let's retest this please Jenkins.

@SparkQA
Copy link

SparkQA commented Jun 4, 2015

Test build #34128 timed out for PR 6598 at commit 4c3c566 after a configured wait of 150m.

@andrewor14
Copy link
Contributor Author

retest this please?

@andrewor14
Copy link
Contributor Author

Oh wait it was a different set of tests that timed out

@SparkQA
Copy link

SparkQA commented Jun 4, 2015

Test build #34144 has finished for PR 6598 at commit 4c3c566.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@andrewor14
Copy link
Contributor Author

Yesss finally I'm merging into 1.4.

asfgit pushed a commit that referenced this pull request Jun 4, 2015
[SPARK-7558] Demarcate tests in unit-tests.log (1.4)
bfe74b3 
This includes the following commits:

original: 9eb222c
hotfix1: 8c99793
hotfix2: a4f2412
scalastyle check: 609c492

---
Original patch #6441
Branch-1.3 patch #6602

Author: Andrew Or <andrew@databricks.com>

Closes #6598 from andrewor14/demarcate-tests-1.4 and squashes the following commits:

4c3c566 [Andrew Or] Merge branch 'branch-1.4' of github.com:apache/spark into demarcate-tests-1.4
e217b78 [Andrew Or] [SPARK-7558] Guard against direct uses of FunSuite / FunSuiteLike
46d4361 [Andrew Or] Various whitespace changes (minor)
3d9bf04 [Andrew Or] Make all test suites extend SparkFunSuite instead of FunSuite
eaa520e [Andrew Or] Fix tests?
b4d93de [Andrew Or] Fix tests
634a777 [Andrew Or] Fix log message
a932e8d [Andrew Or] Fix manual things that cannot be covered through automation
8bc355d [Andrew Or] Add core tests as dependencies in all modules
75d361f [Andrew Or] Introduce base abstract class for all test suites
@SparkQA
Copy link

SparkQA commented Jun 4, 2015

Test build #34147 has finished for PR 6598 at commit 4c3c566.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@andrewor14 andrewor14 closed this Jun 4, 2015
@andrewor14 andrewor14 deleted the demarcate-tests-1.4 branch June 4, 2015 05:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
2 participants
@andrewor14 @SparkQA