[SPARK-22756] [Build] [SparkR] Run SparkR tests if hive_thriftserver module has code changes

[gatorsmile]

What changes were proposed in this pull request?

The recent PR change in hive_thriftserver caused the test failure in CRAN requirements. To some extends, SparkR module also depends on hive_thriftserver module which could output some log files, so we should run SparkR tests if the hive_thriftserver module has code changes.

How was this patch tested?

N/A

[gatorsmile]

Since SparkR already depends on hive, this PR just adds the dependence on hive_thriftserver

[SparkQA]

Test build #84735 has finished for PR 19944 at commit 1fd2d53.

• This patch fails due to an unknown error code, 255.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #84736 has finished for PR 19944 at commit 6439458.

• This patch fails SparkR unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[gatorsmile]

cc @felixcheung Very weird... I can reproduce it in my local environment. Could you take a look at why SparkR failed?

Caused by: org.apache.spark.SparkException: R computation failed with
 Error : requireNamespace("e1071", quietly = TRUE) is not TRUE
    at org.apache.spark.api.r.RRunner.compute(RRunner.scala:108)
    at org.apache.spark.api.r.BaseRRDD.compute(RRDD.scala:51)
    at org.apache.spark.rdd.RDD.computeOrReadCheckpoint(RDD.scala:324)
    at org.apache.spark.rdd.RDD.iterator(RDD.scala:288)
    at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:87)
    at org.apache.spark.scheduler.Task.run(Task.scala:108)
    at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:345)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
    ... 1 more

[HyukjinKwon]

Hm, @gatorsmile. BTW, do you maybe know how CRAN check fails by the changes in the thrift server? I was just double checking for sure but it sounds orthogonal to me now.

The test failure above seems due to missing package e1071 in your local.

[felixcheung]

Yes this seems unrelated - it’s just saying you need the package e1071 to run the test. Anyone know why SparkR tests are failing all of a sudden? There is nothing in the log file like this one https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84726/console

[felixcheung]

To clarify it looks like SparkR test is failing in other PRs.

[HyukjinKwon]

Yup and it looks passing fine back now (roughly just a couple(?) of hours ago).

Seems the problem was this one(?)

* checking CRAN incoming feasibility ...Error in .check_package_CRAN_incoming(pkgdir) : 
  dims [product 39] do not match the length of object [0]
Execution halted
Loading required package: methods

[HyukjinKwon]

retest this please

[SparkQA]

Test build #84768 has finished for PR 19944 at commit 6439458.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[gatorsmile]

<property>
    <name>hive.server2.logging.operation.enabled</name>
    <value>true</value>
</property>
<property>
  <name>hive.server2.logging.operation.level</name>
  <value>VERBOSE</value>
</property>
<property>
    <name>hive.querylog.location</name>
    <value>/data/logs/hive/${user.name}</value>
</property>

I am afraid that thriftserver PR could break it, because it writes to the log if hive.server2.logging.operation.enabled is set to true.

[gatorsmile]

How about merging this PR first? and let @zouchenjun resubmit the PR?

[gatorsmile]

cc @srowen @liancheng