[LIVY-749] Remove explicitly adding datanucleus jars

[shanyu]

What changes were proposed in this pull request?

https://issues.apache.org/jira/browse/LIVY-749

If we start any Livy session with hive support (livy.repl.enable-hive-context=true), we see that 3 datanucleus jars are uploaded to HDFS and downloaded to drivers/executors:

Uploading resource file:/opt/spark/jars/datanucleus-api-jdo-3.2.6.jar -> hdfs://namenode/user/test1/.sparkStaging/application_1581024490249_0002/datanucleus-api-jdo-3.2.6.jar
...

These 3 datanucleus jars are not needed bacause they are already included in Spark 2.x jars folder.

The reason is because in InteractiveSession.scala, method mergeHiveSiteAndHiveDeps(), we merged datanucleus jars to spark.jars list with method datanucleusJars(). We should remove datanucleusJars() function.

How was this patch tested?

existing unit/integration tests, and manual tests.

Please review https://livy.incubator.apache.org/community/ before opening a pull request.

[codecov-io]

Codecov Report

Merging #280 into master will decrease coverage by 0.13%.
The diff coverage is 59.86%.

@@             Coverage Diff             @@
##             master    #280      +/-   ##
===========================================
- Coverage     68.54%   68.4%   -0.14%     
- Complexity      906     964      +58     
===========================================
  Files           100     104       +4     
  Lines          5674    5934     +260     
  Branches        854     895      +41     
===========================================
+ Hits           3889    4059     +170     
- Misses         1228    1295      +67     
- Partials        557     580      +23

Impacted Files	Coverage Δ	Complexity Δ
...ain/java/org/apache/livy/rsc/driver/Statement.java	0% <0%> (ø)	0 <0> (ø)	⬇️
...in/scala/org/apache/livy/server/ui/UIServlet.scala	4.49% <0%> (ø)	1 <0> (ø)	⬇️
...a/org/apache/livy/utils/LivyUncaughtException.java	0% <0%> (ø)	0 <0> (?)
...rver/src/main/scala/org/apache/livy/LivyConf.scala	96.13% <100%> (+0.28%)	21 <0> (ø)	⬇️
...che/livy/server/recovery/ZooKeeperStateStore.scala	100% <100%> (+20.93%)	8 <8> (-9)	⬇️
rsc/src/main/java/org/apache/livy/rsc/Utils.java	81.57% <100%> (-1.35%)	14 <1> (-1)
.../src/main/scala/org/apache/livy/repl/Session.scala	67.69% <100%> (+0.5%)	37 <0> (ø)	⬇️
rsc/src/main/java/org/apache/livy/rsc/RSCConf.java	87.85% <100%> (+2%)	8 <0> (+1)	⬆️
...main/scala/org/apache/livy/server/LivyServer.scala	33.03% <13.63%> (-2.44%)	11 <0> (ø)
...c/src/main/java/org/apache/livy/rsc/RSCClient.java	73.49% <16.66%> (-1.67%)	20 <1> (ø)
... and 32 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d6d8261...d147fc9. Read the comment docs.

[codecov-commenter]

Codecov Report

Merging #280 into master will increase coverage by 0.19%.
The diff coverage is n/a.

@@             Coverage Diff              @@
##             master     #280      +/-   ##
============================================
+ Coverage     68.20%   68.40%   +0.19%     
+ Complexity      965      964       -1     
============================================
  Files           104      104              
  Lines          5951     5934      -17     
  Branches        900      895       -5     
============================================
  Hits           4059     4059              
+ Misses         1314     1295      -19     
- Partials        578      580       +2     

Impacted Files	Coverage Δ	Complexity Δ
...e/livy/server/interactive/InteractiveSession.scala	73.84% <ø> (+3.67%)	51.00 <0.00> (ø)
...cala/org/apache/livy/scalaapi/ScalaJobHandle.scala	52.94% <0.00%> (-2.95%)	7.00% <0.00%> (ø%)
...ain/java/org/apache/livy/rsc/driver/RSCDriver.java	80.00% <0.00%> (-0.84%)	45.00% <0.00%> (-1.00%)
...ain/scala/org/apache/livy/utils/SparkYarnApp.scala	73.75% <0.00%> (+1.87%)	40.00% <0.00%> (ø%)

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3a26856...d147fc9. Read the comment docs.