[SPARK-11194] [SQL] Use MutableURLClassLoader for the classLoader in IsolatedClientLoader.

[yhuai]

https://issues.apache.org/jira/browse/SPARK-11194

[SparkQA]

Test build #43938 has finished for PR 9170 at commit a5d3e0a.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[yhuai]

test this please

[yhuai]

hmm. When we close a session in thrift server, we will close the session state, which internally closes the class loader (the code starts from https://github.com/apache/spark/blob/master/sql/hive-thriftserver/src/main/scala/org/apache/spark/sql/hive/thriftserver/SparkSQLSessionManager.scala#L77 and SessionState.close will call JavaUtils.closeClassLoadersTo(conf.getClassLoader(), parentLoader)). So, the mutable classloader will be closed after we close a jdbc session. Actually, this behavior may cause problems even if we do not use this mutable classloader.

@liancheng For our thrift server, do we need to use HiveSession?

[yhuai]

https://docs.oracle.com/javase/8/docs/technotes/guides/net/ClassLoader.html is an article about closing a classloader. Looks like close is mainly used when we want to hot-reload a jar.

[yhuai]

One hacky way to workaround the problem is that we have a no-op close method for this classloader.

[SparkQA]

Test build #44208 has finished for PR 9170 at commit 65f0a4c.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):\n * case class LongEncoder(fieldName: String = \"value\", ordinal: Int = 0) extends Encoder[Long]\n * case class IntEncoder(fieldName: String = \"value\", ordinal: Int = 0) extends Encoder[Int]\n * case class StringEncoder(\n * |class Tuple$\n * class Tuple2Encoder[T1, T2](e1: Encoder[T1], e2: Encoder[T2]) extends Encoder[(T1, T2)]\n * class Tuple3Encoder[T1, T2, T3](e1: Encoder[T1], e2: Encoder[T2], e3: Encoder[T3]) extends Encoder[(T1, T2, T3)]\n * class Tuple4Encoder[T1, T2, T3, T4](e1: Encoder[T1], e2: Encoder[T2], e3: Encoder[T3], e4: Encoder[T4]) extends Encoder[(T1, T2, T3, T4)]\n * class Tuple5Encoder[T1, T2, T3, T4, T5](e1: Encoder[T1], e2: Encoder[T2], e3: Encoder[T3], e4: Encoder[T4], e5: Encoder[T5]) extends Encoder[(T1, T2, T3, T4, T5)]\n * implicit class AttributeSeq(attrs: Seq[Attribute])\n * case class MapPartitions[T, U](\n * case class AppendColumn[T, U](\n * case class MapGroups[K, T, U](\n * class TypedColumn[T](expr: Expression)(implicit val encoder: Encoder[T]) extends Column(expr)\n * abstract class SQLImplicits\n * case class MapPartitions[T, U](\n * case class AppendColumns[T, U](\n * case class MapGroups[K, T, U](\n * // to its own URLs over the parent class loader (see Executor's createClassLoader method).\n

[davies]

LGTM

[yhuai]

Thanks for reviewing! I am merging it to master.