SPARK-3470 [CORE] [STREAMING] Add Closeable / close() to Java context objects

[srowen]

... that expose a stop() lifecycle method. This doesn't add AutoCloseable, which is Java 7+ only. But it should be possible to use try-with-resources on a Closeable in Java 7, as long as the close() does not throw a checked exception, and these don't. Q.E.D.

[SparkQA]

QA tests have started for PR 2346 at commit 612c21d.

• This patch merges cleanly.

[witgo]

The relevant PR: #991

[SparkQA]

QA tests have finished for PR 2346 at commit 612c21d.

• This patch passes unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class JavaSparkContext(val sc: SparkContext)
  • class JavaStreamingContext(val ssc: StreamingContext) extends Closeable

[mattf]

since spark targets java 7 & 8, why not just use the correct AutoCloseable?

[srowen]

It doesn't target Java 7: https://github.com/apache/spark/blob/master/pom.xml#L113

[mattf]

It doesn't target Java 7: https://github.com/apache/spark/blob/master/pom.xml#L113

you're right. i hope it becomes java 7+ and we can move to AutoCloseable

[srowen]

AutoCloseable is a superinterface of Closeable, so, this is already true. Implementing Closeable is a stronger condition, strangely.

[mattf]

would like a unit test, but lgtm

[andrewor14]

Yup, LGTM

[rxin]

note two space indent here, but don't worry i will fix it.

[rxin]

@srowen any reason you did not add this to the Scala SparkContext?

[srowen]

@rxin (Yeah wasn't sure how to handle the continuation indent, feel free to change it.) I didn't add it to SparkContext because I figured the purpose of the change was to support Java's try-with-resources, which only works in Java, and I don't expect Java users would regularly be accessing SparkContext instead of JavaSparkContext. It wouldn't do harm to implement Closeable but probably doesn't help anyone.