[SPARK-9261] [streaming] Avoid calling APIs that expose shaded classes.

[vanzin]

Doing this may cause weird errors when tests are run on maven, depending
on the flags used. Instead, expose the needed functionality through methods
that do not expose shaded classes.

[vanzin]

/cc @tdas

[SparkQA]

Test build #38113 has finished for PR 7601 at commit 4f64a16.

• This patch fails PySpark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class CreateArray(children: Seq[Expression]) extends Expression
  • case class CreateStruct(children: Seq[Expression]) extends Expression
  • case class CreateNamedStruct(children: Seq[Expression]) extends Expression

[vanzin]

Jenkins retest this please.

[tdas]

cc @zsxwing you have a better idea of this code.

[SparkQA]

Test build #38147 has finished for PR 7601 at commit 4f64a16.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #69 timed out for PR 7601 at commit 4f64a16 after a configured wait of 175m.

[zsxwing]

retest this please.

[zsxwing]

LGTM

[SparkQA]

Test build #83 has finished for PR 7601 at commit 4f64a16.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #38245 has finished for PR 7601 at commit 4f64a16.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[vanzin]

Lots of test flakiness lately.

[vanzin]

retest this please

[SparkQA]

Test build #38278 has finished for PR 7601 at commit 4f64a16.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #84 has finished for PR 7601 at commit 4f64a16.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[vanzin]

Merged to master.