[SPARK-9990] [SQL] Local hash join follow-ups

[andrewor14]

1. Hide LocalNodeIterator behind the LocalNode#asIterator method
2. Add tests for this

[andrewor14]

@zsxwing

[SparkQA]

Test build #42294 has finished for PR 8708 at commit fed4808.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • case class ExecutorLostFailure(execId: String, isNormalExit: Boolean = false)
  • class ExecutorLossReason(val message: String) extends Serializable
  • case class ExecutorExited(exitCode: Int, isNormalExit: Boolean, reason: String)
  • case class RemoveExecutor(executorId: String, reason: ExecutorLossReason)
  • case class GetExecutorLossReason(executorId: String) extends CoarseGrainedClusterMessage
  • case class ConvertToSafeNode(conf: SQLConf, child: LocalNode) extends UnaryLocalNode(conf)
  • case class ConvertToUnsafeNode(conf: SQLConf, child: LocalNode) extends UnaryLocalNode(conf)
  • case class FilterNode(conf: SQLConf, condition: Expression, child: LocalNode)
  • case class HashJoinNode(
  • case class LimitNode(conf: SQLConf, limit: Int, child: LocalNode) extends UnaryLocalNode(conf)
  • abstract class LocalNode(conf: SQLConf) extends TreeNode[LocalNode] with Logging
  • abstract class LeafLocalNode(conf: SQLConf) extends LocalNode(conf)
  • abstract class UnaryLocalNode(conf: SQLConf) extends LocalNode(conf)
  • abstract class BinaryLocalNode(conf: SQLConf) extends LocalNode(conf)
  • case class ProjectNode(conf: SQLConf, projectList: Seq[NamedExpression], child: LocalNode)
  • case class SeqScanNode(conf: SQLConf, output: Seq[Attribute], data: Seq[InternalRow])
  • case class UnionNode(conf: SQLConf, children: Seq[LocalNode]) extends LocalNode(conf)

[rxin]

Is this on the critical path? If yes - it'd actually be faster to expose the LocalNodeIterator type since JIT can inline it.

[andrewor14]

this is called when a HashJoinNode is opened. Will it make much of a difference?

[zsxwing]

LGTM. Thanks @andrewor14

[andrewor14]

Merged into master.