Push parallelism isn't implemented in Hadoop batch ingestion

[kkrugler]

In HadoopSegmentTarPushJobRunner.run(), this is the code at the very end of the method:

    int pushParallelism = _spec.getPushJobSpec().getPushParallelism();
    if (pushParallelism < 1) {
      pushParallelism = segmentsToPush.size();
    }
    // Push from driver
    try {
      SegmentPushUtils.pushSegments(_spec, outputDirFS, segmentsToPush);
    } catch (RetriableOperationException | AttemptsExceededException e) {
      throw new RuntimeException(e);
    }

So it doesn't actually use pushParallelism, and SegmentPushUtils.pushSegments() does a single-threaded (sequential) push.

[xiangfu0]

Yes, we should fix this to mapper jobs.