[YARN][Minor] Avoid hard code port number in YarnShuffleService test

[jerryshao]

Current port number is fixed as default (7337) in test, this will introduce port contention exception, better to change to a random number in unit test.

@squito , seems you're author of this unit test, mind taking a look at this fix? Thanks a lot.

[info] - executor state kept across NM restart *** FAILED *** (597 milliseconds)
[info]   org.apache.hadoop.service.ServiceStateException: java.net.BindException: Address already in use
[info]   at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:59)
[info]   at org.apache.hadoop.service.AbstractService.init(AbstractService.java:172)
[info]   at org.apache.spark.network.yarn.YarnShuffleServiceSuite$$anonfun$1.apply$mcV$sp(YarnShuffleServiceSuite.scala:72)
[info]   at org.apache.spark.network.yarn.YarnShuffleServiceSuite$$anonfun$1.apply(YarnShuffleServiceSuite.scala:70)
[info]   at org.apache.spark.network.yarn.YarnShuffleServiceSuite$$anonfun$1.apply(YarnShuffleServiceSuite.scala:70)
[info]   at org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)
[info]   at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
[info]   at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
[info]   at org.scalatest.Transformer.apply(Transformer.scala:22)
[info]   at org.scalatest.Transformer.apply(Transformer.scala:20)
[info]   at org.scalatest.FunSuiteLike$$anon$1.apply(FunSuiteLike.scala:166)
[info]   at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:42)
...

[SparkQA]

Test build #41741 has finished for PR 8502 at commit bcb93a8.

• This patch fails PySpark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
  • class LogisticRegressionModel @Since("1.3.0") (
  • class SVMModel @Since("1.1.0") (
  • class GaussianMixtureModel @Since("1.3.0") (
  • class KMeansModel @Since("1.1.0") (@Since("1.0.0") val clusterCenters: Array[Vector])
  • class PowerIterationClusteringModel @Since("1.3.0") (
  • class StreamingKMeansModel @Since("1.2.0") (
  • class StreamingKMeans @Since("1.2.0") (
  • class BinaryClassificationMetrics @Since("1.3.0") (
  • class MulticlassMetrics @Since("1.1.0") (predictionAndLabels: RDD[(Double, Double)])
  • class MultilabelMetrics @Since("1.2.0") (predictionAndLabels: RDD[(Array[Double], Array[Double])])
  • class RegressionMetrics @Since("1.2.0") (
  • class ChiSqSelectorModel @Since("1.3.0") (
  • class ChiSqSelector @Since("1.3.0") (
  • class ElementwiseProduct @Since("1.4.0") (
  • class IDF @Since("1.2.0") (@Since("1.2.0") val minDocFreq: Int)
  • class Normalizer @Since("1.1.0") (p: Double) extends VectorTransformer
  • class PCA @Since("1.4.0") (@Since("1.4.0") val k: Int)
  • class StandardScaler @Since("1.1.0") (withMean: Boolean, withStd: Boolean) extends Logging
  • class StandardScalerModel @Since("1.3.0") (
  • class FPGrowthModel[Item: ClassTag] @Since("1.3.0") (
  • class FreqItemset[Item] @Since("1.3.0") (
  • class FreqSequence[Item] @Since("1.5.0") (
  • class PrefixSpanModel[Item] @Since("1.5.0") (
  • class DenseMatrix @Since("1.3.0") (
  • class SparseMatrix @Since("1.3.0") (
  • class DenseVector @Since("1.0.0") (
  • class SparseVector @Since("1.0.0") (
  • class BlockMatrix @Since("1.3.0") (
  • class CoordinateMatrix @Since("1.0.0") (
  • class IndexedRowMatrix @Since("1.0.0") (
  • class RowMatrix @Since("1.0.0") (
  • class PoissonGenerator @Since("1.1.0") (
  • class ExponentialGenerator @Since("1.3.0") (
  • class GammaGenerator @Since("1.3.0") (
  • class LogNormalGenerator @Since("1.3.0") (
  • case class Rating @Since("0.8.0") (
  • class MatrixFactorizationModel @Since("0.8.0") (
  • abstract class GeneralizedLinearModel @Since("1.0.0") (
  • class IsotonicRegressionModel @Since("1.3.0") (
  • case class LabeledPoint @Since("1.0.0") (
  • class LassoModel @Since("1.1.0") (
  • class LinearRegressionModel @Since("1.1.0") (
  • class RidgeRegressionModel @Since("1.1.0") (
  • class MultivariateGaussian @Since("1.3.0") (
  • case class BoostingStrategy @Since("1.4.0") (
  • class Strategy @Since("1.3.0") (
  • class DecisionTreeModel @Since("1.0.0") (
  • class Node @Since("1.2.0") (
  • class Predict @Since("1.2.0") (
  • class RandomForestModel @Since("1.2.0") (
  • class GradientBoostedTreesModel @Since("1.2.0") (
  • public static class AppExecId
  • public static class StoreVersion
  • abstract class SetOperation(left: LogicalPlan, right: LogicalPlan) extends BinaryNode
  • case class Union(left: LogicalPlan, right: LogicalPlan) extends SetOperation(left, right)
  • case class Intersect(left: LogicalPlan, right: LogicalPlan) extends SetOperation(left, right)
  • case class Except(left: LogicalPlan, right: LogicalPlan) extends SetOperation(left, right)

[squito]

thanks @jerryshao , good catch! I suppose I never thought about it since the unit test doesn't actually use the server, but the service will start it in any case.

merging to master