[SPARK-16566][MLLib] sort sparseVector's indices before doing multiplication

[wilson-lauw]

What changes were proposed in this pull request?

https://issues.apache.org/jira/browse/SPARK-16566

sort sparseVector's indices before doing multiplication to make sure the result returned correctly

How was this patch tested?

manual and existing tests

[AmplabJenkins]

Can one of the admins verify this patch?

[hhbyyh]

IMO, order of the indices in SparseVector potentially affects many other BLAS operations. Perhaps we should have it in a more general/reusable pattern if we do want to impose the validation.

Yet the most efficient way as I see is still to perform the check during construction, maybe something like an optional validate parameter.

[srowen]

Many small style issues here. I think this is best as

private def isSorted(array: Array[Int]): Boolean = {
  var index = 1
  while (index < array.length) {
    if (array(index - 1) > array(index)) {
      return false
    }
    index += 1
  }
  true
}

[wilson-lauw]

Thanks for the suggestion, will fix it.

[wilson-lauw]

@hhbyyh I just checked, the implementation of def dot(x: SparseVector, y: SparseVector) also need to be modified.

[wilson-lauw]

Modifications also needed on module mllib-local. Once the changes is okay, I will also apply the changes there.

[jkbradley]

SparseVector indices are assumed to be sorted already. We should not re-sort them during operations. The deeper issue which needs to be fixed is SPARK-14707
I commented in the JIRA as well.
Could you please close this issue?
Thanks!

[wilson-lauw]

Closing this.