MAHOUT-1837: Fixed dense bug in drm/package.blockify() #252
Conversation
|
Looks good to me |
| } else { | ||
| new SparseRowMatrix(vectors.length, blockncol, vectors, true, false) |
There was a problem hiding this comment.
Should the default here be a SparseRowMatrix of Random Access Sparse Vectors? Seems so. I.e.
This line should probably read:
new SparseRowMatrix(vectors.length, blockncol, vectors, true, true)rather than:
new SparseRowMatrix(vectors.length, blockncol, vectors, true, false)as is, correct?
There was a problem hiding this comment.
No, i believe sequential should stay as it will be more natural (ordered)
when converted to CSR which hopefuly will be our most common modus operandi
for exponential algorithms.
in fact, random access rarely makes sense at all for block-wise algorithms.
On Fri, Aug 26, 2016 at 8:03 PM, Andrew Palumbo notifications@github.com
wrote:
In spark/src/main/scala/org/apache/mahout/sparkbindings/drm/package.scala
#252 (comment):} else {
new SparseRowMatrix(vectors.length, blockncol, vectors, true, false)Should the default here be a SparseRowMatrix of Random Access Sparse
Vectors? Seems so. I.e.
This line should probably read:new SparseRowMatrix(vectors.length, blockncol, vectors, true, true)
rather than:
new SparseRowMatrix(vectors.length, blockncol, vectors, true, false)
as is, correct?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/apache/mahout/pull/252/files/5d2f5cc9746f968d0c776f869070dd9a439de9f1#r76509011,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAf7_zKcdpkHa0zDQnNvOFMuXQHNW897ks5qj6j4gaJpZM4JtpfO
.
|
Thanks again @AddictedCS for the bug report. |
|
Thanks @andrewpalumbo for a quick fix. |
Create a
SparseRowMatrixby default in order to keepOOMerrors from occurring inblockify()per conversation in: 727e5be#commitcomment-18771603. rundensityAnalysis()on that and convert to dense if requirements are met.