docs: Adding document and notebooks for ONNXModel #1164
Conversation
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
memoryz
changed the title
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Codecov Report
@@ Coverage Diff @@
## master #1164 +/- ##
==========================================
- Coverage 83.81% 83.75% -0.07%
==========================================
Files 259 259
Lines 12413 12433 +20
Branches 632 630 -2
==========================================
+ Hits 10404 10413 +9
- Misses 2009 2020 +11
Continue to review full report at Codecov.
|
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
| } | ||
| } | ||
|
|
||
| private def writeNestedSeqToStringBuffer(nestedSeq: Seq[_], size: Int): ArrayBuffer[String] = { |
There was a problem hiding this comment.
Could you share a little details on why this stuff is necessary?
There was a problem hiding this comment.
This method writeNestedSeqToStringBuffer, together with writeNestedSeqToBuffer, are used to replace the flattenNested method.
flattenNested flattens a nested Seq to a 1-d array, but is super slow due to repeated memory allocation.
writeNestedSeqToStringBuffer and writeNestedSeqToBuffer both allocates the memory buffer one time, and write the nested Seq to the buffer. I did some local testing, and they appear to be 5-10 times faster than flattenNested.
The reason I need two methods is because there is not a nio.Buffer class for string type, so I had to replace it with a ArrayBuffer[String].
core/src/test/scala/com/microsoft/ml/spark/nbtest/DatabricksUtilities.scala
Show resolved
Hide resolved
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
No description provided.