Skip to content

Commit

Permalink
Added Distributed Linear Algebra Suite
Browse files Browse the repository at this point in the history
Added the following distributed primitives backed by spark RDDs:

1. Distributed Matrix
2. Distributed Row & Column Vector
3. Reference implementations of matrix vector products, sums, inner &
outer products in terms of elementary spark RDD operations.
  • Loading branch information
mandar2812 committed Sep 30, 2016
1 parent b27bd47 commit 3c98f2d
Show file tree
Hide file tree
Showing 65 changed files with 663 additions and 170 deletions.
3 changes: 2 additions & 1 deletion build.sbt
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,8 @@ lazy val core = (project in file("dynaml-core")).settings(baseSettings)
.dependsOn(pipes)
.settings(
name := "dynaml-core",
version := mainVersion
version := mainVersion,
parallelExecution in Test := false
)

lazy val examples = (project in file("dynaml-examples"))
Expand Down
2 changes: 1 addition & 1 deletion docs/api_docs/dynaml-core/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@
<ol class="templates"><li title="io.github.mandar2812.dynaml.DynaMLPipe"><a class="tplshow" href="io/github/mandar2812/dynaml/DynaMLPipe$.html" target="template"><span class="object">(object)</span><div class="placeholder"></div><span class="tplLink">DynaMLPipe</span></a></li></ol>
<ol class="packages"> <li class="pack" title="io.github.mandar2812.dynaml.analysis">
<a class="tplshow" href="io/github/mandar2812/dynaml/analysis/package.html" target="template">io.github.mandar2812.dynaml.analysis</a>
<ol class="templates"><li title="io.github.mandar2812.dynaml.analysis.KernelMatrix"><div class="placeholder"></div><a class="tplshow" href="io/github/mandar2812/dynaml/analysis/KernelMatrix.html" target="template"><span class="trait">(trait)</span><span class="tplLink">KernelMatrix</span></a></li><li title="io.github.mandar2812.dynaml.analysis.SVMKernelMatrix"><div class="placeholder"></div><a class="tplshow" href="io/github/mandar2812/dynaml/analysis/SVMKernelMatrix.html" target="template"><span class="class">(class)</span><span class="tplLink">SVMKernelMatrix</span></a></li><li title="io.github.mandar2812.dynaml.analysis.VectorField"><a class="tplshow" href="io/github/mandar2812/dynaml/analysis/VectorField$.html" target="template"><span class="object">(object)</span></a><a class="tplshow" href="io/github/mandar2812/dynaml/analysis/VectorField.html" target="template"><span class="class">(class)</span><span class="tplLink">VectorField</span></a></li></ol>
<ol class="templates"><li title="io.github.mandar2812.dynaml.algebra.KernelMatrix"><div class="placeholder"></div><a class="tplshow" href="io/github/mandar2812/dynaml/analysis/KernelMatrix.html" target="template"><span class="trait">(trait)</span><span class="tplLink">KernelMatrix</span></a></li><li title="io.github.mandar2812.dynaml.algebra.SVMKernelMatrix"><div class="placeholder"></div><a class="tplshow" href="io/github/mandar2812/dynaml/analysis/SVMKernelMatrix.html" target="template"><span class="class">(class)</span><span class="tplLink">SVMKernelMatrix</span></a></li><li title="io.github.mandar2812.dynaml.analysis.VectorField"><a class="tplshow" href="io/github/mandar2812/dynaml/analysis/VectorField$.html" target="template"><span class="object">(object)</span></a><a class="tplshow" href="io/github/mandar2812/dynaml/analysis/VectorField.html" target="template"><span class="class">(class)</span><span class="tplLink">VectorField</span></a></li></ol>
<ol class="packages"> </ol>
</li><li class="pack" title="io.github.mandar2812.dynaml.dataformat">
<a class="tplshow" href="io/github/mandar2812/dynaml/dataformat/package.html" target="template">io.github.mandar2812.dynaml.dataformat</a>
Expand Down

Large diffs are not rendered by default.

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ <h4 id="signature" class="signature">

<div id="types" class="types members">
<h3>Type Members</h3>
<ol><li name="io.github.mandar2812.dynaml.analysis.KernelMatrix" visbl="pub" data-isabs="true" fullComment="no" group="Ungrouped">
<ol><li name="io.github.mandar2812.dynaml.algebra.KernelMatrix" visbl="pub" data-isabs="true" fullComment="no" group="Ungrouped">
<a id="KernelMatrix[T]extendsSerializable"></a>
<a id="KernelMatrix[T]:KernelMatrix[T]"></a>
<h4 class="signature">
Expand All @@ -84,7 +84,7 @@ <h4 class="signature">
</span>
<p class="shortcomment cmt">Defines a trait which outlines the basic
functionality of Kernel Matrices.</p>
</li><li name="io.github.mandar2812.dynaml.analysis.SVMKernelMatrix" visbl="pub" data-isabs="false" fullComment="no" group="Ungrouped">
</li><li name="io.github.mandar2812.dynaml.algebra.SVMKernelMatrix" visbl="pub" data-isabs="false" fullComment="no" group="Ungrouped">
<a id="SVMKernelMatrixextendsKernelMatrix[breeze.linalg.DenseMatrix[Double]]withSerializable"></a>
<a id="SVMKernelMatrix:SVMKernelMatrix"></a>
<h4 class="signature">
Expand All @@ -93,7 +93,7 @@ <h4 class="signature">
<span class="kind">class</span>
</span>
<span class="symbol">
<a href="SVMKernelMatrix.html"><span class="name">SVMKernelMatrix</span></a><span class="result"> extends <a href="KernelMatrix.html" class="extype" name="io.github.mandar2812.dynaml.analysis.KernelMatrix">KernelMatrix</a>[<span class="extype" name="breeze.linalg.DenseMatrix">DenseMatrix</span>[<span class="extype" name="scala.Double">Double</span>]] with <span class="extype" name="scala.Serializable">Serializable</span></span>
<a href="SVMKernelMatrix.html"><span class="name">SVMKernelMatrix</span></a><span class="result"> extends <a href="KernelMatrix.html" class="extype" name="io.github.mandar2812.dynaml.algebra.KernelMatrix">KernelMatrix</a>[<span class="extype" name="breeze.linalg.DenseMatrix">DenseMatrix</span>[<span class="extype" name="scala.Double">Double</span>]] with <span class="extype" name="scala.Serializable">Serializable</span></span>
</span>
</h4><span class="permalink">
<a href="../../../../../index.html#io.github.mandar2812.dynaml.analysis.package@SVMKernelMatrixextendsKernelMatrix[breeze.linalg.DenseMatrix[Double]]withSerializable" title="Permalink" target="_top">
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -280,10 +280,10 @@ <h4 class="signature">
<span class="kind">def</span>
</span>
<span class="symbol">
<span class="name">buildKernelMatrix</span><span class="tparams">[<span name="S">S &lt;: <span class="extype" name="scala.Seq">Seq</span>[<span class="extype" name="scala.Double">Double</span>]</span>]</span><span class="params">(<span name="mappedData">mappedData: <span class="extype" name="io.github.mandar2812.dynaml.kernels.LocalSVMKernel.buildKernelMatrix.S">S</span></span>, <span name="length">length: <span class="extype" name="scala.Int">Int</span></span>)</span><span class="result">: <a href="../analysis/KernelMatrix.html" class="extype" name="io.github.mandar2812.dynaml.analysis.KernelMatrix">KernelMatrix</a>[<span class="extype" name="breeze.linalg.DenseMatrix">DenseMatrix</span>[<span class="extype" name="scala.Double">Double</span>]]</span>
<span class="name">buildKernelMatrix</span><span class="tparams">[<span name="S">S &lt;: <span class="extype" name="scala.Seq">Seq</span>[<span class="extype" name="scala.Double">Double</span>]</span>]</span><span class="params">(<span name="mappedData">mappedData: <span class="extype" name="io.github.mandar2812.dynaml.kernels.LocalSVMKernel.buildKernelMatrix.S">S</span></span>, <span name="length">length: <span class="extype" name="scala.Int">Int</span></span>)</span><span class="result">: <a href="../analysis/KernelMatrix.html" class="extype" name="io.github.mandar2812.dynaml.algebra.KernelMatrix">KernelMatrix</a>[<span class="extype" name="breeze.linalg.DenseMatrix">DenseMatrix</span>[<span class="extype" name="scala.Double">Double</span>]]</span>
</span>
</h4><span class="permalink">
<a href="../../../../../index.html#io.github.mandar2812.dynaml.kernels.AnovaCovFunc@buildKernelMatrix[S&lt;:Seq[Index]](mappedData:S,length:Int):io.github.mandar2812.dynaml.analysis.KernelMatrix[breeze.linalg.DenseMatrix[Double]]" title="Permalink" target="_top">
<a href="../../../../../index.html#io.github.mandar2812.dynaml.kernels.AnovaCovFunc@buildKernelMatrix[S&lt;:Seq[Index]](mappedData:S,length:Int):io.github.mandar2812.dynaml.algebra.KernelMatrix[breeze.linalg.DenseMatrix[Double]]" title="Permalink" target="_top">
<img src="../../../../../lib/permalink.png" alt="Permalink" />
</a>
</span>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -280,10 +280,10 @@ <h4 class="signature">
<span class="kind">def</span>
</span>
<span class="symbol">
<span class="name">buildKernelMatrix</span><span class="tparams">[<span name="S">S &lt;: <span class="extype" name="scala.Seq">Seq</span>[<span class="extype" name="breeze.linalg.DenseVector">DenseVector</span>[<span class="extype" name="scala.Double">Double</span>]]</span>]</span><span class="params">(<span name="mappedData">mappedData: <span class="extype" name="io.github.mandar2812.dynaml.kernels.LocalSVMKernel.buildKernelMatrix.S">S</span></span>, <span name="length">length: <span class="extype" name="scala.Int">Int</span></span>)</span><span class="result">: <a href="../analysis/KernelMatrix.html" class="extype" name="io.github.mandar2812.dynaml.analysis.KernelMatrix">KernelMatrix</a>[<span class="extype" name="breeze.linalg.DenseMatrix">DenseMatrix</span>[<span class="extype" name="scala.Double">Double</span>]]</span>
<span class="name">buildKernelMatrix</span><span class="tparams">[<span name="S">S &lt;: <span class="extype" name="scala.Seq">Seq</span>[<span class="extype" name="breeze.linalg.DenseVector">DenseVector</span>[<span class="extype" name="scala.Double">Double</span>]]</span>]</span><span class="params">(<span name="mappedData">mappedData: <span class="extype" name="io.github.mandar2812.dynaml.kernels.LocalSVMKernel.buildKernelMatrix.S">S</span></span>, <span name="length">length: <span class="extype" name="scala.Int">Int</span></span>)</span><span class="result">: <a href="../analysis/KernelMatrix.html" class="extype" name="io.github.mandar2812.dynaml.algebra.KernelMatrix">KernelMatrix</a>[<span class="extype" name="breeze.linalg.DenseMatrix">DenseMatrix</span>[<span class="extype" name="scala.Double">Double</span>]]</span>
</span>
</h4><span class="permalink">
<a href="../../../../../index.html#io.github.mandar2812.dynaml.kernels.AnovaKernel@buildKernelMatrix[S&lt;:Seq[Index]](mappedData:S,length:Int):io.github.mandar2812.dynaml.analysis.KernelMatrix[breeze.linalg.DenseMatrix[Double]]" title="Permalink" target="_top">
<a href="../../../../../index.html#io.github.mandar2812.dynaml.kernels.AnovaKernel@buildKernelMatrix[S&lt;:Seq[Index]](mappedData:S,length:Int):io.github.mandar2812.dynaml.algebra.KernelMatrix[breeze.linalg.DenseMatrix[Double]]" title="Permalink" target="_top">
<img src="../../../../../lib/permalink.png" alt="Permalink" />
</a>
</span>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -280,10 +280,10 @@ <h4 class="signature">
<span class="kind">def</span>
</span>
<span class="symbol">
<span class="name">buildKernelMatrix</span><span class="tparams">[<span name="S">S &lt;: <span class="extype" name="scala.Seq">Seq</span>[<span class="extype" name="scala.Double">Double</span>]</span>]</span><span class="params">(<span name="mappedData">mappedData: <span class="extype" name="io.github.mandar2812.dynaml.kernels.LocalSVMKernel.buildKernelMatrix.S">S</span></span>, <span name="length">length: <span class="extype" name="scala.Int">Int</span></span>)</span><span class="result">: <a href="../analysis/KernelMatrix.html" class="extype" name="io.github.mandar2812.dynaml.analysis.KernelMatrix">KernelMatrix</a>[<span class="extype" name="breeze.linalg.DenseMatrix">DenseMatrix</span>[<span class="extype" name="scala.Double">Double</span>]]</span>
<span class="name">buildKernelMatrix</span><span class="tparams">[<span name="S">S &lt;: <span class="extype" name="scala.Seq">Seq</span>[<span class="extype" name="scala.Double">Double</span>]</span>]</span><span class="params">(<span name="mappedData">mappedData: <span class="extype" name="io.github.mandar2812.dynaml.kernels.LocalSVMKernel.buildKernelMatrix.S">S</span></span>, <span name="length">length: <span class="extype" name="scala.Int">Int</span></span>)</span><span class="result">: <a href="../analysis/KernelMatrix.html" class="extype" name="io.github.mandar2812.dynaml.algebra.KernelMatrix">KernelMatrix</a>[<span class="extype" name="breeze.linalg.DenseMatrix">DenseMatrix</span>[<span class="extype" name="scala.Double">Double</span>]]</span>
</span>
</h4><span class="permalink">
<a href="../../../../../index.html#io.github.mandar2812.dynaml.kernels.CauchyCovFunc@buildKernelMatrix[S&lt;:Seq[Index]](mappedData:S,length:Int):io.github.mandar2812.dynaml.analysis.KernelMatrix[breeze.linalg.DenseMatrix[Double]]" title="Permalink" target="_top">
<a href="../../../../../index.html#io.github.mandar2812.dynaml.kernels.CauchyCovFunc@buildKernelMatrix[S&lt;:Seq[Index]](mappedData:S,length:Int):io.github.mandar2812.dynaml.algebra.KernelMatrix[breeze.linalg.DenseMatrix[Double]]" title="Permalink" target="_top">
<img src="../../../../../lib/permalink.png" alt="Permalink" />
</a>
</span>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -282,10 +282,10 @@ <h4 class="signature">
<span class="kind">def</span>
</span>
<span class="symbol">
<span class="name">buildKernelMatrix</span><span class="tparams">[<span name="S">S &lt;: <span class="extype" name="scala.Seq">Seq</span>[<span class="extype" name="breeze.linalg.DenseVector">DenseVector</span>[<span class="extype" name="scala.Double">Double</span>]]</span>]</span><span class="params">(<span name="mappedData">mappedData: <span class="extype" name="io.github.mandar2812.dynaml.kernels.LocalSVMKernel.buildKernelMatrix.S">S</span></span>, <span name="length">length: <span class="extype" name="scala.Int">Int</span></span>)</span><span class="result">: <a href="../analysis/KernelMatrix.html" class="extype" name="io.github.mandar2812.dynaml.analysis.KernelMatrix">KernelMatrix</a>[<span class="extype" name="breeze.linalg.DenseMatrix">DenseMatrix</span>[<span class="extype" name="scala.Double">Double</span>]]</span>
<span class="name">buildKernelMatrix</span><span class="tparams">[<span name="S">S &lt;: <span class="extype" name="scala.Seq">Seq</span>[<span class="extype" name="breeze.linalg.DenseVector">DenseVector</span>[<span class="extype" name="scala.Double">Double</span>]]</span>]</span><span class="params">(<span name="mappedData">mappedData: <span class="extype" name="io.github.mandar2812.dynaml.kernels.LocalSVMKernel.buildKernelMatrix.S">S</span></span>, <span name="length">length: <span class="extype" name="scala.Int">Int</span></span>)</span><span class="result">: <a href="../analysis/KernelMatrix.html" class="extype" name="io.github.mandar2812.dynaml.algebra.KernelMatrix">KernelMatrix</a>[<span class="extype" name="breeze.linalg.DenseMatrix">DenseMatrix</span>[<span class="extype" name="scala.Double">Double</span>]]</span>
</span>
</h4><span class="permalink">
<a href="../../../../../index.html#io.github.mandar2812.dynaml.kernels.CauchyKernel@buildKernelMatrix[S&lt;:Seq[Index]](mappedData:S,length:Int):io.github.mandar2812.dynaml.analysis.KernelMatrix[breeze.linalg.DenseMatrix[Double]]" title="Permalink" target="_top">
<a href="../../../../../index.html#io.github.mandar2812.dynaml.kernels.CauchyKernel@buildKernelMatrix[S&lt;:Seq[Index]](mappedData:S,length:Int):io.github.mandar2812.dynaml.algebra.KernelMatrix[breeze.linalg.DenseMatrix[Double]]" title="Permalink" target="_top">
<img src="../../../../../lib/permalink.png" alt="Permalink" />
</a>
</span>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -280,10 +280,10 @@ <h4 class="signature">
<span class="kind">def</span>
</span>
<span class="symbol">
<span class="name">buildKernelMatrix</span><span class="tparams">[<span name="S">S &lt;: <span class="extype" name="scala.Seq">Seq</span>[<span class="extype" name="scala.Int">Int</span>]</span>]</span><span class="params">(<span name="mappedData">mappedData: <span class="extype" name="io.github.mandar2812.dynaml.kernels.LocalSVMKernel.buildKernelMatrix.S">S</span></span>, <span name="length">length: <span class="extype" name="scala.Int">Int</span></span>)</span><span class="result">: <a href="../analysis/KernelMatrix.html" class="extype" name="io.github.mandar2812.dynaml.analysis.KernelMatrix">KernelMatrix</a>[<span class="extype" name="breeze.linalg.DenseMatrix">DenseMatrix</span>[<span class="extype" name="scala.Double">Double</span>]]</span>
<span class="name">buildKernelMatrix</span><span class="tparams">[<span name="S">S &lt;: <span class="extype" name="scala.Seq">Seq</span>[<span class="extype" name="scala.Int">Int</span>]</span>]</span><span class="params">(<span name="mappedData">mappedData: <span class="extype" name="io.github.mandar2812.dynaml.kernels.LocalSVMKernel.buildKernelMatrix.S">S</span></span>, <span name="length">length: <span class="extype" name="scala.Int">Int</span></span>)</span><span class="result">: <a href="../analysis/KernelMatrix.html" class="extype" name="io.github.mandar2812.dynaml.algebra.KernelMatrix">KernelMatrix</a>[<span class="extype" name="breeze.linalg.DenseMatrix">DenseMatrix</span>[<span class="extype" name="scala.Double">Double</span>]]</span>
</span>
</h4><span class="permalink">
<a href="../../../../../index.html#io.github.mandar2812.dynaml.kernels.CoRegCauchyKernel@buildKernelMatrix[S&lt;:Seq[Index]](mappedData:S,length:Int):io.github.mandar2812.dynaml.analysis.KernelMatrix[breeze.linalg.DenseMatrix[Double]]" title="Permalink" target="_top">
<a href="../../../../../index.html#io.github.mandar2812.dynaml.kernels.CoRegCauchyKernel@buildKernelMatrix[S&lt;:Seq[Index]](mappedData:S,length:Int):io.github.mandar2812.dynaml.algebra.KernelMatrix[breeze.linalg.DenseMatrix[Double]]" title="Permalink" target="_top">
<img src="../../../../../lib/permalink.png" alt="Permalink" />
</a>
</span>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -280,10 +280,10 @@ <h4 class="signature">
<span class="kind">def</span>
</span>
<span class="symbol">
<span class="name">buildKernelMatrix</span><span class="tparams">[<span name="S">S &lt;: <span class="extype" name="scala.Seq">Seq</span>[<span class="extype" name="scala.Int">Int</span>]</span>]</span><span class="params">(<span name="mappedData">mappedData: <span class="extype" name="io.github.mandar2812.dynaml.kernels.LocalSVMKernel.buildKernelMatrix.S">S</span></span>, <span name="length">length: <span class="extype" name="scala.Int">Int</span></span>)</span><span class="result">: <a href="../analysis/KernelMatrix.html" class="extype" name="io.github.mandar2812.dynaml.analysis.KernelMatrix">KernelMatrix</a>[<span class="extype" name="breeze.linalg.DenseMatrix">DenseMatrix</span>[<span class="extype" name="scala.Double">Double</span>]]</span>
<span class="name">buildKernelMatrix</span><span class="tparams">[<span name="S">S &lt;: <span class="extype" name="scala.Seq">Seq</span>[<span class="extype" name="scala.Int">Int</span>]</span>]</span><span class="params">(<span name="mappedData">mappedData: <span class="extype" name="io.github.mandar2812.dynaml.kernels.LocalSVMKernel.buildKernelMatrix.S">S</span></span>, <span name="length">length: <span class="extype" name="scala.Int">Int</span></span>)</span><span class="result">: <a href="../analysis/KernelMatrix.html" class="extype" name="io.github.mandar2812.dynaml.algebra.KernelMatrix">KernelMatrix</a>[<span class="extype" name="breeze.linalg.DenseMatrix">DenseMatrix</span>[<span class="extype" name="scala.Double">Double</span>]]</span>
</span>
</h4><span class="permalink">
<a href="../../../../../index.html#io.github.mandar2812.dynaml.kernels.CoRegDiracKernel@buildKernelMatrix[S&lt;:Seq[Index]](mappedData:S,length:Int):io.github.mandar2812.dynaml.analysis.KernelMatrix[breeze.linalg.DenseMatrix[Double]]" title="Permalink" target="_top">
<a href="../../../../../index.html#io.github.mandar2812.dynaml.kernels.CoRegDiracKernel@buildKernelMatrix[S&lt;:Seq[Index]](mappedData:S,length:Int):io.github.mandar2812.dynaml.algebra.KernelMatrix[breeze.linalg.DenseMatrix[Double]]" title="Permalink" target="_top">
<img src="../../../../../lib/permalink.png" alt="Permalink" />
</a>
</span>
Expand Down

0 comments on commit 3c98f2d

Please sign in to comment.