Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP
Fetching contributors…

Cannot retrieve contributors at this time

364 lines (264 sloc) 13.107 kb
<!DOCTYPE html>
<html>
<head>
<title>aggregate.coffee</title>
<meta http-equiv="content-type" content="text/html; charset=UTF-8">
<link rel="stylesheet" media="all" href="resources/base.css" />
<link rel="stylesheet" media="all" href="resources/module.css" />
</head>
<body>
<header>
<h1>aggregate.coffee</h1>
</header>
<div id="content">
<div id="sidebar">
<h4>Functions</h4>
<ul class="functionlist">
<li><a href="#functions.$sum">functions.$sum</a></li>
<li><a href="#functions.$sumSquares">functions.$sumSquares</a></li>
<li><a href="#functions.$count">functions.$count</a></li>
<li><a href="#functions.$min">functions.$min</a></li>
<li><a href="#functions.$max">functions.$max</a></li>
<li><a href="#functions.$push">functions.$push</a></li>
<li><a href="#functions.$addToSet">functions.$addToSet</a></li>
<li><a href="#functions.$average">functions.$average</a></li>
<li><a href="#functions.$variance">functions.$variance</a></li>
<li><a href="#functions.$standardDeviation">functions.$standardDeviation</a></li>
<li><a href="#percentileCreator">percentileCreator</a></li>
<li><a href="#aggregate">aggregate</a></li>
<li><a href="#aggregateAt">aggregateAt</a></li>
<li><a href="#groupBy">groupBy</a></li>
<li><a href="#groupByAt">groupByAt</a></li>
<li><a href="#timeSeriesCalculator">timeSeriesCalculator</a></li>
<li><a href="#timeSeriesGroupByCalculator">timeSeriesGroupByCalculator</a></li>
</ul>
</div>
<div id="main">
<div id="functions">
<h2>Functions</h2>
<div class="function">
<div class="header">
<a id="functions.$sum"><h3>functions.$sum(values)</h3></a>
</div>
</div>
<div class="function">
<div class="header">
<a id="functions.$sumSquares"><h3>functions.$sumSquares(values)</h3></a>
</div>
</div>
<div class="function">
<div class="header">
<a id="functions.$count"><h3>functions.$count(values)</h3></a>
</div>
</div>
<div class="function">
<div class="header">
<a id="functions.$min"><h3>functions.$min(values)</h3></a>
</div>
</div>
<div class="function">
<div class="header">
<a id="functions.$max"><h3>functions.$max(values)</h3></a>
</div>
</div>
<div class="function">
<div class="header">
<a id="functions.$push"><h3>functions.$push(values)</h3></a>
</div>
<div>
<p>An Array of all values (allows duplicates). Can be used for drill down when you know they will be unique.</p>
</div>
</div>
<div class="function">
<div class="header">
<a id="functions.$addToSet"><h3>functions.$addToSet(values)</h3></a>
</div>
<div>
<p>An Array of unique values. This is good for generating an OLAP dimension or drill down.</p>
</div>
</div>
<div class="function">
<div class="header">
<a id="functions.$average"><h3>functions.$average(values)</h3></a>
</div>
</div>
<div class="function">
<div class="header">
<a id="functions.$variance"><h3>functions.$variance(values)</h3></a>
</div>
</div>
<div class="function">
<div class="header">
<a id="functions.$standardDeviation"><h3>functions.$standardDeviation(values)</h3></a>
</div>
</div>
<div class="function">
<div class="header">
<a id="percentileCreator"><h3>percentileCreator(p)</h3></a>
</div>
<div>
<p>When the user passes in <code>$p&lt;n&gt;</code> as an aggregation function, this <code>percentileCreator</code> is called to return the appropriate percentile function.
The returned function will find the <code>&lt;n&gt;</code>th percentile where <code>&lt;n&gt;</code> is some number in the form of <code>##[.##]</code>. (e.g. <code>$p40</code>, <code>$p99</code>, <code>$p99.9</code>).</p>
<p>Note: <code>$median</code> is an alias for <code>$p50</code>.</p>
<p>There is no official definition of percentile. The function returned by this <code>percentileCreator</code> uses the Excel interpolation algorithm
which is close to the NIST recommendation and makes the most sense to me.</p>
</div>
</div>
<div class="function">
<div class="header">
<a id="aggregate"><h3>aggregate(list, aggregations)</h3></a>
</div>
<div>
<p>Takes a list like this:</p>
<pre><code>{aggregate} = require('../')
list = [
{ ObjectID: '1', KanbanState: 'In progress', PlanEstimate: 5, TaskRemainingTotal: 20 },
{ ObjectID: '2', KanbanState: 'Ready to pull', PlanEstimate: 3, TaskRemainingTotal: 5 },
{ ObjectID: '3', KanbanState: 'Ready to pull', PlanEstimate: 5, TaskRemainingTotal: 12 }
]
</code></pre>
<p>and a list of aggregations like this:</p>
<pre><code>aggregations = [
{field: 'ObjectID', f: '$count'}
{as: 'Drill-down', field:'ObjectID', f:'$push'}
{field: 'PlanEstimate', f: '$sum'}
{as: 'mySum', field: 'PlanEstimate', f: (values) -&gt;
temp = 0
for v in values
temp += v
return temp
}
]
</code></pre>
<p>and returns the aggregations like this:</p>
<pre><code>a = aggregate(list, aggregations)
console.log(a)
# { 'ObjectID_$count': 3,
# 'Drill-down': [ '1', '2', '3' ],
# 'PlanEstimate_$sum': 13,
# mySum: 13 }
</code></pre>
<p>For each aggregation, you must provide a <code>field</code> and <code>f</code> (function) value. You can optionally
provide an alias for the aggregation with the 'as` field. There are a number of built in functions
documented above.</p>
<p>Alternatively, you can provide your own function (it takes one parameter, which is an
Array of values to aggregate) like the <code>mySum</code> example in our <code>aggregations</code> list above.</p>
</div>
</div>
<div class="function">
<div class="header">
<a id="aggregateAt"><h3>aggregateAt(atArray, aggregations)</h3></a>
</div>
<div>
<p>Each row in atArray is passed to the <code>aggregate</code> function and the results are collected into a single output.
This is essentially a wrapper around the aggregate function so the spec parameter is the same.</p>
</div>
</div>
<div class="function">
<div class="header">
<a id="groupBy"><h3>groupBy(list, spec)</h3></a>
</div>
<div>
<p>Takes a list like this:</p>
<pre><code>{groupBy} = require('../')
list = [
{ ObjectID: '1', KanbanState: 'In progress', PlanEstimate: 5, TaskRemainingTotal: 20 },
{ ObjectID: '2', KanbanState: 'Ready to pull', PlanEstimate: 3, TaskRemainingTotal: 5 },
{ ObjectID: '3', KanbanState: 'Ready to pull', PlanEstimate: 5, TaskRemainingTotal: 12 }
]
</code></pre>
<p>and a spec like this:</p>
<pre><code>spec = {
groupBy: 'KanbanState',
aggregations: [
{field: 'ObjectID', f: '$count'}
{as: 'Drill-down', field:'ObjectID', f:'$push'}
{field: 'PlanEstimate', f: '$sum'}
{as: 'mySum', field: 'PlanEstimate', f: (values) -&gt;
temp = 0
for v in values
temp += v
return temp
}
]
}
</code></pre>
<p>Returns the aggregations like this:</p>
<pre><code>a = groupBy(list, spec)
console.log(a)
# { 'In progress':
# { 'ObjectID_$count': 1,
# 'Drill-down': [ '1' ],
# 'PlanEstimate_$sum': 5,
# mySum: 5 },
# 'Ready to pull':
# { 'ObjectID_$count': 2,
# 'Drill-down': [ '2', '3' ],
# 'PlanEstimate_$sum': 8,
# mySum: 8 } }
</code></pre>
<p>The first element of this specification is the <code>groupBy</code> field. This is analagous to
the <code>GROUP BY</code> column in an SQL express.</p>
<p>Uses the same aggregation functions at the <code>aggregate</code> function.</p>
</div>
</div>
<div class="function">
<div class="header">
<a id="groupByAt"><h3>groupByAt(atArray, spec)</h3></a>
</div>
<div>
<p>Each row in atArray is passed to the <code>groupBy</code> function and the results are collected into a single output.</p>
<p>This function also finds all the unique groupBy values in all rows of the output and pads the output with blank/zero rows to cover
each unique groupBy value.</p>
<p>This is essentially a wrapper around the groupBy function so the spec parameter is the same with the addition of the <code>uniqueValues</code> field.
The ordering specified in <code>spec.uniqueValues</code> (optional) will be honored. Any additional unique values that aggregateAt finds will be added to
the uniqueValues list at the end. This gives you the best of both worlds. The ability to specify the order without the risk of the
data containing more values than you originally thought when you created spec.uniqueValues.</p>
<p>Note: <code>groupByAt</code> has the side-effect that <code>spec.uniqueValues</code> are upgraded with the missing values.
You can use this if you want to do more calculations at the calling site.</p>
</div>
</div>
<div class="function">
<div class="header">
<a id="timeSeriesCalculator"><h3>timeSeriesCalculator(snapshotArray, config)</h3></a>
</div>
<div>
<p>Takes an MVCC style <code>snapshotArray</code> array and returns the time series calculations <code>At</code> each moment specified by
the ChartTimeRange spec (<code>rangeSpec</code>) within the config object.</p>
<p>This is really just a thin wrapper around various ChartTime calculations, so look at the documentation for each of
those to get the detail picture of what this timeSeriesCalculator does. The general flow is:</p>
<ol>
<li>Use <code>ChartTimeRange</code> and <code>ChartTimeIterator</code> against the <code>rangeSpec</code> to find the points for the x-axis.
We're interested in the ends of those time ranges so the output of this work is a <code>listOfAtCTs</code> array.</li>
<li>Use <code>snapshotArray_To_AtArray</code> to figure out what state those objects were in at each point in the <code>listOfAtCTs</code> array.
The output of this operation is called an <code>atArray</code></li>
<li>Use <code>deriveFieldsAt</code> to add fields in each object in the <code>atArray</code> whose values are derived from the other fields in the object.</li>
<li>Use <code>aggregateAt</code> to calculate aggregations into an <code>aggregationAtArray</code> which contains chartable values.</li>
</ol>
<p>Note: We assume the snapshotArray is sorted by the config.snapshotValidFromField</p>
</div>
</div>
<div class="function">
<div class="header">
<a id="timeSeriesGroupByCalculator"><h3>timeSeriesGroupByCalculator(snapshotArray, config)</h3></a>
</div>
<div>
<p>Takes an MVCC syle <code>snapshotArray</code> array and returns the data groupedBy a particular field <code>At</code> each moment specified by
the ChartTimeRange spec (<code>rangeSpec</code>) within the config object. </p>
<p>This is really just a thin wrapper around various ChartTime calculations, so look at the documentation for each of
those to get the detail picture of what this timeSeriesGroupByCalculator does. The general flow is:</p>
<ol>
<li>Use <code>ChartTimeRange</code> and <code>ChartTimeIterator</code> against the <code>rangeSpec</code> to find the points for the x-axis.
We're interested in the ends of those time ranges so the output of this work is a <code>listOfAtCTs</code> array.</li>
<li>Use <code>snapshotArray_To_AtArray</code> to figure out what state those objects were in at each point in the <code>listOfAtCTs</code> array.
The output of this operation is called an <code>atArray</code></li>
<li>Use <code>groupByAt</code> to create a <code>groupByAtArray</code> of grouped aggregations to chart</li>
</ol>
<p>Note: We assume the snapshotArray is sorted by the config.snapshotValidFromField</p>
</div>
</div>
</div>
</div>
</div>
</body>
</html>
Jump to Line
Something went wrong with that request. Please try again.