[Grouping] Custom aggregation functions

[begemott]

Hi

I've tried the new grouping feature and find out that it is impossible to provide a function that calculates a group value. For example, if I want to group by a column that contains DateTime values, now the Grid creates a group for each distinct value in the column. Instead it would be better to have groups like Today, Yesterday, This Week, This Month and so on. It can be done by adding a new field like valueFormatter or valueBuilder in the columnDef.grouping object. The type of the field is function that accept a cell value and return a formatted group value.

[c0bra]

Thanks for the suggestion!

[AgDude]

I am looking into what it will take to implement this. Note that the current implementation holds a "running" value as we iterate over the rows, which will make writing these functions a little less intuitive than simply operating on an array.

For reference, this is the list of functions I need to implement for a project I am migrating to uiGrid (some are already included):

• 'average'
• 'median'
• 'mode'
• 'stdevp'
• 'varp'
• 'stdev'
• 'var'
• 'min'
• 'max'
• 'sum'
• 'count'
• 'list' - In our application, this concatenates unique values.
• 'concat' - concatenate all values (may need to set a length limit)

[AgDude]

Here is my current thought on an API. I haven't started implementing this, but I am interested in feedback on whether this sounds reasonable. One of the challenges I am considering is calculating something like "median" or "mode", where you need to have access to the entire array (admittedly at a performance cost).

Add a columnDef option groupingAggregationTypes which is an object where the keys are the aggregation name, and the values could be either a function or an object with two functions. The first function would need to accept the aggregation object which is stored in processingState.runningAggregations and the fieldValue, and would be responsible for modifying the value property of the object. It could also add/modify other properties if needed. Providing two functions would allow you to use one function to build the aggregation object during iteration, and another to be executed at writeOut. The writeOut method of the service should probably do some cleanup also, so when it re-runs it will always start with a fresh aggregation object.

For example this would be the option object to add 'mean' (same as build in average) and the 'median':

{mean: function(aggregation, fieldValue){
    aggregation.count++;
    if (!isNaN(numValue)){
      aggregation.sum += fieldValue;
     }
     aggregation.value = aggregation.sum / aggregation.count;
  },
median: {
  running: function(aggregation, fieldValue){
    if ( typeof aggregation.allValues === 'undefined' ) { aggregation.allValues = []; }
    aggregation.allValues.push(fieldValue);
  },
  writeOut: function(aggregation){
     aggregation.allValues.sort();
     var half = Math.floor(aggregation.allValues.length/2);
     return aggregation.allValues[half];
  }
}

This proposal is by no means simple, so if you have other ideas please share. I think it is unavoidable that we sometimes need a way to process the result after all rows have been read. One option is that custom functions will always be presented with all of the values, and simply take a performance hit compared to the built-in ones.

[AgDude]

Another possible way to handle functions which need all of the data would be to support only running functions, but add an event beforeWriteOutAggregation, which would allow the value to be modified. This probably makes implementing those functions even more complicated, but I think it would take some of the responsibility out of uiGrid and put it on the user. Another advantage to this method would be it would allow modifying of a built-in function (for example rounding an avg).

[begemott]

I think the first option is better.

[PaulL1]

This should be included in my treeBase branch, it includes a custom running aggregator, and a custom aggregation finalizer. In theory it can do many cool things.

[PaulL1]

This should be merged now, can you take a look when you get a chance?

[AgDude]

@PaulL1 I am looking at this today. This looks like it should accomplish what I had in mind, but I have some thoughts on cleaning up the api a bit. If you are working on grid issues today, try to catch me on gitter. I am happy to write the code, but want to run some thoughts past you first. I like the two grid options of customTreeAggregationFn and customTreeAggregationFinalizerFn (especially the fact that the finalizer can be used with builtin functions.)

Here are a few things on my mind:

• Let's make it easier to put a custom aggregations in the menu (probably a public api method)
• Refactor the builtin aggregators to be more similar to custom aggregators. This would allow the user to still choose a builtin aggregator with a string, but then we assign a function to the col object, and get rid of the switch/case in treeBase.aggregate. I think it would remove a few if/else statements in other places too. It would also allow more shared code with the menu feature mentioned above.
• Some means of applying the built-in finalizer after a custom finalizer. This is really a minor issue, but in this case I want to set aggergation.value in my finalizer, and it seems redundant that I also need to set aggregation.rendered.

[AgDude]

Here is a plunker which demonstrates a median calculation and a couple flexible factories for creating custom aggregators. Maybe I'll add this to the tutorial at some point.

Be warned that this adds an additional sort within each group, which will be slow on large datasets.
http://plnkr.co/edit/3lAuYzzGBwaQ3PJRSvh8?p=preview