Replace data with stats in vl.toVegaSpec

[domoritz]

{
  fieldname:{min: value, max:value} if Q or {cardinality: value} if O 
}

[kanitw]

The 'fieldname' should be aggr_fieldname right?E.g.,

sum_profit

—
Sent from Mailbox

On Fri, Dec 12, 2014 at 4:11 PM, Dominik Moritz notifications@github.com
wrote:

{
  fieldname:{min: value, max:value} if Q or {cardinality: value} if O 
}

---

Reply to this email directly or view it on GitHub:
#57

[domoritz]

why?

[kanitw]

Suppose we still allow casting binned Q to be O.
We should just get cardinality of the binned field instead of min, max.

For clarity, I will explain why min, max is insufficient first.

In that case, having min, max give us b.start, b.end, b.step in the first part of code below.

bin.numbins = function(input){
    var stats = {min: +Infinity, max: -Infinity};
    input.forEach(function(d) {
      var v = accessor(d);
      if (v > stats.max) stats.max = v;
      if (v < stats.min) stats.min = v;
    });
    var b = bins(stats, {maxbins: 20});  

However, we still need to to go through the input to really get # of unique bins:

    var uniqueBins = {}; // reset (HACK)
    input.forEach(function(d) {
      var v = accessor(d);
      var vbin = b.start + b.step * ~~((v - b.start) / b.step);
      uniqueBins[vbin] = 1;
    });
    return vg.keys(uniqueBins).length;
  }

But the # of unique bin is actually cardinality of the new derived column from binning.
So maybe we just get cardinality of the resulting table.

[domoritz]

Okay, if we don't want to show empty in, then I agree with your first point about min and max not being sufficient. But in that case we will just use the cardinality of the query result.

I do not agree with your second point because if we have only a small number of non-empty (is that what you mean by unique?) bins, then we might want to make them smaller. But let's maybe not worry about this case for now and just use the cardinality of the query result as you said.

[kanitw]

Okay, if we don't want to show empty in, then I agree with your first point about min and max not being sufficient. But in that case we will just use the cardinality of the query result.

If we don't hide empty bins, we should not cast the data to be ordinal because ordinal scale would not know about empty bins. (It just use the output from binning fn as ordinal).

In the case the we keep the binned Q to be Q type. We don't even need cardinality (# of non-empty bins)/# of bin because we can assume that it will be roughly below 20. (But having the # of bin -- including empty ones -- is a little helpful.)

(is that what you mean by unique?)

Yeah uniqueBins = unique non-empty bins.

then we might want to make them smaller

make what smaller? (what is them?)

[domoritz]

Well, the binning function could return empty bins as well (maybe need to refactor code to support this, though).

them is the bins. Assume we have very dense regions, then we might want more bins because a lot of the bins are empty. This way we have a not so small number of non-empty bins.

Okay, let's go with the simplest version first. We only put min and max in the stats and if the user chooses O, we just take the cardinality of the query result but not change the bin size.

[kanitw]

Ok. The simplest version didn't even need min, max for size calculation though.

But you should go ahead and implement min, max as they are useful for brushing & link + dynamic query, which should be included in the UI somehow anyway.

[domoritz]

Well, we need min and max to calculate optimal bin sizes.

I'm working on the stats part.