Update contour-utils.js normalizeSeries to do filtering

[wasbridge]

Filter data to a desired num pts (configurable with new data object ({filter:true, filterNumPts:1000}))

Always keeps first/last -- and then for all the ones in the middle it grabs the high/low per interval

[wasbridge]

Anything you want me to do to make it easier to merge this? I can revert the /dist folder if you need.

[jaimedp]

Hi billy, I'm thinking of what would be the correct way to incorporate the point simplification into the data workflow. I'm not sure about 'loosing' the data points on normalization, because the next visualization may actually use the full data set do display other characteristics of the data. That's why I liked the idea of filtering the data that you will be rendering not not the original data set.

Also, I think we should provide a way to specify or pass in different filtering/data simplification functions so the user decides what's the best way to simplify the data.

What do you think?

PD. Yes we also need to remove the /dist from the PR :)

[wasbridge]

I definitely agree with the ability to pass a custom filtering function.

As for where to 'loose' the data it depends on if we want all
visualizations to have the 'same' data or not. Think of a visualization
that displays the mean, median, mode, min max etc. Do we want those stats
to be based on the full data set or the filtered set? If you answer the
full set then we need to filter per visualization -- if you answered the
filtered set we should do it in normalization.

My answer after thinking about it is per visualization. Do you agree?

On Tue, Feb 10, 2015 at 4:30 PM, Jaime del Palacio <notifications@github.com

wrote:

Hi billy, I'm thinking of what would be the correct way to incorporate the
point simplification into the data workflow. I'm not sure about 'loosing'
the data points on normalization, because the next visualization may
actually use the full data set do display other characteristics of the
data. That's why I liked the idea of filtering the data that you will be
rendering not not the original data set.

Also, I think we should provide a way to specify or pass in different
filtering/data simplification functions so the user decides what's the best
way to simplify the data.

What do you think?

—
Reply to this email directly or view it on GitHub
#202 (comment).

[jaimedp]

Yes, I think the simplification should be per visualization without modifying the original data set. How about something like this:

• We add a preprocess function to each visualization's config object (by default is a noop, returning the same dataset)
• We provide a set (we can start with only one) simplification functions on _.nw namespace.

Then you would use it something like:

new Contour(..)
  .cartesian()
  .line(myBidDataSet, { preprocess: _.nw.simplifications.minMaxFilter(1000) })
  .statsVisualization(myBidDataSet) // <- this gets original data set
  .render()

the minMaxFilter(1000) would return a simplification function configured for 1000 points.

then inside the line we would have

var data = options.preprocess(raw);

What do you think? (not crazy about the simplifications name for the namespace)

[wasbridge]

I like the idea of a preprocess function per viz

How about we call it a dataFilter?

Then the filter I coded could be bound to _.nw.dataFilters.minMaxFilter

On Wed, Feb 11, 2015 at 12:35 PM, Jaime del Palacio <
notifications@github.com> wrote:

Yes, I think the simplification should be per visualization without
modifying the original data set. How about something like this:

• We add a preprocess function to each visualization's config object
  (by default is a noop, returning the same dataset)
• We provide a set (we can start with only one) simplification
  functions on _.nw namespace.

Then you would use it something like:

new Contour(..)
.cartesian()
.line(myBidDataSet, { preprocess: _.nw.simplifications.minMaxFilter(1000) })
.statsVisualization(myBidDataSet) // <- this gets original data set
.render()

the minMaxFilter(1000) would return a simplification function configured
for 1000 points.

What do you think? (not crazy about the simplifications name for the
namespace)

—
Reply to this email directly or view it on GitHub
#202 (comment).

[jaimedp]

Sorry for the delay,

dataFilters sounds good

[wasbridge]

Closing in favor of optimization2 pull request