Flexible map functions

[michael]

What about using a traditional Map/Reduce API like CouchDB for instance? This could also solve the problem with filtering multivalued properties.

var countries = crossfilter([
  {name: "USA", languages: ["Spanish", "English", "Chinese", "French"], continent: "North America"},
  {name: "Canada", languages: ["German", "English"], continent: "North America"},
  {name: "UK", languages: ["English"], continent: "Europe"}
]);

function map(d) {
  d.languages.forEach(function(lang) {
    emit(lang, d);
  }
}

countriesByLanguage = countries.dimension(map);
countriesByLanguage.filter('English');

countriesByLanguage.top(Infinity) // => [USA, Canada, UK]

If you'd rather want to search for language+continent the map function would look like this:

function map(d) {
  d.languages.forEach(function(lang) {
    emit([lang,d.continent], d);
  }
}


countriesByLanguageAndContinent = countries.dimension(map);
countriesByLanguageAndContinent.filter(['English', 'North America']);

countriesByLanguage.top(Infinity) // => [USA, Canada]

Actually what you'd get is flexible indices that you can use for fast lookups.
Obviously this breaks the current API, but I just wanted to drop that thought. :)

What do you think?

[mbostock]

Interesting. Thanks for the suggestion!

[jeroen]

This would be great. The emit function is also used in mongo map/reduce.

The classic example is to use map/reduce for e.g. creating a wordcloud from records containing text sentences, so every sentence needs to get mapped to multiple words.

[optimuspaul]

I need this too. Finding it very frustrating to try and deal with a dimension where each object can have multiple values assigned.

[RandomEtc]

As discussed in #151 an active fork is being developed in a new Crossfilter Organization. Please take further discussion there (if you haven't already) where it should be warmly welcomed by the new maintainers. Cheers!