API Reference

Kaleb Hornsby edited this page Jul 6, 2018 · 53 revisions

API Reference

Everything in Crossfilter is scoped under the crossfilter namespace, which is also the constructor. Crossfilter uses semantic versioning. You can find the current version as crossfilter.version, which is a string of the form "X.Y.Z", where X is the major version number, Y is the minor version number, and Z is the patch version number.

Crossfilter

A crossfilter represents a multi-dimensional dataset.

# crossfilter([records])

Constructs a new crossfilter. If records is specified, simultaneously adds the specified records. Records can be any array of JavaScript objects or primitives. For example, you might construct a small crossfilter of payments like so:

var payments = crossfilter([
  {date: "2011-11-14T16:17:54Z", quantity: 2, total: 190, tip: 100, type: "tab", productIDs:["001"]},
  {date: "2011-11-14T16:20:19Z", quantity: 2, total: 190, tip: 100, type: "tab", productIDs:["001", "005"]},
  {date: "2011-11-14T16:28:54Z", quantity: 1, total: 300, tip: 200, type: "visa", productIDs:["004" ,"005"]},
  {date: "2011-11-14T16:30:43Z", quantity: 2, total: 90, tip: 0, type: "tab", productIDs:["001", "002"]},
  {date: "2011-11-14T16:48:46Z", quantity: 2, total: 90, tip: 0, type: "tab", productIDs:["005"]},
  {date: "2011-11-14T16:53:41Z", quantity: 2, total: 90, tip: 0, type: "tab", productIDs:["001", "004" ,"005"]},
  {date: "2011-11-14T16:54:06Z", quantity: 1, total: 100, tip: 0, type: "cash", productIDs:["001", "002", "003", "004" ,"005"]},
  {date: "2011-11-14T16:58:03Z", quantity: 2, total: 90, tip: 0, type: "tab", productIDs:["001"]},
  {date: "2011-11-14T17:07:21Z", quantity: 2, total: 90, tip: 0, type: "tab", productIDs:["004" ,"005"]},
  {date: "2011-11-14T17:22:59Z", quantity: 2, total: 90, tip: 0, type: "tab", productIDs:["001", "002", "004" ,"005"]},
  {date: "2011-11-14T17:25:45Z", quantity: 2, total: 200, tip: 0, type: "cash", productIDs:["002"]},
  {date: "2011-11-14T17:29:52Z", quantity: 1, total: 200, tip: 100, type: "visa", productIDs:["004"]}
]);

# crossfilter.add(records)

Adds the specified records to this crossfilter.

# crossfilter.remove(predicate)

Removes all records that match the current filters from this crossfilter.

New in 1.4.3

Takes an optional predicate function that should takes a record and index as arguments and returns a boolean value. For example:

data.remove(function (d,i) {
  return d.foo === 0;
});

# crossfilter.size()

Returns the number of records in the crossfilter, independent of any filters. For example, if you only added a single batch of records to the Crossfilter, this method would return records.length.

# crossfilter.all()

New in 1.4

Returns all of the raw records in the crossfilter, independent of any filters.

# crossfilter.allFiltered()

New in 1.4

Returns all of the raw records in the crossfilter, with filters applied.

# crossfilter.groupAll()

A convenience function for grouping all records and reducing to a single value. See groupAll for details. Note: unlike a dimension's groupAll, this grouping observes all current filters.

# crossfilter.onChange(callbackFunction)

New in 1.4

Calls callbackFunction when certain changes happen on the Crossfilter. Crossfilter will pass the event type to callbackFunction as one of the following strings:

  • dataAdded
  • dataRemoved
  • filtered

The return value of onChange is a function which when invoked will remove the event listener.

const disposeHandler = data.onChange(eventType => console.log('data changed:', eventType));
// ... then when done with the listener
disposeHandler();

# crossfilter.isElementFiltered(i, [ignoreDimensions])

New in 1.4

Check if the data record at index i is excluded by the current filter set. Can optionally ignore filters that are defined on an array of dimension ids [ignoreDimensions]. The ids are retrieved using dimension.id().

Dimension

# crossfilter.dimension(value [, isArray])

Constructs a new dimension using the specified value accessor function. The function must return naturally-ordered values, i.e., values that behave correctly with respect to JavaScript's <, <=, >= and > operators. Arrays are also supported via passing true as the optional isArray parameter (for more information see Dimensions with Arrays). This typically means primitives: booleans, numbers or strings; however, you can override object.valueOf to provide a comparable value from a given object, such as a Date.

In particular, note that incomparable values such as NaN and undefined are not supported. In addition, care should be taken when mixing types, e.g., strings and numbers. If strings and numbers are mixed, the strings will be coerced to numbers, and hence the strings must all be coercible to number, otherwise unsupported NaN values will result.

This means that if your value accessor function branches, it must always return a value (as the default JavaScript return value is undefined).

:warning: see Crossfilter Gotchas: Natural ordering of dimension and group values for details.

For example, to create a dimension by payment total:

var paymentsByTotal = payments.dimension(function(d) { return d.total; });

The value returned by a dimension's accessor function for a given record should be deterministic and never change for the existence of the crossfilter. Performance note: internally, the values for a given dimension are cached. Therefore, if the dimension value is derived from other properties, it is not necessary to cache derived values outside of the crossfilter. The value function is only called when records are added to the Crossfilter.

Dimensions are bound to the crossfilter once created. Creating more than 8 dimensions, and more than 16 or 32 dimensions, introduces additional overhead. But dimensions may be disposed of using dimension.dispose to make room for new dimensions. Dimensions are stateful, recording the associated dimension-specific filters, if any. Initially, no filters are applied to the dimension: all records are selected. Since creating dimensions is expensive, you should be careful to keep a reference to whatever dimensions you create.

# Dimensions with Arrays

Dimensions now support arrays for keys, to support scenarios where a row is associated with multiple keys. Supply true for the optional isArray parameter of the dimension constructor to enable this feature.

The value returned by the dimension's accessor must be an array of values that can be ordered naturally (as ordinary dimension keys are). Each item in the array is treated as its own accessor value. This causes the row to be indexed by each of the keys. A standard use case is tagging records with attributes and grouping/reducing on the tags.

In the example below, a dimension of productIDs is created for each product purchased in each record.

Example:

var paymentsByProductID = payments.dimension(function(d) { return d.productIDs; }, true);

// Filter to all transactions purchasing a particular product
paymentsByProductID.filter("004");

// Group by product and sum total quantity. Group keys are product IDs.
var quantityGroupByProduct = paymentsByProductID.group().reduceSum(function(d) { return d.quantity; });

// Retrieve top result (as usual, unaffected by filter on associated dimension)
quantityGroupByProduct.top(1)
// { key: "001", value: 13 }

It is extremely important to note that due to the many-to-one relationships created when using array accessors, all methods pertaining to that dimension will return and utilize individual records multiple times.

Example:

paymentsByTotal.top(Infinity).length === 12
paymentsByProductID.top(Infinity).length === 25

# dimension.filter(value)

Filters records such that this dimension's value matches value, and returns this dimension.

:warning: see Crossfilter Gotchas: dimension filters don't return the current values for details.

The specified value may be null, in which case this method is equivalent to filterAll; or, value may be an array, in which case this method is equivalent to filterRange; or, value may be a function, in which case this method is equivalent to filterFunction; otherwise, this method is equivalent to filterExact.

For example:

paymentsByTotal.filter([100, 200]); // selects payments whose total is between 100 and 200
paymentsByTotal.filter(120); // selects payments whose total equals 120
paymentsByTotal.filter(function(d) { return d % 2; }); // selects payments whose total is odd
paymentsByTotal.filter(null); // selects all payments

Calling filter replaces the existing filter for this dimension, if any.

# dimension.filterExact(value)

Filters records such that this dimension's value equals value, and returns this dimension. For example:

paymentsByTotal.filterExact(120); // selects payments whose total equals 120

Note that exact comparisons are performed using the ordering operators (<, <=, >=, >). For example, if you pass an exact value of null, this is equivalent to 0; filtering does not use the == or === operator.

Calling filterExact replaces the existing filter on this dimension, if any.

# dimension.filterRange(range)

Filters records such that this dimension's value is greater than or equal to range[0], and less than range[1], returning this dimension.

:warning: see Crossfilter Gotchas: filterRange does not include the top point for details.

For example:

paymentsByTotal.filterRange([100, 200]); // selects payments whose total is between 100 and 200

Calling filterRange replaces the existing filter on this dimension, if any.

# dimension.filterFunction(function)

Filters records such that the specified function returns truthy when called with this dimension's value, and returns this dimension. For example:

paymentsByTotal.filterFunction(function(d) { return d % 2; }); // selects payments whose total is odd

This can be used to implement a UNION filter, e.g.

// Selects payments whose total is between 0 and 10 or 20 and 30:
paymentsByTotal.filterFunction(function(d) { return 0 <= d && d < 10 || 20 <= d && d < 30; });

# dimension.filterAll()

Clears any filters on this dimension, selecting all records and returning this dimension. For example:

paymentsByTotal.filterAll(); // selects all payments

# dimension.top(k, offset)

New in 1.4: offset

Returns a new array containing the top k records, according to the natural order of this dimension. The returned array is sorted by descending natural order. This method intersects the crossfilter's current filters, returning only records that satisfy every active filter (including this dimension's filter). Optionally, retrieve k records offset by offset (records offset - offset + k - 1). For example, to retrieve the top 4 payments by total:

var topPayments = paymentsByTotal.top(4); // the top four payments, by total
topPayments[0]; // the biggest payment
topPayments[1]; // the second-biggest payment
// etc.

If there are fewer than k records selected according to all of the crossfilter's filters, then an array smaller than k will be returned. For example, to retrieve all selected payments in descending order by total:

var allPayments = paymentsByTotal.top(Infinity);

# dimension.bottom(k, offset)

New in 1.4: offset

Returns a new array containing the bottom k records, according to the natural order of this dimension. The returned array is sorted by ascending natural order. This method intersects the crossfilter's current filters, returning only records that satisfy every active filter (including this dimension's filter). Optionally, retrieve k records offset by offset. For example, to retrieve the bottom 4 payments by total:

var bottomPayments = paymentsByTotal.bottom(4); // the bottom four payments, by total
bottomPayments[0]; // the smallest payment
bottomPayments[1]; // the second-smallest payment
// etc.

# dimension.accessor(record)

New in 1.4

Applies the dimension accessor (the value from crossfilter.dimension(value)) to record and returns the result.

# dimension.dispose()

Removes this dimension (and its groups) from its crossfilter. This frees up space for other dimensions to be added to this crossfilter.

# dimension.id()

New in 1.4

Returns the numeric id of the dimension. For use with crossfilter.isElementFiltered.

Group (Map-Reduce)

# dimension.group([groupValue])

Constructs a new grouping for the given dimension, according to the specified groupValue function, which takes a dimension value as input and returns the corresponding rounded value. The groupValue function is optional; if not specified, it defaults to the identity function. Like the value function, groupValue must return a naturally-ordered value; furthermore, this order must be consistent with the dimension's value function!

:warning: see Crossfilter Gotchas: Consistent dimension and group value functions for details.

For example, to count payments by dollar amount:

var paymentGroupsByTotal = paymentsByTotal.group(function(total) { return Math.floor(total / 100); });

By default, the group's reduce function will count the number of records per group. In addition, the groups will be ordered by count.

Note: a grouping intersects the crossfilter's current filters, except for the associated dimension's filter. Thus, group methods consider only records that satisfy every filter except this dimension's filter. So, if the crossfilter of payments is filtered by type and total, then group by total only observes the filter by type.

:warning: see Crossfilter Gotchas: A group does not observe its dimension's filters for details.

# group.size()

Returns the number of distinct values in the group, independent of any filters; the cardinality.

# group.reduce(add, remove, initial)

Specifies the reduce functions for this grouping, and returns this grouping. The default behavior, reduce by count, is implemented as follows:

function reduceAdd(p, v, nf) {
  return p + 1;
}

function reduceRemove(p, v, nf) {
  return p - 1;
}

function reduceInitial() {
  return 0;
}

To reduce by sum of total, you could change the add and remove functions as follows:

function reduceAdd(p, v) {
  return p + v.total;
}

function reduceRemove(p, v) {
  return p - v.total;
}

The remove function is needed in addition to add because the group reduces are updated incrementally as records are filtered; in some cases, it is necessary to remove records from a previously-computed group reduction. To work with many different attributes, you can build your add and remove functions in a Javascript closure.

The nf ("not filter") parameter to the reduceAdd and reduceRemove functions provides information about data lifecycle. It evaluates to true if the record is being added or removed because of crossfilter.add or crossfilter.remove operations (i.e it is being permanently added or removed) and to false if the record is being added or removed from the group because of a filter. Records are guaranteed to pass through the reduceAdd function on crossfilter.add, even if they are outside the current filter. In this case the record is added with nf = true and then removed with nf = false. This allows groups to reliably track 'unfiltered' values for comparison purposes (e.g. display both the filtered and unfiltered values of the group), or even to implement their own group-specific filtering independent of any existing filters (e.g. the sum of ages of all people except those named "Bob"). This parameter is in place as of Crossfilter version 2.0 (including alpha.03 and later releases).

# group.reduceCount()

A convenience method for setting the reduce functions to count records; returns this group.

# group.reduceSum(value)

A convenience method for setting the reduce functions to sum records using the specified value accessor function; returns this group. For example, to group payments by type and sum by total:

var paymentsByType = payments.dimension(function(d) { return d.type; }),
    paymentVolumeByType = paymentsByType.group().reduceSum(function(d) { return d.total; }),
    topTypes = paymentVolumeByType.top(1);
topTypes[0].key; // the top payment type (e.g., "tab")
topTypes[0].value; // the payment volume for that type (e.g., 900)

# group.order(orderValue)

Specifies the order value for computing the top-K groups. The default order is the identity function, which assumes that the reduction values are naturally-ordered (such as simple counts or sums). For example, to reduce both the count and sum simultaneously, and to order by sum:

function reduceAdd(p, v, nf) {
  ++p.count;
  p.total += v.total;
  return p;
}

function reduceRemove(p, v, nf) {
  --p.count;
  p.total -= v.total;
  return p;
}

function reduceInitial() {
  return {count: 0, total: 0};
}

function orderValue(p) {
  return p.total;
}

var topTotals = paymentVolumeByType.reduce(reduceAdd, reduceRemove, reduceInitial).order(orderValue).top(2);
topTotals[0].key;   // payment type with highest total (e.g., "tab")
topTotals[0].value; // reduced value for that type (e.g., {count:8, total:920})

This technique can likewise be used to compute the number of unique values in each group, by storing a map from value to count within each group's reduction, and removing the value from the map when the count reaches zero.

# group.orderNatural()

A convenience method for using natural order for reduce values. Returns this grouping.

# group.top(k)

Returns a new array containing the top k groups, according to the group order of the associated reduce value. The returned array is in descending order by reduce value. For example, to retrieve the top payment type by count:

var paymentsByType = payments.dimension(function(d) { return d.type; }),
    paymentCountByType = paymentsByType.group(),
    topTypes = paymentCountByType.top(1);
topTypes[0].key; // the top payment type (e.g., "tab")
topTypes[0].value; // the count of payments of that type (e.g., 8)

If there are fewer than k groups according to all of the crossfilter's filters, then an array smaller than k will be returned. If there are fewer than k non-empty groups, this method may also return empty groups (those with zero selected records).

# group.all()

Returns the array of all groups, in ascending natural order by key. Like top, the returned objects contain key and value attributes. The returned array may also contain empty groups, whose value is the return value from the group's reduce initial function. For example, to count payments by type:

var types = paymentCountByType.all();

This method is faster than top(Infinity) because the entire group array is returned as-is rather than selecting into a new array and sorting. Do not modify the returned array!

# group.dispose()

Removes this group from its dimension. This group will no longer update when new filters are applied to the crossfilter, and it may be garbage collected if there are no other references to it remaining.

Group All (Reduce)

# dimension.groupAll()

A convenience function for grouping all records into a single group. The returned object is similar to a standard grouping, except it has no top or order methods. Instead, use value to retrieve the reduce value for all matching records.

Note: a grouping intersects the crossfilter's current filters, except for the associated dimension's filter. Thus, group methods consider only records that satisfy every filter except this dimension's filter. So, if the crossfilter of payments is filtered by type and total, then groupAll by total only observes the filter by type.

# groupAll.reduce(add, remove, initial)

Equivalent to reduce.

# groupAll.reduceCount()

Equivalent to reduceCount.

# groupAll.reduceSum(value)

Equivalent to reduceSum.

# groupAll.value()

Equivalent to all()[0].value.

Extras

Crossfilter has a few extra goodies that you might find useful.

# crossfilter.bisect

The identity bisector; suitable for numbers, dates, strings, and other naturally-comparable objects.

# crossfilter.bisect.by(value)

Constructs a new bisector using the specified value accessor function, which must return a naturally-ordered value. For example, to bisect an array of objects by their property foo, say:

var bisectByFoo = crossfilter.bisect.by(function(d) { return d.foo; });

The returned object is also the bisect.right function.

# bisect(array, value, lo, hi)
# bisect.right(array, value, lo, hi)

Similar to bisect.left, but returns an insertion point which comes after (to the right of) any existing entries of value in array. The returned insertion point i partitions the array into two halves so that all v <= value for v in array.slice(lo, i) for the left side and all v > value for v in array.slice(i, hi) for the right side.

# bisect.left(array, value, lo, hi)

Locate the insertion point for value in array to maintain sorted order. The arguments lo and hi specify a subset of the array which should be considered; to search the entire array, use 0 and array.length, respectively. If value is already present in array, the insertion point will be before (to the left of) any existing entries. The return value is suitable for use as the first argument to splice assuming that array is already sorted. The returned insertion point i partitions the array into two halves so that all v < value for v in array.slice(lo, i) for the left side and all v >= value for v in array.slice(i, hi) for the right side.

# crossfilter.heap

The identity heap function; suitable for numbers, dates, strings, and other naturally-comparable objects.

# crossfilter.heap.by(value)

Constructs a new heap function using the specified value accessor function, which must return a naturally-ordered value. For example, to create a heap function for objects based on their property foo, say:

var heapByFoo = crossfilter.heap.by(function(d) { return d.foo; });

The returned object is a heap function.

# heap(array, lo, hi)

Reorders the specified subset of the array into a binary heap; the lower bound lo is an inclusive index, and the upper bound hi is an exclusive index. To convert the entire array into a heap, specify a lo of 0 and a hi of array.length.

# heap.sort(array, lo, hi)

Sorts the subset of the specified array, which must be a binary heap, in descending order; the lower bound lo is an inclusive index, and the upper bound hi is an exclusive index. To sort the entire heap, specify a lo of 0 and a hi of array.length.

# crossfilter.heapselect

The identity heapselect function; suitable for numbers, dates, strings, and other naturally-comparable objects.

# crossfilter.heapselect.by(value)

Constructs a new heapselect function using the specified value accessor function, which must return a naturally-ordered value. For example, to create a heapselect function for objects based on their property foo, say:

var heapselectByFoo = crossfilter.heapselect.by(function(d) { return d.foo; });

The returned object is a heapselect function.

# heapselect(array, lo, hi, k)

Selects from the specified subset of the array, returning a new array containing the top k elements; the lower bound lo is an inclusive index, and the upper bound hi is an exclusive index. To select from the entire array, specify a lo of 0 and a hi of array.length.

# crossfilter.insertionsort

The identity insertionsort function; suitable for numbers, dates, strings, and other naturally-comparable objects. Note: you probably want to use quicksort instead.

# insertionsort(array, lo, hi)

Sorts the specified subset of the array in-place, returning the array; the lower bound lo is an inclusive index, and the upper bound hi is an exclusive index. To sort the entire array, specify a lo of 0 and a hi of array.length.

# crossfilter.insertionsort.by(accessor)

Constructs a new insertionsort function using the specified value accessor function, which must return a naturally-ordered value. For example, to create a insertionsort function for objects based on their property foo, say:

var sortByFoo = crossfilter.insertionsort.by(function(d) { return d.foo; });

The returned object is an insertionsort function.

# crossfilter.quicksort

The identity quicksort function; suitable for numbers, dates, strings, and other naturally-comparable objects. This implementation uses Vladimir Yaroslavskiy’s dual-pivot quicksort algorithm, and switches to insertionsort for small partitions.

# quicksort(array, lo, hi)

Sorts the specified subset of the array in-place, returning the array; the lower bound lo is an inclusive index, and the upper bound hi is an exclusive index. To sort the entire array, specify a lo of 0 and a hi of array.length.

# crossfilter.quicksort.by(accessor)

Constructs a new quicksort function using the specified value accessor function, which must return a naturally-ordered value. For example, to create a quicksort function for objects based on their property foo, say:

var sortByFoo = crossfilter.quicksort.by(function(d) { return d.foo; });

The returned object is a quicksort function.

# crossfilter.permute(array, index)

Returns a permutation of the specified array using the specified index. The returned array contains the corresponding element in array for each index in index, in order. For example, permute(["a", "b", "c"], [1, 2, 0]) returns ["b", "c", "a"]. It is acceptable for the array and index to be different lengths, and for indexes to be duplicated or omitted.

Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.