Compute multiple float aggregations in one go #12547
Conversation
|
|
||
| void initializeValueCounters() { | ||
| if (values == null) { | ||
| values = new float[aggregationFunctions.size()][taxoReader.getSize()]; |
There was a problem hiding this comment.
this seems like a scary amount of RAM we could end up requiring. Are we sure that all the labels in taxoReader will have nonzero values? I wonder if we ought to switch to a sparse approach?
There was a problem hiding this comment.
This is a great point. IntTaxonomyFacets has the ability to choose sparse values if the taxonomy is large and there aren't a lot of hits. We can have the same functionality in FloatTaxonomyFacets. This was also mentioned recently in another issue, which puts into question the way we decide between sparse and dense values.
Fundamentally, I think the user of this feature will have to decide if they can make the space for time tradeoff for computing multiple aggregations.
There was a problem hiding this comment.
Hmm, you mean FloatTaxonomyFacets today is never sparse in its aggregation?
There was a problem hiding this comment.
That is correct. Compare initializeValueCounters for IntTaxonomyFacets and FloatTaxonomyFacets. I don't think there's a good reason for Int/FloatTaxonomyFacets to differ here. Maybe sparse values just never got implemented for FloatTaxonomyFacets.
There was a problem hiding this comment.
Maybe open a spinoff to implement sparse values for FloatTaxonomyFacets? But let's not block this otherwise great change?
|
This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the dev@lucene.apache.org list. Thank you for your contribution! |
|
This PR has not had activity in the past 2 weeks, labeling it as stale. If the PR is waiting for review, notify the dev@lucene.apache.org list. Thank you for your contribution! |
Usually facets maintain a one-dimensional array indexed by ordinal which keeps the values they're supposed to compute.
The change here is simple in principle - use a two-dimensional array, indexed by aggregation and ordinal, so we can do multiple aggregations at once.
For the methods that get top children / all children / spcific values / dims, the default is to get the values corresponding to the first aggregation, but the aggregation can be specified.
There is one tricky bit when we aggregate using provided values sources. In this case, we advance the values sources and get the values for each aggregation before iterating through the ordinals in each doc. When we iterate through the ordinals, we load each of the values we've already retrieved and update the corresponding accumulator.
Addresses #12546