Facet counts with distinct=true or grouping (related to refinementList widget, but really a general Q)

[u3804]

Hello! My apologies in advance if this question should be asked elsewhere. The question I describe below came from my use of the refinementList widget, but from a few quick tests I think it also applies to the ruby API, dashboard and demo wizard.

I run searches with either distinct=true or grouping turned on (distinct=2..5). The filter counts appear to ignore the distinct setting. Two pictures are included below.

This doesn't seem like a bug -- having the raw result counts seems useful (e.g. using the example from the official documentation for distinct, it's still useful to know the # of Burger Kings even if the results will only include the N closest locations) -- but at the same time, it would be nice to have facet counts that reflect the distinct setting (so that users, when selecting a facet, don't have to guess what impact it the choice will have on the actual # of results they are shown -- and also so that the facet counts make sense without explanation in the context of the total # of hits shown).

If it's possible, I think it would be great if both sets of facet counts would be available. I'm trying to think of a way to calculate those numbers (additional indexed fields, brute force), but I haven't come up with anything practical.

Pictures showing facet counts > # of hits (for simplicity, I used screenshots of the dashboard and a demo wizard page):

Again, no worries if this is the wrong place -- or if obtaining the alternate facet counts with distinct=true isn't possible.

• UG

[vvo]

Hi @u3804 indeed you are currently experimenting a bug/limitation in our search engine. It's not a bug of instantsearch.js itself.

As soon as you are using DISTINCT then the number of hits is well computed but the facet values count are not.

We are tracking this issue in our engine and will add more comments here as we have them.

[redox]

As soon as you are using DISTINCT then the number of hits is well computed but the facet values count are not.

The exact reason is that the counts of the facets are computed BEFORE applying the distinct feature. You probably expected them to be computed AFTER. The thing is we need to apply them before in order to NOT miss any facet values: if record { objectID: "A", group_id: 42, facet: "a" } and record { objectID: "B", group_id: 42, facet: "b" } are grouped together, only one or the other will be in the hits but you still want to refine either in the facet a or facet b therefore their counts needs to be computed before (and will result in {a:1, b: 1}).

When combining grouping+faceting, we generally recommend to hide the counts, what do you think?

[u3804]

Thanks so much @vvo and @redox for those responses. That recommendation makes sense; so far I've been hiding the counts. As a potential solution (in case I still want the counts), in the next few days I'll see whether I'm able to calculate these numbers with a second search call + processing (and preprocessing) on my end. Will let you know how it turns out. @redox, thanks for the additional detail. I don't fully understand it yet, but I'm sure I will once I get a chance to focus on this (have been traveling, so I haven't had a chance to work through it with pen and paper :)).

Thanks again!

[vvo]

Great! Reopen the issue if needed, have fun with instantsearch.js

[abhalla-atl]

Hi, is there a public issue tracking this limitation in the Algolia engine?

[vvo]

Hi @abhalla-atl no there is no public tracking of the Algolia engine issues, sorry.

[ChrisMcKee]

https://www.algolia.com/doc/guides/search/distinct#example-de-duplicating-variants-of-products

Apparently hits are also affected (going off the warning in the docs)

[aramesh23]

Any update in facet count issue with distinct. Its been an year since this issue encountered and still looking for solution. Any tentative time of this issue resolution as we are planning to the change the design or solution.

[BryanSchuetz]

For anyone stumbling onto this issue like I did. You can now set facetingAfterDistinct: true in your search parameters to correct facet counts for de-duplication. Find out more in the Algolia documentation for distinct.

[daanvosdewael]

The documentation page @BryanSchuetz links to, does not clearly explain how to implement this with the instantsearch.js library. Nor does the SearchParameter section on the algoliasearch-helper-js reference page.

I've got it working with the following code:

const search = instantsearch({
  appId: '',
  apiKey: '',
  indexName: '',
  searchParameters: {
    facetingAfterDistinct: true,
  },
});

[soulshake]

In case this can help someone coming across this issue like I did, here's how to provide custom search parameters in React instantSearch in order to fix the incorrect facet counts in the refinement list:

<InstantSearch 
  // ...
>

  <Configure distinct={1} facetingAfterDistinct={true} />

</InstantSearch>