facetsDistribution counts don't take into account filters

[lerouxrgd]

Describe the bug
The facetsDistribution counts don't take into account filters.

To Reproduce
Compare the facetsDistribution of a query with and without filters.

Expected behavior
The facetsDistribution counts should change based on the filters.

[MarinPostma]

Hello @lerouxrgd, we are aware of this. In fact there is already an open pr that tries to partially address this issue (#849).

There are also two fields in the search response (exhaustiveNbHits and exhaustiveFacetsCount), whose intent is to tell you if the returned count are approximate of not. Let me explain a bit further:
When you perform a search a search, a collection of relevant documents is lazily created from a set of candidates documents; that means that not all documents are procesed, but only enough to return you the number of documents you asked. That means that in most cases it is impossible to return an exact count, because that would mean we would have to process all the documents.
Now there are still some things that can be done, and that are not yet implemented:

• remove the filtered documents from the count, to return a better approximation of the number of documents (this is what Update nbHits count with filtered documents #849 does in part)
• when the number of candidates documents is small enough, process them all and return an exhaustive count (setting the relevant fields to true in the response)
• do some clever statistics on the database, and return even more accurate results (maybe in more distant future)

I hope this helps you better understand the what's going on under the hood.

[hadjedjvincent]

Hi @MarinPostma I’ve a question regarding the facets distributions. It looks like that the count for each criterions is missing except for the one that are currently selected.
Is it a known bug or is there something I have not configured correctly ?
It looks like after at least one criterion is selected, the group is being calculated as "AND" rules between criterions. Whereas most of the time, we want to do "OR" between criterions of the same facet, then "AND" between each facets.

I’ve tried latest release 0.13, also compiled from master (0.14) and also compiled using PR #849, the result is the same (i'm only talking about facets, not filters ;)

[MarinPostma]

Hello @hadjedjvincent, I am not completely sure to understand what you mean by "currently selected criterion", but the only thing that facet distribution does is returning an approximation of the number of documents that belong to each variant of the facet categories you used in your facet filter. If a document belongs to 2 categories, then it will be counted twice, one for each category.

[hadjedjvincent]

@MarinPostma

If a document belongs to 2 categories, then it will be counted twice, one for each category.

Yes, i perfectly understand this :)
BTW, this could the case for a hierarchical component, like a category tree.

However, if I want to look for all dresses that are available in Red or in Blue, it looks like it's not possible at the time.
When we are selecting the first Red facet, all others colors count are set to 0 in this facets.
This is why i'm talking about the OR and AND operators here. MeiliSearch facets are acting as "AND" between all available values in a facet.

[MarinPostma]

@hadjedjvincent thanks for reporting this bug, this is fixed with #944.

[hadjedjvincent]

@MarinPostma thank you for your fix. I've made a try, it works fine for the current facet.
However, for the others facets, the count does not reflect the current applied facets.

Following my previous example. If i select Red & Blue facets, i have all my dresses that have Red OR Blue color.
Now, i would want to filter by Size, only for Red & blue colors.
If i provide the right facets while searching (red or blue colors), the facets distribution is unchanged for any others Facets than "Color". Meaning that i have all sizes related to the search query, and not the search query + current facets.
Is that related to any other issue?

[lerouxrgd]

@hadjedjvincent I observed the same issue as you and it is related to the original issue I think, that is, regular filters are not taken into account for the facetsDistribution counts (whether you apply a facetFilter or not)

[MarinPostma]

@hadjedjvincent Can you open another issue, this is unrelated to this issue, we'll continue discussing there, thanks!

[qdequele]

@MarinPostma What is the status of this issue?

[jobygoude]

The correction of this bug is in progress on your side?
When do you hope to deliver a fix?

[curquiza]

Hello @jobygoude!

Sorry for the delay on this issue. The core team is currently working on a huge refacto of the core engine and does not spend time for the moment on improving features.
However we know there is an issue with this behavior and we will discuss the best behavior to adopt and fix it in the future.
Sorry again for the delay!

[bb]

I think this issue is solved when #1576 is closed, @curquiza.

[curquiza]

Hello @bb, thanks for making pop this issue in my notifications: I checked with @ManyTheFish, and this issue should have been closed when we merged the new search engine 😂 I completely missed it
The new search engine has a completely different behavior compared to v0.20.0 and this issue will be solved once v0.21.0 is out!
The fix he did with #1576 is also related, but is more a "finishing touch" 😄

So, closing this with #1421 🎉