Retrieve facet values sorted by the number of corresponding documents

[gmourier]

Currently, facet values are returned in ascending lexicographical order according to their names.

It may be interesting to have the possibility to sort them by the number of corresponding documents.

Example (a subset of a search response body):

{
    "facetDistribution": {
        "genres": {
            "Action": 3,
            "Animation": 2,
            "Comedy": 5,
            "Fantasy": 18
        }
}

could be retrieved this way (ideally based on a parameter at the index setting level or at search time)

{
    "facetDistribution": {
        "genres": {
            "Fantasy": 18
            "Comedy": 5,
            "Action": 3,
            "Animation": 2
       }
}

Since there is a limit to the number of values per facet that can be returned (although customizable via this index setting), the default lexicographical order does not allow to return the facets containing the most documents if that facet value is beyond the limit.

[raustin-m]

Since there is a limit to the number of values per facet that can be returned (although customizable via this index setting), the default lexicographical order does not allow to return the facets containing the most documents if that facet value is beyond the limit.

This is the biggest problem with the current approach IMO. When I want to give the user the ability to narrow down their current search using facets, I think it's a very common use case to show them the facets with the highest count first. That list should be stable, and if by chance (due to the limit on number of facets that can be returned and the current ordering) I don't get back some of the highest count results, the list of offered filters becomes unstable and seems to change randomly, which is not a great UX.

[Kerollmops]

Hello everyone 👋,

Here is the current prototype you can use to test the new sortFacetValuesBy search parameter to define the way you want to return your facet values.

How to run the prototype?

You need to start from a fresh new database (remove the previously used data.ms) and use the following Docker image:

docker run -it --rm -p 7700:7700 -v $(pwd)/meili_data:/meili_data getmeili/meilisearch:prototype-sort-facet-values-by-count-0

How to use it?

You can use the feature this way or look at the original PR for more info.

Send some documents to Meilisearch

cat movies.json | xh 'localhost:7700/indexes/movies/documents' 'content-type:application/json'

Define the attributes you want to filter on

echo '{ "filterableAttributes": ["genres"] }' | xh PATCH 'localhost:7700/indexes/movies/settings'

Order Values in Alphanumeric Order

This is the default one. Values will be returned by lexicographic order, ascending from A to Z.

curl 'localhost:7700/indexes/movies/search?facets=genres&sortFacetValuesBy=alpha'

{
    "hits": [
        /* list of results */
    ],
    "query": "",
    "processingTimeMs": 0,
    "limit": 20,
    "offset": 0,
    "estimatedTotalHits": 1000,
    "facetDistribution": {
        "genres": {
            "Action": 3215,
            "Adventure": 1972,
            "Animation": 1577,
            "Comedy": 5883,
            "Crime": 1808,
            // ...
        }
    },
    "facetStats": {}
}

Order Values in Count Order

Facet values are sorted by decreasing count. The count is the number of records containing this facet value in the query results.

curl 'localhost:7700/indexes/movies/search?facets=genres&sortFacetValuesBy=count'

{
    "hits": [
        /* list of results */
    ],
    "query": "",
    "processingTimeMs": 0,
    "limit": 20,
    "offset": 0,
    "estimatedTotalHits": 1000,
    "facetDistribution": {
        "genres": {
            "Drama": 7337,
            "Comedy": 5883,
            "Action": 3215,
            "Thriller": 3189,
            "Romance": 2507,
            // ...
        }
    },
    "facetStats": {}
}

---

⚠️ We do NOT recommend using this prototype in production. This is only for test purposes.

Everyone is more than welcome to give feedback and to report any issue or bug you might encounter when using this prototype. Thanks in advance for your involvement. It means a lot to us ❤️

[mech]

Any chance the default in the future should be by count instead of alpha. I would think generally people prefer to see the most count first.

[gmourier]

Hey @mech 👋

Thanks for your feedback. We will consider that for a v2.0 since it's a breaking change. In the meantime, we will set up telemetry to determine the proportion of users changing the default ordering to make an informed decision.

[macraig]

Hello everyone 👋

We have just released the first RC (release candidate) of Meilisearch containing this new feature!
You can test it by using

• the release assets
• the Meilisearch Docker image

docker run -it --rm -p 7700:7700 -v $(pwd)/meili_data:/meili_data getmeili/meilisearch:v1.3.0-rc.0

You are welcome to leave your feedback in this discussion.

If you encounter any bugs, please report them here.
Thanks in advance for your help and your involvement in Meilisearch ❤️

🎉 The official and stable release containing this change will be available on July 31st, 2023

⚠️ RC (release candidates) are not recommended for production

[macraig]

Hey folks 👋

v1.3 has been released! 🦁 You can now search for facet values and sort them by count ✨

📚 https://www.meilisearch.com/docs/learn/fine_tuning_results/faceted_search