0.21 meilisearch breaking changes

[nezaboravi]

• Scout Version: v9.2.8
• Scout Driver: MeiliSearch
• Laravel Version: 8.58.0
• PHP Version: 8.0.9
• Database Driver & Version: mySql 8.0.25 for macos11.3 on x86_64 (Homebrew)

Description:

Since last changes on meilisearch, filters are not working.
I have seen changes and updated code, so instead of 'filters' now is a 'filter' as a index

MeiliSearch\Exceptions\ApiException
Json deserialize error: unknown field `filters`, expected one of `q`, `offset`, `limit`, `attributesToRetrieve`, `attributesToCrop`, `cropLength`, `attributesToHighlight`, `matches`, `filter`, `facetsDistribution` at line 1 column 31

When I replace it with filter, I am getting error that my field cant be indexed, as its not indexable.

--> 1:1 | 1 | active=1 | ^----^ | = attribute `active` is not filterable, available filterable attributes are: (View: 

Meili search documentation says this:

"By default, filterableAttributes is empty. This means that filters do not work without first explicitly adding attributes to the filterableAttributes list."

I cant find a way to add attributes to filterableAttributes list. I suppose I can manually 'curl' update, but this just is not a solution. Anyone has idea how to make filterableAttributes?

Steps To Reproduce:

ad filter and and search using filter

Model::search(
            $query,
            function ($meilisearch, $query, $options) {
                // $options['filters'] = 'active=1'; // breaking change in meilisearch 0.21
                 // $options['filter'] = 'active=1'; // change it to filter, without s
                return $meilisearch->search($query, $options);
            }
        )
        ```

[driesvints]

@curquiza @georgeboot didn't we fix this with the latest PR's? Should we revert everything?

@nezaboravi what's your Meilisearch PHP version?

[driesvints]

I'm still finding some filters references in the source code in the search and paginate methods amongst else. Should we convert these?

[georgeboot]

Meilisearch doesn't support filters without fist specifying the filterable columns first, anymore. Before 0.21 they had filters and facet filters. Filters has been removed and facet filters is now named filter.

It is thus up to the user to add the columns to filter on, to their index settings, or to just simply not use the where function.

Not really anything Scout can do something about I think. Scout is only a driver, it doesn't take away the need to configure your index to your needs.

[driesvints]

In that case I think we shouldn't be updating to newer Meilisearch versions anymore on stable Scout releases. We can't have people's apps breaking when they run a simple composer update.

[georgeboot]

Also note that the changes done in the closure (2nd argument of search) are applied AFTER the internal rename from filters to filter. So that is full user-land code.

[georgeboot]

I'm still finding some filters references in the source code in the search and paginate methods amongst else. Should we convert these?

Internally, it's still called filters. Before sending it to the API, we rename if needed. So I think we should leave that as is.

[curquiza]

Hello!

Internally, it's still called filters. Before sending it to the API, we rename if needed. So I think we should leave that as is.

Indeed!
Cf

scout/src/Engines/MeiliSearchEngine.php

Lines 131 to 135 in 0db23cc

	if (version_compare(MeiliSearch::VERSION, '0.19.0') >= 0 && isset($searchParams['filters'])) {
	$searchParams['filter'] = $searchParams['filters'];
	unset($searchParams['filters']);
	}

I think the error can come from the meilisearch-php version.

FYI the detailed changelogs of MeiliSearch are described on GitHub Release and in this article. The users are aware they are migrating from a major version of MeiliSearch to another one and some changes have to be applied in their code.
As @georgeboot said, some of them cannot be anticipated by laravel scout 🙂

We can't have people's apps breaking when they run a simple composer update.

They did not only did a composer update but they also upgraded their MeiliSearch version from a major version to another. Regarding the changelogs on the MeilISearch side, they can expect some changes in their own code. These errors are not related to Laravel scout.

These errors are coming from MeiliSearch (meilisearch-php for the MeiliSearch\Exceptions\ApiException and MeiliSearch itself for the breaking changes). This kind of situation can also be solved with MeiliSearch support, not the laravel one, I think 🙂

@nezaboravi, if you need any help with using any new MeiliSearch version, feel free to ask questions

• on the Discussion section of the MeiliSearch repository
• on our Slack dedicated to the community. Feel free to join us! (Same for @georgeboot and @driesvints if you want to contact me directly for any question, I'm Clémentine on our Slack)

But yes, maybe only support major MeiliSearch version breaking on search side with a new stable version of Laravel Scout is the best solution.

[driesvints]

Thanks for the explanation @curquiza. I can agree that if the OP makes a conscience choice of upgrading their Meilisearch binary that they also need to review the Meilisearch changelog.

@nezaboravi I hope you understand from the above why this isn't a Scout issue and that you should always review the breaking changes of the binary.

I've sent in a PR to the docs for this: laravel/docs#7286

[curquiza]

Thanks for the PR to the docs!

[nezaboravi]

@driesvints Thank you all guys for the follow-ups.

Meilisearch doesn't support filters without fist specifying the filterable columns first, anymore. Before 0.21 they had filters and facet filters. Filters have been removed and facet filters is now named filter.

It is thus up to the user to add the columns to filter on, to their index settings, or to just simply not use the where function.

This exactly is a problem. In Meilisearch-php is a method on website documentation, where you can set filters.

$client->index('movies')->updateFilterableAttributes([
  'genres',
  'director'
]);

Cant you in scout make updateFilterableAttributes to be used with MeiliEngine?

@georgeboot how to use "where" or to make. changes in filter settings? I see an option to send Curl, but this is not really good solution to do manually. Prefer some scout method we can call and send it to meilie to set filterableAttributes.

ON meilisearch slack channel, I got asked if I have some settings in laravel for meili search. We don't have it.

Using where throws the same error:

Model::search(
            $query,
            function ($meilisearch, $query, $options) {
                // $options['filter'] = 'active=1'; // breaking change in meilisearch 0.21
                return $meilisearch->search($query, $options);
            }
        )->query(function ($builder) {
            $builder->with('business')
                    ->with('industry')
                    ->with('sector')
                    ->with('position')
                    ->with('country');
        })->where('active', 1);

THrows:

MeiliSearch\Exceptions\ApiException
Json deserialize error: unknown field `filters`, expected one of `q`, `offset`, `limit`, `attributesToRetrieve`, `attributesToCrop`, `cropLength`, `attributesToHighlight`, `matches`, `filter`, `facetsDistribution` at line 1 column 21