Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

->searchable() update everything to Algolia, even with no changes #226

elfeffe opened this issue Sep 28, 2017 · 6 comments

->searchable() update everything to Algolia, even with no changes #226

elfeffe opened this issue Sep 28, 2017 · 6 comments


Copy link

elfeffe commented Sep 28, 2017

I'm using Laravel 5.3 and MongoDB database.
If I run ->searchable() in my model, I resend everything to Algolia, even when there is no changes. Incurring in charges in Algolia.
Is that normal?
This is not supposed to update only if there is any change?
I'm trying this because in some cases my items are not being updated in Algolia when I save it in my database, I don't know if it's related to MongoDB or to Scout. In any case I think there should be a way to update the index completely but only when the document is different to your local one.

@elfeffe elfeffe changed the title ->searchable() update everything to Algolia, even whith no changes ->searchable() update everything to Algolia, even with no changes Sep 28, 2017
Copy link

julienbourdeau commented Sep 28, 2017

If you want to achieve that, you will need to keep track locally of what's in Algolia index and I guess it can become pretty tricky.

You might be able to do something based on the updated_at columns. When you do your query, maybe you can a where statement to push only recently updated entries.

Copy link

zanematthew commented Dec 1, 2017

By nature this is how Elasticsearch works.

documents are immutable: they cannot be changed, only replaced.

So if you run ->searchable() its not going to check if there are changes. For that, you'll want to write your own implementation following the partial updates guide.

Note the latest version of Elasticsearch 6.0 introduces breaking changes! They are removing support for _type, which means your models will no longer map to a type in Elasticsearch. You'll have to map them on your own.

Copy link

Gummibeer commented Jan 19, 2018

I think it would be an idea to use the isDirty() method in the models.
It should be done in the observer and handled Like the isEmpty call.
But I think that this isn't something package related cause it could bring in some problems - out of sync algolia.

But I've solved it the same - I check if there is any searchable value in dirty state, otherwise I ignore this model.
Atm it's working pretty well and I never had problems. But I also have to do all changes via eloquent or trigger manually an algolia sync.

Copy link

timramseyjr commented Mar 26, 2018

I might be missing something, but is there a reason the update() method of AlgoliaEngine.php couldn't first check isDirty() and toSearchableArray()? Something like:

public function update($models)
	$models = $models->filter(function($model){
	   return $model->isDirty() && !empty($model->toSearchableArray());
	if ($models->isEmpty()) {

	$index = $this->algolia->initIndex($models->first()->searchableAs());

and then isEmpty() would return and there would be no API calls?

Or at least have a configuration option to enable/disable the above logic?

Edit: After reading all the comments, I see where this could cause algolia to be out of sync(I personally would rather have this than all the API calls), so I will just end up doing this in a custom engine.

Copy link

elfeffe commented May 15, 2018

I have created a custom engine for it
Works good, Algolia don’t charge per query, only indexing is charged.
It is designed for MongoDB, but should work with other DB I think.

Copy link

driesvints commented Oct 5, 2018

Gonna close this for the same reasons as in #285 and mentioned above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
None yet

No branches or pull requests

6 participants