Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add experimental support for Symfony Messenger #16

Open
wants to merge 15 commits into
base: master
from

Conversation

@damienalexandre
Copy link
Member

damienalexandre commented Jan 9, 2020

Symfony Messenger is really simple to use. In this pull request I just add the basic of handling indexation request via Messenger.

There is a simple DTO, the provided handler must be overwritten to implement the fetchModel method, then I call the proper methods on the Indexer.

I use a very close version of this in production already 馃構

composer.json Outdated Show resolved Hide resolved
src/Messenger/IndexationRequest.php Outdated Show resolved Hide resolved
src/Messenger/IndexationRequest.php Outdated Show resolved Hide resolved
src/Messenger/IndexationRequestHandler.php Outdated Show resolved Hide resolved
src/Messenger/IndexationRequestHandler.php Outdated Show resolved Hide resolved
src/Messenger/IndexationRequestHandler.php Outdated Show resolved Hide resolved
src/Messenger/IndexationRequest.php Outdated Show resolved Hide resolved
$model = $this->fetchModel($message->getType(), $message->getId());

if (!$model) {
// ID does not exists, delete

This comment has been minimized.

Copy link
@lyrixx

This comment has been minimized.

Copy link
@damienalexandre

damienalexandre Jan 12, 2020

Author Member

To me, if fetchModel fail to found the entity/the data, that means it does not exists. If it did exists before, triggering a delete is a safe way to sync the fetchModel behavior with what we have in Elasticsearch.

This comment has been minimized.

Copy link
@lyrixx

lyrixx Jan 13, 2020

Member

But if the does not exist, if you delete it you basically delete nothing.

@damienalexandre

This comment has been minimized.

Copy link
Member Author

damienalexandre commented Jan 16, 2020

I just pushed a better version and the documentation.

There this a "MultipleIndexationRequest" message now, and a listener to group single messages in this one, allowing spool / bulk of all the changes into one message!

@damienalexandre

This comment has been minimized.

Copy link
Member Author

damienalexandre commented Jan 16, 2020

The pooling feature is tested successfully on a real world project 馃帀

README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
damienalexandre and others added 4 commits Jan 24, 2020
Fix typo
Co-Authored-By: Lo茂ck Piera <pyrech@gmail.com>
Co-Authored-By: Lo茂ck Piera <pyrech@gmail.com>
Copy link
Member

lyrixx left a comment

Really cool 馃憤

README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
composer.json Outdated Show resolved Hide resolved
src/Messenger/IndexationRequestHandler.php Outdated Show resolved Hide resolved
/**
* Return a model (DTO) to send for indexation.
*/
abstract public function fetchModel(string $className, string $id);

This comment has been minimized.

Copy link
@lyrixx

lyrixx Jan 24, 2020

Member
  1. You may add the return type Document (and so remove the PHP Doc
  2. It could be abstract

This comment has been minimized.

Copy link
@damienalexandre

damienalexandre Jan 24, 2020

Author Member

I see... Naming it "fetchDocument" instead. Why not 馃 Will look into that.

src/Messenger/IndexationRequestSpoolSubscriber.php Outdated Show resolved Hide resolved
src/Messenger/MultipleIndexationRequest.php Outdated Show resolved Hide resolved
$this->schedule($indexer, $message);
}

$indexer->flush();

This comment has been minimized.

Copy link
@renanbr

renanbr Jan 24, 2020

Could we check for errors, then requeue the MultipleIndexationRequest (or IndexationRequest) only with those that have failed?

This comment has been minimized.

Copy link
@damienalexandre

damienalexandre Jan 27, 2020

Author Member

With Messenger, any Exception is gonna re-queue the entire message.

In the case of Elasticsearch indexing, a bulk can fail only for one doc, and the other docs will be OK in the index (no rollback). I like the idea but it means we should edit the message that's being "re-queue"; I don't know how to do that yet :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
4 participants
You can鈥檛 perform that action at this time.