XReadGroup can't listen event from other languange, even from redis-cli

[dhymasriyanto]

So, I'm building a microservice app, then I use Laravel Streamer for my service's data transfer communication. Each service use different programming languange, such as PHP (Laravel), JavaScript (ExpressJS), Java (SpringBoot). When I try to listen data for same framework (in this case using Laravel with Laravel Streamer), the listener works. But when I try to publish a message from other service (JavaScript), the listener doesn't work. So I try to figure out with publish the data from redis-cli, but it's still same. Even the group name, and the stream name is correct. Is there a special way to use it? Cause I am new at this.

[prwnr]

@dhymasriyanto hello.
as far as I remember, when you are listening with group and consumer, having same group - but different consumer - won't get the message to both consumers. The first consumer in the group gets the message, the other consumers are there waiting for another message in case the first one is still busy -> this is Redis Streams behavior: https://redis.io/commands/xreadgroup/#consumer-groups-in-30-seconds

it would be best if you could test this out with different groups and see if the outcome changes.

the other thing to consider is that this package follows its own format of a message and it wont process message that wont match that format (it expects the Stream message to have at least data field in it

[dhymasriyanto]

Okay, thanks in advance, but sorry I know that if we listen with groups and consumers, having the same group, with different consumers won't get the message for both. So, in that GIF, I'm trying to send it twice, the first message will be listened to by the JavaScript listener, and the second one is supposed to be listened to by Laravel Stream.

This GIF below, I'm trying to send it from JS, which will send message (with the data field) three times (using looping) into two JS listener and one Laravel Stream listener, but the Laravel listener did not respond anything.

And GIF below I'm try to send from Laravel Event, which are all listener works.

[dhymasriyanto]

it would be best if you could test this out with different groups and see if the outcome changes.

I try to test with different group, but laravel listener still not respond anything

[prwnr]

@dhymasriyanto this looks for me like a format mismatch. The second gif proves that the message emitted by the package itself comply with its listener, but that message emitted by you in first gif does not.
Look, here is a sample of how the XADD looks like if the package emits it (sample from a project where I am using this package):

"XADD" "product.inventory.updated" "*" "_id" "*" "type" "event" "version" "1.3" "name" "product.inventory.updated" "domain" "Laravel" "created" "1655212111" "data" "{\"policy\":\"stop\",\"track\":true,\"is_available\":false,\"is_restockable\":true,\"quantity\":500,\"min_per_order\":0,\"max_per_order\":0,\"is_bundle_exclusive\":false,\"organisation\":1,\"store\":1,\"_meta\":{\"policy\":{\"resource\":\"Catalog\\\\Product\",\"own_by\":null},\"tenancy\":{\"organisation\":\"purdy.org\",\"store\":\"westcom\"}},\"id\":\"29ebd893-8605-4725-b9af-517a7694f62e\"}" "hash" "418afef69412e046797b581bed04af8bd3f3b5b465888995743b14d4dea1b80a"

so, from what I see based on your gifs (would be much better to paste text samples next time, hard to get stuff out of moving gifs :D), is that you emit is something like this:

XADD item.restock * message '{"foo":"bar"}'

while it should be:

XADD item.restock * data '{"foo":"bar"}'

The main content of the Stream should be inside "data" key.

I must admit that streamer should warn in some way that it received message without the "data" in it, instead of simply ignoring it. However it does log it inside storage/logs fiel as:

[2022-06-14 13:18:14] local.ERROR: Listener error. Failed processing message with ID 1655212670640-0 on 'item.restock' stream. Error: Undefined index: data  

so keep an eye on that file if something bad happens.

[dhymasriyanto]

Ah... I see. :D Thank you so much!!! Finally, I figure it out how it works.

while it should be:

XADD item.restock * data '{"foo":"bar"}'

The main content of the Stream should be inside "data" key.

And I just realized if there is a log there :D