If QueueEventsOptions lastEventId is not set to `$`, it replays the full (all-time) event history on process restart

[jbr]

Possible solutions:

• Mark stream events as processed with XACK
• Set the default lastEventId to "$" instead of "0-0", only streaming new events. Downside of this approach: Events are ignored during QueueEvents listener unavailability

[stansv]

Hmm, I thought we've already replaced "0-0" to "$" (#29)..

[jbr]

@stansv https://github.com/taskforcesh/bullmq/blob/master/src/classes/queue-events.ts#L29 is 0-0 and I don’t see any other redis commands than the XREAD

---

update: I found

bullmq/src/classes/queue-scheduler.ts

Line 92 in 7eae653

this.client.xtrim(key, 'MAXLEN', '~', 100);

, but this still means that the last 100 events will always be replayed on process restart, which would only be appropriate for some uses of global events. I'm not super familiar with the streams api, but it seems like the intended use is to use XREADGROUP and XACK to consume events on behalf of a read group.

In my use case, replaying those events immediately in a loop on process restart actually blocks a server from binding to a port until those hundred events have been processed. It's probably a different Issue that event processing should be on a setTimeout(,0)/setImmediate/process.nextTick, not in a blocking loop in new QueueEvents(...)

[jbr]

Would a PR that uses XREADGROUP be welcome?

[manast]

@jbr the code above regarding XTRIM is for internal delayed jobs events, 100 is a extremely safe value so that we do not need to rely on watchdogs as we had in Bull3.

Regarding QueueEVents, the workaround is to pass '$' to only receive new events. I thought I had changed that already but for some reason is not the default value yet.