For lightblue impl document events, store processingDate or expirationDate?

[alechenninger]

In the event handler, we have event processing expiration. This is because once an event starts processing, a node could die. We need a way to recover. We do this using timeouts, which is not ideal because in rare cases it can result in duplicate events, and it can take a while before recovery happens, but it works, and is simple.

There are two ways we could persist timestamps when determining if an event has timed out.

1. Persist the time it started processing, and in config search for events whose processing timestamp is X duration in the past (expired).
2. Persist the time that after which the event will be expired. Then search for events whose expired date is before $now.

They both have pro/con.

The pro to option 1 is that you have knowledge about when an event started processing, and updates to timeout configurations affect already processing events.

The pro to option 2 is that you can search for what events are going to be treated as expired (which should be picked up/reprocessed shortly) without looking at current config values. However, this means updates to timeout config will not affect events which already started processing (they will still expire after whatever timeout was configured when they started processing).

Currently option 1 is implemented.

[alechenninger]

A 3rd option is to store both.