Events published in same millisecond aren't seen by clients

[concreted]

When two events are published to the LongpollManager in the same millisecond, the behavior I observed is that clients will receive the first event only. The other events in the same millisecond can be seen by trying a request again with the same since_time, but if clients listen again using the timestamp received (as advised in https://github.com/jcuga/golongpoll#http-subscription-handler), they will never see those events.

For example, here is sample output from a client listening to a simple golongpoll app that is hardcoded to publish two events in quick succession every 10s:

# Initial call
> curl -i "localhost:8081/events?category=connectivity&timeout=60&since_time=1588816900233"
HTTP/1.1 200 OK
Cache-Control: no-cache, no-store, must-revalidate
Content-Type: application/json
Expires: 0
Pragma: no-cache
Date: Thu, 07 May 2020 02:01:50 GMT
Content-Length: 97

{"events":[{"timestamp":1588816910237,"category":"connectivity","data":{"message":"message 1"}}]}

# If calling again with the same since_time, I see the event with duplicate timestamp:
> curl -i "localhost:8081/events?category=connectivity&timeout=60&since_time=1588816900233"
HTTP/1.1 200 OK
Cache-Control: no-cache, no-store, must-revalidate
Content-Type: application/json
Expires: 0
Pragma: no-cache
Date: Thu, 07 May 2020 02:01:51 GMT
Content-Length: 182

{"events":[{"timestamp":1588816910237,"category":"connectivity","data":{"message":"message 1"}},{"timestamp":1588816910237,"category":"connectivity","data":{"message":"message 2"}}]}                         

# If calling again with the received timestamp, the "message 2" message is never received.
> curl -i "localhost:8081/events?category=connectivity&timeout=60&since_time=1588816910237"
HTTP/1.1 200 OK
Cache-Control: no-cache, no-store, must-revalidate
Content-Type: application/json
Expires: 0
Pragma: no-cache
Date: Thu, 07 May 2020 02:02:00 GMT
Content-Length: 97

{"events":[{"timestamp":1588816920241,"category":"connectivity","data":{"message":"message 1"}}]}

Is this behavior expected? I am planning to work around this by forcing a 1ms delay between publishes, but perhaps this could be handled in the LongpollManager by aggregating all events in the same millisecond and sending them together?

[jcuga]

@concreted this is absolutely a bug. The real fix would be to use a counter instead of timestamp to pick up where the previous longpoll results left off, but that would be a breaking change to any existing clients.

I will see if I can come up with a fix that still allows clients to use the since_time query parameter that doesn't require major refactoring.

I plan on revisiting this library and making a new version addressing all of the feedback I've received thus far, so if I can't find an easy fix for this now, then it will be addressed in the next major version.

Thanks for reporting this.

[jcuga]

Fixed in v1.3.0

[jcuga]

The fix requires using the new last_id subscription request parameter (documented in HttpLongPollAPI.md).

The updated golang client and new js-client already use this new param Only custom clients will need to update to use this to get the fixed behavior.