Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Persist websocket cursor and resume upon restart #66

Closed
strideynet opened this issue Jul 25, 2023 · 0 comments
Closed

Persist websocket cursor and resume upon restart #66

strideynet opened this issue Jul 25, 2023 · 0 comments
Assignees
@itstolf
Labels
backend enhancement New feature or request performance

Comments

@strideynet
Copy link
Owner

Right now we lose events during restarts or outages, let's persist the cursor and resume from last known position.

Potentially need to persist the cursor every X seconds (5 ?) to avoid hammering the database - the cursor can be a bit behind as long as we ensure we gracefully handle duplicate inserts.
@strideynet strideynet added the enhancement New feature or request label Jul 25, 2023
itstolf added a commit that referenced this issue Aug 1, 2023
@itstolf
persist firehose cursor. closes #66
0479eb8 
currently the cursor is saved every event to the database: i'm not sure how expensive this is, but we'll see :')
itstolf added a commit that referenced this issue Aug 1, 2023
@itstolf
persist firehose cursor. closes #66
c9dcfc0 
currently the cursor is saved every event to the database: i'm not sure how expensive this is, but we'll see :')
itstolf added a commit that referenced this issue Aug 1, 2023
@itstolf
persist firehose cursor. closes #66
a799d54 
currently the cursor is saved every event to the database: i'm not sure how expensive this is, but we'll see :')
itstolf added a commit that referenced this issue Aug 1, 2023
@itstolf
persist firehose cursor. closes #66
1c3f61d 
currently the cursor is saved every event to the database: i'm not sure how expensive this is, but we'll see :')
itstolf added a commit that referenced this issue Aug 1, 2023
@itstolf
persist firehose cursor. closes #66
e9d15ee 
currently the cursor is saved every event to the database: i'm not sure how expensive this is, but we'll see :')
itstolf added a commit that referenced this issue Aug 1, 2023
@itstolf
persist firehose cursor. closes #66
e5dd0ef 
currently the cursor is saved every event to the database: i'm not sure how expensive this is, but we'll see :')
itstolf added a commit that referenced this issue Aug 1, 2023
@itstolf
persist firehose cursor. closes #66
95d9863 
currently the cursor is saved every event to the database: i'm not sure how expensive this is, but we'll see :')
itstolf added a commit that referenced this issue Aug 1, 2023
@itstolf
persist firehose cursor. closes #66
fc476f7 
currently the cursor is saved every event to the database: i'm not sure how expensive this is, but we'll see :')
itstolf added a commit that referenced this issue Aug 2, 2023
@itstolf
persist firehose cursor. closes #66
43956f7 
currently the cursor is saved every event to the database: i'm not sure how expensive this is, but we'll see :')
itstolf added a commit that referenced this issue Aug 2, 2023
@itstolf
persist firehose cursor. closes #66
3c2b83f 
currently the cursor is saved every event to the database: i'm not sure how expensive this is, but we'll see :')
itstolf added a commit that referenced this issue Aug 9, 2023
@itstolf
persist firehose cursor. closes #66
e6bcc00 
currently the cursor is saved every event to the database: i'm not sure how expensive this is, but we'll see :')
itstolf added a commit that referenced this issue Aug 9, 2023
@itstolf
ersist firehose cursor. closes #66
bea773d 
currently the cursor is saved every event to the database: i'm not sure how expensive this is, but we'll see :')
itstolf added a commit that referenced this issue Aug 9, 2023
@itstolf
ersist firehose cursor. closes #66
7bdc399 
currently the cursor is saved every event to the database: i'm not sure how expensive this is, but we'll see :')
itstolf added a commit that referenced this issue Aug 10, 2023
@itstolf
persist firehose cursor. closes #66
05cd910
@itstolf itstolf self-assigned this Aug 12, 2023
strideynet added a commit that referenced this issue Aug 13, 2023
@strideynet
Merge pull request #97 from strideynet/itstolf/persist-firehose-cursor
13e3df1 
persist firehose cursor. closes #66
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@itstolf itstolf

Labels
backend enhancement New feature or request performance
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@strideynet @KevSlashNull @itstolf