Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

EventFilterManager needs to deal in epochs #11680

Open
rvagg opened this issue Mar 5, 2024 · 1 comment
Open

EventFilterManager needs to deal in epochs #11680

rvagg opened this issue Mar 5, 2024 · 1 comment

Comments

@rvagg
Copy link
Member

rvagg commented Mar 5, 2024

ToHeight should lead to a stop-condition, or at least be communicated to a consumer that a stop-condition has been reached. When subscribing, you should be able to receive a signal that there are no more events because you've reached the maximum height and there won't be anymore. See SubscribeActorEvents internals for the juggling needed to work around this.

Stretch goal: organise the API by epoch: batch them up such that you get all events for a particular epoch in one call, that way you can make some smarter consumption decisions on the outside and maybe even offer APIs that have this information in them.

Currently there's no way to know precisely how many events are in an epoch; even if you do a GetActorEvents there's a maximum number of events we'll give you according to config option and you have no insight. Perhaps a better API would give you a block, { "height": 12345, "totalEvents": 56, "matchedEvents": 45, "events": [ ... ] } ?

@rvagg
Copy link
Member Author

rvagg commented Mar 7, 2024

@masih FYI this is related to events index, when you have your head in there notice how there's no easy stop condition, you just install a filter and it gets applied until you remove it .. but how do you know when to remove it? You have to watch from outside. EventFilterManager knows both the current height and maxHeight of a filter. Unfortunately reverts make this complicated, I'm not clear on how to handle that. Maybe it's entirely appropriate to handle this outside because you can't assume you won't be rolling back and forward again?

The related concern is that events are already grouped by tipset - we load all messages then their receipts for an entire tipset and process them in a batch in CollectEvents but we get them one at a time with the subscription channel or all matching for a filter with TakeCollectedEvents (check out how maxResults messes things up too). Perhaps if we rearranged this whole thing to chunk by tipset then we might be able to have better downstream handling?

I'm not sure how this issue should get resolved, for now I think it's more of a bookmark of a set of concerns that would be nice to deal with somehow in the guts of this thing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: No status
Development

No branches or pull requests

1 participant