fetcher: provide api to fetch objects without validating if they are available in db #4927
Labels
Comments
|
|
bors bot
pushed a commit
that referenced
this issue
Sep 1, 2023
related: #4927 this change eliminates database lookups on ballot validation path, with an exception of when lru cache is not sufficient to hold vrf nonces and activations. in that case we will have to load them from db, that can be optimized later by fitting more in lru cache #4935 or reusing tortoise dataset as well - fetcher will be asked for ballots only if we don't have them in memory - reference ballots for eligibility validation will be fetched from tortoise
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
existing api always tries to lookup if object exists in the database, this is both inefficient and not robust.
as an example ballot validation
in fact all this objects are available in memory in the tortoise, so there is absolutely no reason to waste resources on doing this database lookups.
another concern is that this is not robust, if tortoise doesn't have dependency when dependant object is processed - it will fail validation and it will retry hopefully in some future.
to address both of those concern component that maintain state should be able to provide information about missing objects (for example as an error), and fetcher should do request based on missing object, regardless of that is stored in the database. after fetching complete validation should be re-done. in other words it should be up to handler to check that data is available, some handlers may do it using database, but in some places we already have all necessary data in memory.
@countvonzero
The text was updated successfully, but these errors were encountered: