Read from a Farcaster Hub instead of the Warpcast API #15
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
gskril
changed the title
error creating a signer with `EthersEip712Signer`, patch coming tomorrow
* signers * Add storage and registrations * Refactor code to use MAX_PAGE_SIZE constant * Refactor getOnChainEventsByFidInBatchesOf function * Update page size constant to use MAX_PAGE_SIZE * Minor cleanup * Remove unused import * Patches * Patch date logic * Fix fid registration time, delete unused table --------- Co-authored-by: Greg Skriloff <35093316+gskril@users.noreply.github.com>
Co-authored-by: Greg <35093316+gskril@users.noreply.github.com>
Redis consumes a ton of memory with this method, but is too slow with just storing event ids
Batches caused Redis memory issues. This change will put more stress on Postgres, so we should look into batching on the worker side https://docs.bullmq.io/bullmq-pro/batches
It's faster to first request cast hashes from the `casts` table, then follow up to get full cast metadata from `casts_enhanced`
It's faster to call the function directly for some reason
Encoding is to try and improve Redis memory consumption. Batch size is so we can more easily see which FIDs are throwing errors during backfill, if any
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resources: