-
Notifications
You must be signed in to change notification settings - Fork 388
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
fix: Run validateOrRevoke only if signer is changed #1695
Conversation
🦋 Changeset detectedLatest commit: d5ed452 The changes in this PR will be included in the next version bump. This PR includes changesets to release 3 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #1695 +/- ##
==========================================
+ Coverage 73.98% 74.57% +0.59%
==========================================
Files 99 99
Lines 9263 9309 +46
Branches 2082 2090 +8
==========================================
+ Hits 6853 6942 +89
+ Misses 2289 2254 -35
+ Partials 121 113 -8 ☔ View full report in Codecov by Sentry. |
const lastJobTimestamp = hubState.validateOrRevokeState?.lastJobTimestamp ?? 0; | ||
const lastFid = hubState.validateOrRevokeState?.lastFid ?? 0; | ||
|
||
log.info({ lastJobTimestamp, lastFid, numFids }, "ValidateOrRevokeMessagesJob: starting"); | ||
|
||
for (let i = 0; i < numFids; i++) { | ||
const fid = allFids[i] as number; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we still need to read all fids into memory?
} | ||
const numChecked = await this.doJobForFid(lastJobTimestamp, fid); | ||
// We'll alwats check for username proof messages | ||
const numUsernamesChecked = await this.doUsernamesJobForFid(fid); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Most users won't have this. Checking the storage cache message count for the UserNameProof messages will save db reads here.
|
||
do { | ||
// First, find if any signers have changed since the last time the job ran | ||
const signers = await this._engine.getOnChainSignersByFid(fid); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm, too bad the on chain event has fid before the timestamp, otherwise we could scan all onchain events in one go after a particular block number.
But maybe it's still cheaper to fetch the onchain events by block number than iterate through all fids/onchain events?
Anyway, not a huge deal. This is still a significant improvment from what it used to do.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll look into this in a new PR. But yeah, this new way is like much faster. It now takes only an hour or so (depending on signers changed) vs ~12 hours before.
* fix: Run validateOrRevoke only if signer is changed * cleanup * timebug * review
* fix: Run validateOrRevoke only if signer is changed * cleanup * timebug * review
Motivation
The validateOrRevokeMessages job takes ~12 hours to run everyday. It is very inefficient because it checks every single message.
Change Summary
Merge Checklist
Choose all relevant options below by adding an
x
now or at any time before submitting for reviewPR-Codex overview
This PR focuses on adding the
validateOrRevokeState
field to theHubState
message and implementing theValidateOrRevokeMessagesJobScheduler
class.Detailed summary
validateOrRevokeState
field toHubState
message inhub_state.proto
andhub_state.ts
ValidateOrRevokeJobState
message inhub_state.ts
createBaseValidateOrRevokeJobState
function inhub_state.ts
ValidateOrRevokeJobState
inhub_state.ts
doJobForFid
anddoJobs
methods toValidateOrRevokeMessagesJobScheduler
invalidateOrRevokeMessagesJob.ts
validateOrRevokeMessagesJob.test.ts