fix: Run validateOrRevoke only if signer is changed #1695
Conversation
🦋 Changeset detectedLatest commit: d5ed452 The changes in this PR will be included in the next version bump. This PR includes changesets to release 3 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
|
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #1695 +/- ##
==========================================
+ Coverage 73.98% 74.57% +0.59%
==========================================
Files 99 99
Lines 9263 9309 +46
Branches 2082 2090 +8
==========================================
+ Hits 6853 6942 +89
+ Misses 2289 2254 -35
+ Partials 121 113 -8 ☔ View full report in Codecov by Sentry. |
| const lastJobTimestamp = hubState.validateOrRevokeState?.lastJobTimestamp ?? 0; | ||
| const lastFid = hubState.validateOrRevokeState?.lastFid ?? 0; | ||
|
|
||
| log.info({ lastJobTimestamp, lastFid, numFids }, "ValidateOrRevokeMessagesJob: starting"); | ||
|
|
||
| for (let i = 0; i < numFids; i++) { | ||
| const fid = allFids[i] as number; |
There was a problem hiding this comment.
Do we still need to read all fids into memory?
| } | ||
| const numChecked = await this.doJobForFid(lastJobTimestamp, fid); | ||
| // We'll alwats check for username proof messages | ||
| const numUsernamesChecked = await this.doUsernamesJobForFid(fid); |
There was a problem hiding this comment.
Most users won't have this. Checking the storage cache message count for the UserNameProof messages will save db reads here.
|
|
||
| do { | ||
| // First, find if any signers have changed since the last time the job ran | ||
| const signers = await this._engine.getOnChainSignersByFid(fid); |
There was a problem hiding this comment.
hmm, too bad the on chain event has fid before the timestamp, otherwise we could scan all onchain events in one go after a particular block number.
But maybe it's still cheaper to fetch the onchain events by block number than iterate through all fids/onchain events?
Anyway, not a huge deal. This is still a significant improvment from what it used to do.
There was a problem hiding this comment.
I'll look into this in a new PR. But yeah, this new way is like much faster. It now takes only an hour or so (depending on signers changed) vs ~12 hours before.
* fix: Run validateOrRevoke only if signer is changed * cleanup * timebug * review
* fix: Run validateOrRevoke only if signer is changed * cleanup * timebug * review
Motivation
The validateOrRevokeMessages job takes ~12 hours to run everyday. It is very inefficient because it checks every single message.
Change Summary
Merge Checklist
Choose all relevant options below by adding an
xnow or at any time before submitting for reviewPR-Codex overview
This PR focuses on adding the
validateOrRevokeStatefield to theHubStatemessage and implementing theValidateOrRevokeMessagesJobSchedulerclass.Detailed summary
validateOrRevokeStatefield toHubStatemessage inhub_state.protoandhub_state.tsValidateOrRevokeJobStatemessage inhub_state.tscreateBaseValidateOrRevokeJobStatefunction inhub_state.tsValidateOrRevokeJobStateinhub_state.tsdoJobForFidanddoJobsmethods toValidateOrRevokeMessagesJobSchedulerinvalidateOrRevokeMessagesJob.tsvalidateOrRevokeMessagesJob.test.ts