Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove network observer module #115

Merged
merged 16 commits into from
Dec 17, 2022
Merged

Remove network observer module #115

merged 16 commits into from
Dec 17, 2022

Conversation

nickgarfield
Copy link
Member

@nickgarfield nickgarfield commented Dec 14, 2022

This PR removes the network observer module from the plugin. The network observer was designed to listen for changes to accounts belonging to the network program. It would then cache these accounts in memory and use them to in the transaction execution flows. We began noticing issues a few days ago (see ticket below for details) where this cached data was stale or incorrect. This would cause a worker would think it's in the pool even when it is not. Or visa versa, a worker would think it's not in the pool when it is.

Looking at transaction histories, it's still clear why the cached account data was incorrect. The only theory that makes sense to me is that workers can sometimes be on the bad fork and when jumping to the correct fork, account updates are not pushed to geyser. This theory is not confirmed, but in the meantime this PR serves as a fix for the issue. Instead of relying on the cached account data, this PR updates the plugin to simply use the client to fetch the fresh account data for registry, pool, and snapshot accounts.

https://linear.app/clockwork-xyz/issue/CLO-122/workers-not-recognizing-pool-position-updates

@nickgarfield nickgarfield changed the title Resolve issue where workers are not observing pool rotations [🚧 WIP] Resolve issue where workers are not observing pool rotations Dec 16, 2022
@nickgarfield nickgarfield changed the title [🚧 WIP] Resolve issue where workers are not observing pool rotations [🚧 WIP] Investigate issue where workers are not observing pool rotations Dec 16, 2022
@nickgarfield nickgarfield changed the title [🚧 WIP] Investigate issue where workers are not observing pool rotations Remove network observer module Dec 17, 2022
@nickgarfield
Copy link
Member Author

nickgarfield commented Dec 17, 2022

I have this deployed on the mainnet nodes, and all looks good. Will continue monitoring for a few more hours and if things continue to look good this evening, we can land.

@nickgarfield nickgarfield merged commit 7e36503 into main Dec 17, 2022
@nickgarfield nickgarfield deleted the nick/pool-position-bug branch January 10, 2023 23:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant