Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

analyzer/runtime: Refactor: Separate download, extract, sqlize steps #358

Merged
merged 10 commits into from
Mar 21, 2023

Conversation

mitjat
Copy link
Collaborator

@mitjat mitjat commented Mar 16, 2023

Prework for https://app.clickup.com/t/8669n7f41

The runtime analyzer is currently an amalgamation of the early Emerald code and the imported code from oasis-nexus.

This PR refactors the analyzer to unify their approaches more, though there's still a lot of cleanup opportunities let.
In particular, the PR refactors the analyzer so that each block is processed in three stages:

  • fetch all data from node (this was already separate as of recently; thanks Andy)
  • preprocess structures from the node into a rich BlockData struct (this was the approach from nexus/EVM/Warren, but the data was only used in some places, while others used the raw structures directly. That makes it a lot harder to rework the structures coming from the node, as we need to do to support multiple versions of oasis-core/oasis-sdk)
  • convert the BlockData structure into sql statements

Testing:
Analyzed 5k blocks before and after the refactor, compared DB dumps. No diffs except that the unused chain.runtime_gas_used table is gone.

Copy link
Collaborator

@pro-wh pro-wh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good so far

@mitjat mitjat force-pushed the mitjat/runtime-three-step-workflow branch 2 times, most recently from 7ddf927 to 51259fb Compare March 17, 2023 20:48
@mitjat mitjat marked this pull request as ready for review March 17, 2023 20:50
Copy link
Collaborator

@Andrew7234 Andrew7234 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, thanks!

analyzer/runtime/extract.go Outdated Show resolved Hide resolved
@mitjat mitjat force-pushed the mitjat/runtime-three-step-workflow branch 2 times, most recently from 178df7c to 31dfc19 Compare March 21, 2023 19:55
@mitjat mitjat enabled auto-merge March 21, 2023 19:56
@mitjat mitjat merged commit 705e527 into main Mar 21, 2023
@mitjat mitjat deleted the mitjat/runtime-three-step-workflow branch March 21, 2023 20:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants