Add support for geth traces

[medvedev1088]

The scope of this task:

1. Design the model for geth traces (trace.py can be used as a basis).
2. Implement the export and extract jobs.
   2.a The export job should use traceBlockByNumber (cmd, core, eth/tracers: support fancier js tracing ethereum/go-ethereum#15516) and output raw JSON responses. It should use callTracer, an example JS API call: debug.traceBlockByNumber('0x250252', {'tracer': 'callTracer'}); the same is available via JSON RPC. The job should use BatchWeb3Provider similar to export_blocks_job.py.
   2.b The extract job should take the file with raw JSON responses, parses them, map to our domain entity and output to CSV file.
3. Write unit tests (real mainnet responses should be used in test cases, so the same tests can be run against geth node).
4. Add documentation to README (an example can be found here). Include instructions for how to start geth i.e. what options to use (e.g. geth --gcmode archive)

=======================
Notes:

• Parity traces are already supported implementation of internal transaction exporter #104.

[medvedev1088]

@evgeniuz are you up for taking this task?

[evgeniuz]

@medvedev1088 ok, will try to implement this.

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

This issue now has a funding of 400.0 DAI (400.0 USD @ $1.0/DAI) attached to it as part of the Ethereum Community Fund via ECF Web 3.0 Infrastructure Fund fund.

• If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $50,556.08 more funded OSS Work available on the Gitcoin Issue Explorer

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work has been started.

These users each claimed they can complete the work by 7 months, 2 weeks from now.
Please review their action plans below:

1) evgeniuz has been approved to start work.

Already started work on implementation. Will try to go with the plan suggested (two commands, one to export JSON, second one to process it to domain format). I will also try to look into on-node tracers (not sure if they good enough for this exporter) to optimize performance (as exported JSON from traceBlock might be quite big).

Learn more on the Gitcoin Issue Details page.

[medvedev1088]

@evgeniuz I added more details in task description, in particular callTracer option for traceBlockByNumber api. See if it helps.

[holiman]

Geth developer here. I would suggest you add support instead for the standard-json trace format, described here. It is supported by both geth and parity.

However, the callTracer uses a format which is not as good. I've made this PR which should make post-processing of blocks and tracers much nicer: ethereum/go-ethereum#17914 .

[medvedev1088]

@holiman thanks for the suggestion. Exporting "raw" traces sounds like an interesting idea. By "raw" traces I mean the list of opcodes with parameters as seen on the "Raw traces" page on Etherscan https://etherscan.io/vmtrace?txhash=0xa2dd0f7e60eb13d22739cdf9ed34a24c12f642e17db66308beabba7852f9c0b2#raw. I assume you meant the same in your comment. In Parity they are called vmTrace.

Having this trace it will be possible to reimplement higher level tracers in BigQuery UDFs in JavaScript, e.g. you could implement callTracer or any other type of tracer. The results can be retrieved in seconds with the power of BigQuery. Do you see any potential caveats?

Could you try to estimate the size of raw traces for all blocks? I suspect it would be in the order of a few terabytes, given that all Parity traces are around 300GB.

[evgeniuz]

@medvedev1088 Should I continue implementation with callTracer? It might be still useful to quickly get just internal calls without parsing huge datasets.

[medvedev1088]

@evgeniuz yes please continue as planned. Raw traces can be a separate task.

[gitcoinbot]

@evgeniuz Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• warning (3 days)
• escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[gitcoinbot]

@evgeniuz Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• warning (3 days)
• escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

Work for 400.0 DAI (400.0 USD @ $1.0/DAI) has been submitted by:

1. @evgeniuz

@mkosowsk please take a look at the submitted work:

• PR by @evgeniuz

---

• Learn more on the Gitcoin Issue Details page
• Want to chip in? Add your own contribution here.
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $58,847.29 more funded OSS Work available on the Gitcoin Issue Explorer

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

The funding of 400.0 DAI (400.0 USD @ $1.0/DAI) attached to this issue has been approved & issued to @evgeniuz.

• Learn more on the Gitcoin Issue Details page
• Questions? Checkout Gitcoin Help or the Gitcoin Slack
• $57,755.09 more funded OSS Work available on the Gitcoin Issue Explorer