Skip to content
This repository has been archived by the owner on May 29, 2024. It is now read-only.

ContractCreationTx Register Pipeline POC #3

Merged
merged 15 commits into from
Apr 3, 2023
Merged

Conversation

epociask
Copy link
Collaborator

@epociask epociask commented Mar 28, 2023

Disclaimer

This implementation is currently suboptimal and is done to provide a baseline scaffolding for the pessimism development team to begin iterating and developing on top of. Please note that the chain indexing logic for the GethBlock register oracle is very poor and was only implemented in it's existing state for quick iteration. This logic will need to be revisited, heavily worked, and deeply tested to ensure that:

  1. Backtest support can be fully integratabtle for the oracle
  2. All potential failure cases are threat modeled with successful fail safe procedures
  3. Optional persistence is allowed where backtesting data reads can be performed on some relational database

Additionally, it's worth noting that that the oracle implementation is rather lackluster as well and will require some further attention.

Changes

  • Added Pipe component that takes arbitrary injected callback to transform some TransitData register type to another
  • Added Oracle component that takes an OracleDefinition interface, allowing for semi-intuitive creation of arbitrary external data source readers
  • Added OutputRouter construct to allow for n number subscriptions between oracles and pipes
  • Tested all OutputRouter struct logic
  • Added a GethBlock register definition that
  • Added configuration logic that reads some env file into process env vars
  • Added a config.env.template to template cfg vars
  • Fixed broken lint target in Makefile
  • Modified .golangci.yml file to disable unnecessary go linters
  • Added registry implementation that supports arbitrary DataRegister types with dependencies
  • Added go-test CI action for running go unit tests

Proof-of-Concept

About

This POC was performed to ensure that component interaction operates as intended. Additionally, the exercise of manually building components and binding them together allows us to better understand how we can automate this creation with higher level abstractions. All the POC does is creates, binds, and runs two components together to extract Contract Creation transactions from an oracle fetched block. Effectively, it implicitly generates a ContractCreateTx register pipeline. The register pipeline looks like so:

GethBlockOracle --Block--> ContractCreationTxPipe --Tx--> readRoutine

To Run

  1. go build -a -tags netgo -o bin/block_read_poc ./cmd/block_read_poc/
  2. ./bin/block_read_poc

Example Output

Screenshot 2023-03-29 at 6 23 53 PM

NOTE

This driver script will be deleted sometime in the foreseeable future, it's only current purpose is to showcase that these components work in a useful way. Additionally, once we have automation support for generating register pipelines, we can generate this pipeline using that logic and run this script to ensure outputs match.

@epociask epociask changed the title ETL Pipe Component Implementation ETL Pipe && Oracle Component Implementation Mar 28, 2023
@epociask epociask changed the title ETL Pipe && Oracle Component Implementation ETL Pipe && Oracle Component Implementations Mar 28, 2023
@epociask epociask requested a review from anupsv-cb March 29, 2023 21:52
@epociask epociask changed the base branch from development to master March 29, 2023 22:43
@epociask epociask changed the title ETL Pipe && Oracle Component Implementations Preliminary Pipeline Implementation Mar 29, 2023
@epociask epociask changed the title Preliminary Pipeline Implementation Preliminary Pipeline Implementation POC Mar 29, 2023
@epociask epociask changed the title Preliminary Pipeline Implementation POC Contract Transaction Pipeline Implementation POC Mar 30, 2023
@epociask epociask changed the title Contract Transaction Pipeline Implementation POC Contract Creation Tx Detection Pipeline POC Mar 30, 2023
@epociask epociask marked this pull request as ready for review March 30, 2023 09:13
@epociask epociask changed the title Contract Creation Tx Detection Pipeline POC Contract Creation Tx Register Pipeline POC Mar 30, 2023
@epociask epociask changed the title Contract Creation Tx Register Pipeline POC Contract Creation Tx Register Pipeline POC Mar 30, 2023
@epociask epociask changed the title Contract Creation Tx Register Pipeline POC ContractCreationTx Register Pipeline POC Mar 30, 2023
Copy link
Contributor

@anupsv-cb anupsv-cb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets get this goin.

@cb-heimdall
Copy link
Collaborator

Review Error for anupsv-cb @ 2023-04-03 21:23:24 UTC
User failed mfa authentication, public email is not set on your github profile. see go/mfa-help

@epociask epociask merged commit b0c5642 into master Apr 3, 2023
@nadir-akhtar-coinbase nadir-akhtar-coinbase deleted the epociask/PIPE_COMPONENT branch June 26, 2023 22:02
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants