Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Map out operations input and output data structures for CLI #26

Closed
Isaiah-Turner opened this issue Jul 6, 2020 · 1 comment
Closed
Assignees

Comments

@Isaiah-Turner
Copy link
Contributor

What

We need to map out what information we get about operations from the new ingestion system. We also need to know what the output to BigQuery should look like. Part of #19.

Why

This issue helps us know what information about operations we can get from the ingestion system and what information may have to be reconstructed or omitted.

@Isaiah-Turner
Copy link
Contributor Author

Isaiah-Turner commented Jul 9, 2020

In order to get Operations info, we need to look through Transaction information. The NewLedgerTransactionReader() function creates a transaction reader for a given ledger, which allows for iterating through each transaction. Calling txReader.Read() gives a LedgerTransaction struct, which is defined as:

type LedgerTransaction struct {
	Index    uint32
	Envelope xdr.TransactionEnvelope
	Result   xdr.TransactionResultPair
	// FeeChanges and Meta are low level values.
	// Use LedgerTransaction.GetChanges() for higher level access to ledger
	// entry changes.
	FeeChanges xdr.LedgerEntryChanges
	Meta       xdr.TransactionMeta
}

Operations are stored in tx.Envelope.V1.Tx.Operations (call a given operation op)
In order to export the data to BigQuery, we need a struct with the following fields:

source_account: string - get from op.SourceAccount.ToAccountId().GetAddress(); if it doesn't exist then use the transaction source account
transaction_id: integer - TODO: figure out history table mapping in order to connect operations to their parent transaction
type: integer - get from op.Body.Type
application_order:  integer - get from index of op in the operations list; starts at 1 instead of 0
details: struct
details.account: string
details.amount: float
details.asset_code: string
details.asset_issuer: string
details.asset_type: string
details.authorize: boolean
details.buying_asset_code: string
details.buying_asset_issuer: string
details.buying_asset_type: string
details.from: string
details.funder: string
details.high_threshold: integer
details.home_domain: string
details.inflation_dest: string
details.into: string
details.limit: float
details.low_threshold: integer
details.master_key_weight: integer
details.med_threshold: integer
details.name: string
details.offer_id: integer
details.path: Path struct
details.path.asset_code: string
details.path.asset_issuer: string
details.path.asset_issuer: string
details.price: float
details.price_r: Price struct
details.price_r.d: integer
details.price_r.n: integer
details.selling_asset_code: string
details.selling_asset_issuer: string
details.selling_asset_type: string
details.set_flags: integer
details.set_flags_s: string
details.signer_key: string
details.signer_weight: integer
details.source_amount: float
details.source_asset_code: string
details.source_asset_issuer: string
details.source_asset_type: string
details.source_max: float
details.starting_balance: float
details.to: string
details.trustee: string
details.trustor: string
details.value: string
details.clear_flags: integer
details.clear_flags_s: string
details.destination_min: string
details.bump_to: string

The details struct must be filled out on a case by case basis. The fields that get filled out are dependent on the type of operation that is being performed. If a field is not filled, it should default to null. See the Operations_Processor's Details() method for an example of a switch that fills out the details based on operation type.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant