Skip to content

Rectify Transactions and BlockchainEvents #408

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
Jan 13, 2022
Merged

Rectify Transactions and BlockchainEvents #408

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
ec8e390
Rename ContractEvent to BlockchainEvent
awrichar Jan 7, 2022
5211a3d
Record BlockchainEvents for batch pin, token pool, and token transfer
awrichar Jan 11, 2022
74efe7e
Remove unneeded fields from Transaction
awrichar Jan 12, 2022
ccaa89a
Link BlockchainEvents to Transactions
awrichar Jan 12, 2022
0e654d2
Link TokenTransfers to BlockchainEvents (instead of Transactions)
awrichar Jan 12, 2022
d45a13d
Merge remote-tracking branch 'origin/onchain-logic' into HEAD
awrichar Jan 13, 2022
52ecc32
Clean up database migrations and indexes
awrichar Jan 13, 2022
be94b48
Adjust token E2E tests to wait for exact events
awrichar Jan 13, 2022
9988275
Restore remaining test coverage
awrichar Jan 13, 2022
95e8f4b
Add missing "Source" field when storing BlockchainEvents
awrichar Jan 13, 2022
9f24622
Rename "outputs" to "output" in BlockchainEvent JSON
awrichar Jan 13, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 1 addition & 2 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,6 @@ $(eval $(call makemock, internal/oapiffi, FFISwaggerGen, oapiffimo
$(eval $(call makemock, internal/orchestrator, Orchestrator, orchestratormocks))
$(eval $(call makemock, internal/apiserver, Server, apiservermocks))
$(eval $(call makemock, internal/apiserver, IServer, apiservermocks))
$(eval $(call makemock, internal/txcommon, Helper, txcommonmocks))

firefly-nocgo: ${GOFILES}
CGO_ENABLED=0 $(VGO) build -o ${BINARY_NAME}-nocgo -ldflags "-X main.buildDate=`date -u +\"%Y-%m-%dT%H:%M:%SZ\"` -X main.buildVersion=$(BUILD_VERSION)" -tags=prod -tags=prod -v
Expand All @@ -88,4 +87,4 @@ deps:
swagger:
$(VGO) test ./internal/apiserver -timeout=10s -tags swagger
manifest:
./manifestgen.sh
./manifestgen.sh
3 changes: 3 additions & 0 deletions db/migrations/postgres/000054_create_blockchainevents_table.down.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
BEGIN;
DROP TABLE IF EXISTS blockchainevents;
COMMIT;
21 changes: 21 additions & 0 deletions db/migrations/postgres/000054_create_blockchainevents_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
BEGIN;
CREATE TABLE blockchainevents (
Copy link
Contributor

@peterbroadhurst peterbroadhurst Jan 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think we need an index on tx_id for this one

Copy link
Contributor

@peterbroadhurst peterbroadhurst Jan 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and id

Copy link
Contributor Author

@awrichar awrichar Jan 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see that the onchain-logic branch has these indexes now (they are on contractevents but I just renamed that table):

CREATE INDEX blockchainevents_name ON blockchainevents(namespace,name);
CREATE INDEX blockchainevents_timestamp ON blockchainevents(timestamp);
CREATE INDEX blockchainevents_subscription_id ON blockchainevents(subscription_id);

If we add 2 more indexes, is that a bit much for a potentially high-write-volume table?

Copy link
Contributor Author

@awrichar awrichar Jan 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do think the ones you mentioned are useful... I guess just wondering if I should also prune any others? Possibly name?

Copy link
Contributor

@peterbroadhurst peterbroadhurst Jan 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think we go with this for now, and test - can't see any obvious candidates to cull.

Copy link
Contributor Author

@awrichar awrichar Jan 13, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the end I did remove namespace,name. It's easy enough to add back later if we want.

seq INTEGER PRIMARY KEY AUTOINCREMENT,
id UUID NOT NULL,
source VARCHAR(256) NOT NULL,
namespace VARCHAR(64) NOT NULL,
name VARCHAR(256) NOT NULL,
protocol_id VARCHAR(256) NOT NULL,
timestamp BIGINT NOT NULL,
subscription_id UUID,
output BYTEA,
info BYTEA,
tx_type VARCHAR(64),
tx_id UUID
);

CREATE INDEX blockchainevents_id ON blockchainevents(id);
CREATE INDEX blockchainevents_tx ON blockchainevents(tx_id);
CREATE INDEX blockchainevents_timestamp ON blockchainevents(timestamp);
CREATE INDEX blockchainevents_subscription_id ON blockchainevents(subscription_id);
COMMIT;
3 changes: 0 additions & 3 deletions db/migrations/postgres/000054_create_contractevents_table.down.sql
View file
Open in desktop

This file was deleted.

15 changes: 0 additions & 15 deletions db/migrations/postgres/000054_create_contractevents_table.up.sql
View file
Open in desktop

This file was deleted.

10 changes: 10 additions & 0 deletions db/migrations/postgres/000056_drop_transactions_columns.down.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
BEGIN;
ALTER TABLE transactions ADD COLUMN ref UUID;
ALTER TABLE transactions ADD COLUMN signer VARCHAR(1024);
ALTER TABLE transactions ADD COLUMN hash CHAR(64);
ALTER TABLE transactions ADD COLUMN protocol_id VARCHAR(256);
ALTER TABLE transactions ADD COLUMN info BYTEA;

CREATE INDEX transactions_protocol_id ON transactions(protocol_id);
CREATE INDEX transactions_ref ON transactions(ref);
COMMIT;
10 changes: 10 additions & 0 deletions db/migrations/postgres/000056_drop_transactions_columns.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
BEGIN;
DROP INDEX transactions_protocol_id;
DROP INDEX transactions_ref;

ALTER TABLE transactions DROP COLUMN ref;
ALTER TABLE transactions DROP COLUMN signer;
ALTER TABLE transactions DROP COLUMN hash;
ALTER TABLE transactions DROP COLUMN protocol_id;
ALTER TABLE transactions DROP COLUMN info;
COMMIT;
5 changes: 5 additions & 0 deletions db/migrations/postgres/000057_add_tokentransfer_blockchainevent.down.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
BEGIN;
ALTER TABLE tokentransfer DROP COLUMN blockchain_event;
ALTER TABLE tokentransfer ADD COLUMN tx_type VARCHAR(64);
ALTER TABLE tokentransfer ADD COLUMN tx_id UUID;
COMMIT;
5 changes: 5 additions & 0 deletions db/migrations/postgres/000057_add_tokentransfer_blockchainevent.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
BEGIN;
ALTER TABLE tokentransfer DROP COLUMN tx_type;
ALTER TABLE tokentransfer DROP COLUMN tx_id;
ALTER TABLE tokentransfer ADD COLUMN blockchain_event UUID;
COMMIT;
1 change: 1 addition & 0 deletions db/migrations/sqlite/000054_create_blockchainevents_table.down.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
DROP TABLE IF EXISTS blockchainevents;
19 changes: 19 additions & 0 deletions db/migrations/sqlite/000054_create_blockchainevents_table.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
CREATE TABLE blockchainevents (
seq INTEGER PRIMARY KEY AUTOINCREMENT,
id UUID NOT NULL,
source VARCHAR(256) NOT NULL,
namespace VARCHAR(64) NOT NULL,
name VARCHAR(256) NOT NULL,
protocol_id VARCHAR(256) NOT NULL,
timestamp BIGINT NOT NULL,
subscription_id UUID,
output TEXT,
info TEXT,
tx_type VARCHAR(64),
tx_id UUID
);

CREATE INDEX blockchainevents_id ON blockchainevents(id);
CREATE INDEX blockchainevents_tx ON blockchainevents(tx_id);
CREATE INDEX blockchainevents_timestamp ON blockchainevents(timestamp);
CREATE INDEX blockchainevents_subscription_id ON blockchainevents(subscription_id);
1 change: 0 additions & 1 deletion db/migrations/sqlite/000054_create_contractevents_table.down.sql
View file
Open in desktop

This file was deleted.

14 changes: 0 additions & 14 deletions db/migrations/sqlite/000054_create_contractevents_table.up.sql
View file
Open in desktop

This file was deleted.

8 changes: 8 additions & 0 deletions db/migrations/sqlite/000056_drop_transactions_columns.down.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
ALTER TABLE transactions ADD COLUMN ref UUID;
ALTER TABLE transactions ADD COLUMN signer VARCHAR(1024);
ALTER TABLE transactions ADD COLUMN hash CHAR(64);
ALTER TABLE transactions ADD COLUMN protocol_id VARCHAR(256);
ALTER TABLE transactions ADD COLUMN info BYTEA;

CREATE INDEX transactions_protocol_id ON transactions(protocol_id);
CREATE INDEX transactions_ref ON transactions(ref);
8 changes: 8 additions & 0 deletions db/migrations/sqlite/000056_drop_transactions_columns.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
DROP INDEX transactions_protocol_id;
DROP INDEX transactions_ref;

ALTER TABLE transactions DROP COLUMN ref;
ALTER TABLE transactions DROP COLUMN signer;
ALTER TABLE transactions DROP COLUMN hash;
ALTER TABLE transactions DROP COLUMN protocol_id;
ALTER TABLE transactions DROP COLUMN info;
3 changes: 3 additions & 0 deletions db/migrations/sqlite/000057_add_tokentransfer_blockchainevent.down.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
ALTER TABLE tokentransfer DROP COLUMN blockchain_event;
ALTER TABLE tokentransfer ADD COLUMN tx_type VARCHAR(64);
ALTER TABLE tokentransfer ADD COLUMN tx_id UUID;
3 changes: 3 additions & 0 deletions db/migrations/sqlite/000057_add_tokentransfer_blockchainevent.up.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
ALTER TABLE tokentransfer DROP COLUMN tx_type;
ALTER TABLE tokentransfer DROP COLUMN tx_id;
ALTER TABLE tokentransfer ADD COLUMN blockchain_event UUID;
Loading