Fetch TokenBalances values in Realtime and Catchup Indexers #583

igorffs · 2018-08-21T21:57:29Z

#514

Motivation

This PR makes the Indexers to fetch the TokenBalances value and value_fetched_at both in their own way:

The Realtime indexer fetches the data synchronously
The Catchup indexer fetches the data assynchronously

WIP

The values are already fetching the values after we've tested it local, but the code still needs some improvements and tests.

feliperenan · 2018-08-22T16:25:06Z

apps/explorer/lib/explorer/chain.ex

+          from(
+            tb in TokenBalance,
+            where: is_nil(tb.value_fetched_at),
+            select: %TokenBalance{


Since we want to send the TokenBalance struct, I think we don't need to select the specific fields. Even we don't need the whole TokenBalance, IMO sending the all attributes it's not a problem at all.

I have changed it already. Please, take a look and see if it makes sense to you as well.

cc @igorffs

feliperenan · 2018-08-22T21:04:41Z

apps/ethereum_jsonrpc/lib/ethereum_jsonrpc.ex

    params = [%{to: address, data: data}]
    request(%{id: id, method: "eth_call", params: params})
  end

+  # Block.block_number()


KronicDeth · 2018-08-23T13:03:45Z

Rebased on iff-index-token-balances after iff-index-token-balances was rebased on master

KronicDeth · 2018-08-23T23:16:01Z

Changed to be against master now that the old base's PR ( #582) has been merged to master

feliperenan · 2018-08-27T13:50:15Z

Hey @KronicDeth, I applied your suggestions. Could please you take a look again? 🙏

KronicDeth · 2018-08-27T14:33:40Z

@feliperenan you fixed apps/explorer/config/dev/parity.exs, but missed the duplicate change in apps/explorer/config/prod/parity.exs

feliperenan · 2018-08-27T14:58:52Z

@KronicDeth fixed.

acravenho

LGTM 👍 Great work!

feliperenan · 2018-08-27T19:13:54Z

@gfreh This one is ready for QA :).

alexgaribay

Looks good 👍 !

KronicDeth · 2018-08-28T17:26:24Z

If you run into ON CONFLICT DO UPDATE command cannot affect row a second time with insert_token_balances in the stacktrace while running this locally or testing, you're hitting #614 and there's a fix in #619.

coveralls · 2018-08-28T17:37:45Z

Pull Request Test Coverage Report for Build b05dc011-82e2-4f18-9dce-289f9ebf9e11

15 of 15 (100.0%) changed or added relevant lines in 5 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.2%) to 89.769%

Totals
Change from base Build 0e13a1c6-ee6d-4286-a4ba-f630d5908717:	0.2%
Covered Lines:	1711
Relevant Lines:	1906

💛 - Coveralls

KronicDeth · 2018-08-28T17:54:56Z

The new lines added to execute_contract_function and build_eth_call_payload are uncovered

The old lines for build_eth_call_payload are also uncovered, so you'll likely need to add tests for build_eth_call_payload in general since we missed the coverage while coveralls.io reporting was broken.

We're now able to get the balances from a Address given it's address_hash, block_number and token_contract_address_hash.

acravenho · 2018-08-29T06:55:50Z

@igorffs @feliperenan Is there any further work that needs to be completed on this PR? If not I would like to get this merged asap. We have a DApp that needs this data today through an API endpoint.

feliperenan · 2018-08-29T16:32:35Z

@acravenho done!, we are just doing some tests here 😅.

acravenho · 2018-08-29T16:36:41Z

@feliperenan Thank you for the quick work 👍

@sabondano This has now been merged which was a previous blocker to the tokenbalance API endpoint.

igorffs added the wip label Aug 21, 2018

igorffs assigned feliperenan Aug 21, 2018

igorffs requested a review from feliperenan August 21, 2018 21:57

igorffs mentioned this pull request Aug 21, 2018

Index TokenBalances #582

Merged

igorffs added the team: developer label Aug 21, 2018

feliperenan force-pushed the iff-index-token-balances branch 2 times, most recently from 68bd356 to 31a6756 Compare August 21, 2018 23:16

feliperenan force-pushed the iff-frg-fetch-token-balances-values branch from 9222b21 to 7fd5768 Compare August 22, 2018 14:30

feliperenan reviewed Aug 22, 2018

View reviewed changes

feliperenan force-pushed the iff-frg-fetch-token-balances-values branch from dd232fe to 3d26a79 Compare August 22, 2018 19:23

igorffs force-pushed the iff-index-token-balances branch from ecdc3f5 to a4ebffa Compare August 22, 2018 19:25

feliperenan force-pushed the iff-frg-fetch-token-balances-values branch from 0d42fff to 60bc01d Compare August 22, 2018 20:30

feliperenan reviewed Aug 22, 2018

View reviewed changes

KronicDeth force-pushed the iff-frg-fetch-token-balances-values branch from 02cb530 to 900bb22 Compare August 23, 2018 12:58

KronicDeth force-pushed the iff-index-token-balances branch from a4ebffa to 1f4951a Compare August 23, 2018 13:01

KronicDeth force-pushed the iff-frg-fetch-token-balances-values branch from 900bb22 to 2cb5ba9 Compare August 23, 2018 13:03

KronicDeth force-pushed the iff-index-token-balances branch from 1f4951a to 717ea05 Compare August 23, 2018 13:19

KronicDeth force-pushed the iff-frg-fetch-token-balances-values branch 2 times, most recently from 17bc6c8 to 49226a1 Compare August 23, 2018 13:21

KronicDeth force-pushed the iff-index-token-balances branch from 717ea05 to 518c9da Compare August 23, 2018 13:26

KronicDeth force-pushed the iff-frg-fetch-token-balances-values branch from 49226a1 to 47cd800 Compare August 23, 2018 13:27

KronicDeth force-pushed the iff-index-token-balances branch from 518c9da to 0f7ba5d Compare August 23, 2018 13:39

KronicDeth force-pushed the iff-frg-fetch-token-balances-values branch from 47cd800 to e83209c Compare August 23, 2018 13:40

feliperenan force-pushed the iff-frg-fetch-token-balances-values branch from e83209c to a7b9272 Compare August 23, 2018 13:49

KronicDeth force-pushed the iff-index-token-balances branch from 939a613 to b05196d Compare August 23, 2018 22:48

KronicDeth force-pushed the iff-frg-fetch-token-balances-values branch from c8bd5b5 to 09752d3 Compare August 23, 2018 23:15

KronicDeth changed the base branch from iff-index-token-balances to master August 23, 2018 23:16

feliperenan added the ready for review This PR is ready for reviews. label Aug 24, 2018

feliperenan force-pushed the iff-frg-fetch-token-balances-values branch from 022a515 to 0a5b735 Compare August 27, 2018 13:44

feliperenan force-pushed the iff-frg-fetch-token-balances-values branch from 0a5b735 to c765560 Compare August 27, 2018 14:45

KronicDeth approved these changes Aug 27, 2018

View reviewed changes

feliperenan requested a review from acravenho August 27, 2018 16:56

acravenho approved these changes Aug 27, 2018

View reviewed changes

alexgaribay approved these changes Aug 27, 2018

View reviewed changes

feliperenan force-pushed the iff-frg-fetch-token-balances-values branch from c765560 to fdf5d8c Compare August 27, 2018 20:40

feliperenan mentioned this pull request Aug 28, 2018

Use indexed token balances in the address page balances dropdown #612

Merged

KronicDeth force-pushed the iff-frg-fetch-token-balances-values branch from fdf5d8c to 59c60e3 Compare August 28, 2018 17:28

Igor Florian and others added 5 commits August 28, 2018 13:00

Get TokensBalance in the Blockchain

c2ff177

We're now able to get the balances from a Address given it's address_hash, block_number and token_contract_address_hash.

Fetch TokenBalances values in the Realtime Indexer

b56f8d4

Fetch TokenBalances value in the Catchup Indexer

53e7866

Include TokenBalanceFetcher to Indexer.Supervisor test support

01af13e

Fix dialyzer

ad50a21

KronicDeth force-pushed the iff-frg-fetch-token-balances-values branch from 59c60e3 to ad50a21 Compare August 28, 2018 18:00

Add tests to EthereumJSONRPC.execute_contract_functions/3

e4a9463

feliperenan force-pushed the iff-frg-fetch-token-balances-values branch from 4277b53 to e4a9463 Compare August 28, 2018 18:52

feliperenan merged commit 196f298 into master Aug 29, 2018

KronicDeth deleted the iff-frg-fetch-token-balances-values branch August 29, 2018 19:06

igorffs mentioned this pull request Aug 30, 2018

Change Balance to CoinBalance and make its references consistent to TokenBalances #644

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fetch TokenBalances values in Realtime and Catchup Indexers #583

Fetch TokenBalances values in Realtime and Catchup Indexers #583

igorffs commented Aug 21, 2018

feliperenan Aug 22, 2018

feliperenan Aug 22, 2018

feliperenan Aug 22, 2018

KronicDeth commented Aug 23, 2018

KronicDeth commented Aug 23, 2018 •

edited

feliperenan commented Aug 27, 2018

KronicDeth commented Aug 27, 2018

feliperenan commented Aug 27, 2018

acravenho left a comment

feliperenan commented Aug 27, 2018

alexgaribay left a comment

KronicDeth commented Aug 28, 2018

coveralls commented Aug 28, 2018 •

edited

KronicDeth commented Aug 28, 2018

acravenho commented Aug 29, 2018

feliperenan commented Aug 29, 2018

acravenho commented Aug 29, 2018

Fetch TokenBalances values in Realtime and Catchup Indexers #583

Fetch TokenBalances values in Realtime and Catchup Indexers #583

Conversation

igorffs commented Aug 21, 2018

Motivation

WIP

feliperenan Aug 22, 2018

Choose a reason for hiding this comment

feliperenan Aug 22, 2018

Choose a reason for hiding this comment

feliperenan Aug 22, 2018

Choose a reason for hiding this comment

KronicDeth commented Aug 23, 2018

KronicDeth commented Aug 23, 2018 • edited

feliperenan commented Aug 27, 2018

KronicDeth commented Aug 27, 2018

feliperenan commented Aug 27, 2018

acravenho left a comment

Choose a reason for hiding this comment

feliperenan commented Aug 27, 2018

alexgaribay left a comment

Choose a reason for hiding this comment

KronicDeth commented Aug 28, 2018

coveralls commented Aug 28, 2018 • edited

Pull Request Test Coverage Report for Build b05dc011-82e2-4f18-9dce-289f9ebf9e11

💛 - Coveralls

KronicDeth commented Aug 28, 2018

acravenho commented Aug 29, 2018

feliperenan commented Aug 29, 2018

acravenho commented Aug 29, 2018

KronicDeth commented Aug 23, 2018 •

edited

coveralls commented Aug 28, 2018 •

edited