Migrate dodo #4043
Migrate dodo #4043
Conversation
|
CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅ |
|
Workflow run id 5831069201 approved. |
|
Workflow run id 5831069430 approved. |
|
I have read the CLA Document and I hereby sign the CLA |
|
Workflow run id 5831505926 approved. |
|
Workflow run id 5831506061 approved. |
|
recheck |
|
Workflow run id 5832014811 approved. |
|
Workflow run id 5832014641 approved. |
|
Workflow run id 5832401343 approved. |
|
Workflow run id 5832401563 approved. |
|
Workflow run id 5832559346 approved. |
|
Workflow run id 5832559416 approved. |
|
Workflow run id 5832706047 approved. |
|
Workflow run id 5832706275 approved. |
|
Workflow run id 5832833242 approved. |
|
Workflow run id 5832833362 approved. |
|
Workflow run id 5832984891 approved. |
|
Workflow run id 5832984419 approved. |
|
Workflow run id 5833129258 approved. |
|
Workflow run id 5833129491 approved. |
|
Workflow run id 5867276001 approved. |
|
Workflow run id 5867275884 approved. |
|
Hey @jeff-dude @Hosuke , How to identify the issue of repeated line, and how to debug it? Currently, I'm stuck at this point. |
I will take a look. |
|
Hi @owen05 Please take a look at these records: They are duped records with exactly same in all columns. You may need to add either |
| {% endif %} | ||
|
|
||
| UNION ALL | ||
|
|
||
| -- DODOFeeRouteProxy | ||
| SELECT | ||
| evt_block_time AS block_time, | ||
| 'DODO' AS project, | ||
| 'DODO X' AS project, |
There was a problem hiding this comment.
@owen05 I would suggest that you add a proxy field here, to distinguish records.
The proxy field may help you to identify where the dupes come from.
There may be overlapping records, like proxy calls each other.
|
hi @owen05 -- we are looking to finalize the migration of |
|
The dupes may be produced during left join, as there are no dupes in source table. The simplest way may be an |
if you have time to pick this up and try to fix, please do, otherwise we'll take it up on our side soon |
| fromAmount AS token_bought_amount_raw, | ||
| returnAmount AS token_sold_amount_raw, | ||
| cast(NULL as double) AS amount_usd, | ||
| fromToken AS token_bought_address, | ||
| toToken AS token_sold_address, | ||
| contract_address AS project_contract_address, | ||
| evt_tx_hash AS tx_hash, | ||
| CAST(ARRAY() as array<bigint>) AS trace_address, | ||
| CAST(NULL as array<bigint>) AS trace_address, |
There was a problem hiding this comment.
Problem identified:
null comparison involved in the incremental updates.
There was a problem hiding this comment.
Just confirmed with @owen05, all trace_address in dodo_aggregator.trades are null.
I have tried to fix the dupes issue. |
amazing! much appreciated. i will look to finalize soon |
| fromAmount AS token_bought_amount_raw, | ||
| returnAmount AS token_sold_amount_raw, | ||
| cast(NULL as double) AS amount_usd, | ||
| fromToken AS token_bought_address, | ||
| toToken AS token_sold_address, | ||
| contract_address AS project_contract_address, | ||
| evt_tx_hash AS tx_hash, | ||
| CAST(ARRAY() as array<bigint>) AS trace_address, | ||
| CAST(NULL AS ARRAY<bigint>) AS trace_address, |
There was a problem hiding this comment.
i think rather than force null and have incremental logic break, we should standardize to match other models in the dex_aggregator.trades lineage to default to ARRAY[-1] as trace_address, then re-add to the unique key, so that it's consistent across all models in lineage
this is based on other aggregators i've seen migrated recently, where the -1 was used
|
Workflow run id 5977336515 approved. |
|
Workflow run id 5977336402 approved. |
jeff-dude
added
ready-for-merging
and removed
in review
Migrate DODO to DuneSQL and change the name of DODO Aggregator from 'DODO' to 'DODO X'
Thank you for contributing to Spellbook!
Please refer to the top of the
readmein the root of Spellbook to learn how to contribute to Spellbook on DuneSQL.