Fixing Solana NFT Token Classification + Retrieving Missing DEX Trades

[fergmolina]

Description:

We were investigating some missing trades from the dex_solana.trades table. Example: Solscan Link

We found out that these trades were being filtered because, in the dex_solana.trades dbt transformation. We have the condition 'fungible' (source). This is in fact a fungible token, so it shouldn't be filtered.

The problem is that the solana_utils_token_accounts dbt transformation is marking some account tokens as 'nft' when it shouldn't be (source).

The exact issue lies in the tokens_solana_nft transformation:
According to Metaplex docs, inside the createArgs, we can find the following TokenStandards:

• 0 / NonFungible: A non-fungible token with a Master Edition.
• 1 / FungibleAsset (1): A token with metadata that can also have attributes, sometimes called Semi-Fungible.
• 2 / Fungible (2): A token with simple metadata.
• 3 / NonFungibleEdition (3): A non-fungible token with an Edition account (printed from a Master edition).
• 4 / ProgrammableNonFungible (4): A special NonFungible token that is frozen at all times to enforce custom authorization rules.

There are no filters in the CTE query executed against the mpl_token_metadata_solana.mpl_token_metadata_call_create table source, so it retrieves all token accounts, including fungible tokens like the example below.

Solution:

Adding a WHERE condition to filter only token accounts related to non-fungible tokens will ensure that these fungible

---

quick links for more information:

• README.md
• spellbook docs
• CONTRIBUTING.md

[github-actions]

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

[fergmolina]

I have read the CLA Document and I hereby sign the CLA

[fergmolina]

recheck

[jeff-dude]

thank you for proposing this fix. i wanted to ensure you were aware and able to test the output of your change, in the table built during CI builds attached to PR. you can find table name here, then query on dune to test it's as expected.

since this is upstream of many important solana spells, we should be doing in-depth testing here to ensure it does what is intended. in addition to testing the table directly, we can also compile queries from downstream spells which join to it to ensure results downstream are also as expected. it looks like you have a bit of the lineage in mind in the PR description, so would be nice if we could test those with this CI table in place of the prod table for joins. we can filter on just a few days of data to make queries run a bit faster to iterate checks.

for a visual of what is all downstream of this, can see list here. we don't need to test them all, but ones such as solana_utils_token_accounts or raydium v5 spell.

[fergmolina]

@jeff-dude the fixed worked as expected and is filtering now the Fungible Token in the example:

New model:

select *
from test_schema.git_dunesql_8e3f361_tokens_solana_nft
WHERE account_mint = 'Bo9jh3wsmcC2AjakLWzNmKJ3SgtZmXEcSaW7L2FAvUsU'

Old model:

select *
from solana_utils.tokens_solana_nft
where account_mint = 'Bo9jh3wsmcC2AjakLWzNmKJ3SgtZmXEcSaW7L2FAvUsU'

But the problem is that I still see some Fungible tokens, probably coming from the other CTEs.

select distinct token_standard
from test_schema.git_dunesql_8e3f361_tokens_solana_nft

I changed the way to filter the Fungible Tokens in my last commit. After this new model is finished to compile, can you please compile also raydium_v5.base_trades, raydium_v5.trades, tokens_solana.nft, solana_utils.token_accounts, dex_solana.trades ?

[fergmolina]

Last model is working as expected:

select distinct token_standard
from test_schema.git_dunesql_5e8e81e_tokens_solana_nft

[jeff-dude]

on your local setup, you can run dbt compile in the solana subproject directory and have all queries built. from there, you can navigate to target directory and find the few spells downstream, then copy/paste on to dune and find/replace this table with the CI version. again, to get to run on the app, i'd recommend hardcoding a few date filters to limit rows

[jeff-dude]

closing in favor of #8042
refer to this comment where boxer mentioned the PR will fix this issue.