KodaDot - AssetsHub NFT indexer - Milestone 1 #1864

yangwao · 2023-07-25T11:51:33Z

Project Abstract

The AssetHub Indexer is a GraphQL service designed to streamline the interaction with Uniques, NFT non-fungible assets, and fungible assets on AssetHubs (Statemine/Statemint). This service is specifically tailored to serve as a robust data layer for NFT-oriented dApps, leveraging the simplicity of GraphQL and REST API.

In the current landscape, developers face challenges interacting with Uniques and Assets on Statemine due to the complexity and time-consuming nature of querying fungible and non-fungible assets on AssetHub. AssetHub Indexer aims to address these challenges by providing a user-friendly GraphQL interface, thus reducing the time and effort required to query these assets.

The key advantages of the AssetHub Indexer include

its focus on ease of use,
versatility for a broad range of use cases, and
its commitment to bringing Assets Hub (Statemine & Statemint) utilization & documentation closer to developers.

Furthermore, the AssetHub Indexer is a completely decentralized, open-source solution that respects user privacy by not collecting user data.

By reducing the time required to query assets and providing a more user-friendly interface, the AssetHub Indexer aims to foster the growth and development of the Web 3.0 ecosystem in Polkadot.

Grant level

Level 2: Up to $30,000, 3 approvals

Application Checklist

The application template has been copied and aptly renamed (kodadot_assethub_nft_indexer_statemine_statemint.md).
I have read the application guidelines.
Payment details have been provided (bank details via email or BTC, Ethereum (USDC/DAI) or Polkadot/Kusama (USDT) address in the application).
The software delivered for this grant will be released under an open-source license specified in the application.
The initial PR contains only one commit (squash and force-push if needed).
The grant will only be announced once the first milestone has been accepted (see the announcement guidelines).

CLAassistant · 2023-07-25T11:51:38Z

All committers have signed the CLA.

Polkadot-Forum · 2023-07-26T15:24:26Z

This pull request has been mentioned on Polkadot Forum. There might be relevant details there:

https://forum.polkadot.network/t/polkadot-nft-assets-hub-indexer-by-kodadot/3528/1

semuelle

Thanks for the thorough application, @yangwao! I will share it with the rest of the committee. I just have two questions:

Since you mentioned that KodaDot is already using the indexer, could you expand on which parts of the project are already implemented and which are still work in progress?
To simplify the evaluation, could you compress the list of deliverables? We don't need a separate deliverable for every event. For example, 3a and b, 4a to f and 6a to e could be one deliverable each.

yangwao · 2023-08-04T19:32:26Z

Since you mentioned that KodaDot is already using the indexer, could you expand on which parts of the project are already implemented and which are still work in progress?

You can have a realtime check here - https://github.com/kodadot/stick/milestone/1

2. To simplify the evaluation, could you compress the list of deliverables? We don't need a separate deliverable for every event. For example, 3a and b, 4a to f and 6a to e could be one deliverable each.

From our experience, the best is to have micro deliverables that the w3f reviewer can quickly evaluate.

Noc2

Thanks for the application, and sorry for the delay here. I'm generally happy to support the project, but could you reduce the price? The current FTE of 4, for example, seems too high to me, considering your current progress as well as the fact that so far, only three people contributed to the project. Reducing the grant to a level 2 project would also make it easier to accept it ;-)

semuelle

I agree with David. Twelve person months seems a lot for the scope.

semuelle · 2023-08-15T07:51:02Z

Are you planning to make any further changes to the application, @yangwao, or should we continue reviewing as is?

yangwao · 2023-08-15T08:56:28Z

considering your current progress as well as the fact that so far, only three people contributed to the project

There will be more people working on this.

Reducing the grant to a level 2 project would also make it easier to accept it

Yes, for now, it makes sense to split.

Are you planning to make any further changes to the application

Will be updating this week.

keeganquigley · 2023-08-15T21:21:58Z

Thanks @yangwao feel free to ping us when we should take another look at it.

semuelle · 2023-08-28T19:30:02Z

Will be updating this week.

any updates, @yangwao?

yangwao · 2023-09-01T08:36:53Z

Sure. I've been OOO, will look at the upcoming days

github-actions · 2023-09-06T10:36:25Z

CLA Assistant Lite bot All contributors have signed the CLA ✍️ ✅

yangwao · 2023-09-06T10:37:29Z

I have read and hereby sign the Contributor License Agreement.

yangwao · 2023-09-06T10:42:39Z

Hey, @semuelle @keeganquigley @Noc2, I've split our proposal into Milestones.
Now, we will ask only for Milestone 1, which is already done live in production.
It's used by KodaDot, PlutoWallet and soon by others.

Ref

https://github.com/kodadot/stick/milestone/1
KodaDot using Stick in the production of Kusama Assets Hub and Polkadot Assets Hub
- https://kodadot.xyz/ahk/explore/items
- https://kodadot.xyz/ahp/explore/items

semuelle

Thanks for the update, @yangwao. Please see some questions below.

applications/kodadot_assethub_nft_indexer_statemine_statemint.md

semuelle · 2023-09-06T21:46:35Z

applications/kodadot_assethub_nft_indexer_statemine_statemint.md

+| 9.      | Metadata Support for Fungible Assets | Implement a handler to add metadata to a fungible asset event from the chain.                          |
+| 9c.   | Fungible Asset Allowlist Setup        |Setting up allows list-based indexing of fungible assets.                                                       |
+| 10a.  | Data Views Development                |Construction of data views for efficient access to indexed data.                                               |
+| 10b.  | Implementing Metadata Caching Layer |Develop and retry IPFS metadata if un-indexed by 7j.                                                               |


What does 'unindexed' mean here? Unpinned?

I think this means that we will store those data if particular IPFS hashes are not pinned by original author for better UX.

What does 'unindexed' mean here? Unpinned?

We have come up eager approach of fetching metadata from ipfs (aka to index). If indexer fails to do that in 8 seconds metadata are unidexed and we try again after some period of time?

This is usefull because end users of this indexer does not nees to suffer high costs of IPFS gateways.

applications/kodadot_assethub_nft_indexer_statemine_statemint.md

semuelle · 2023-09-11T14:23:51Z

Please ping me here when the changes relevant to my requests have been made, @yangwao.

yangwao · 2023-09-12T11:54:05Z

@semuelle please have check.

semuelle

Hi @yangwao. Thanks for the updates. However, I am slightly confused as the issues listed in the M1 tracker have been closed for six weeks, and the commits referenced in them are largely from April and May. Since we don't do retroactive funding, I'd suggest you apply for M2 instead.

vikiival · 2023-09-14T12:11:14Z

Since we don't do retroactive funding, I'd suggest you apply for M2 instead.

Hello @semuelle.
sorry for confusion this proposal is aimed to AssetHub NFT indexer with ArrowSquid, which was released two days ago.

I am slightly confused as the issues listed in the M1 tracker have been closed for six weeks

Yes that was a proof-of-work that we are able to deliver good quality software.

largely from April and May

Yes, all this is implemented using FireSquid technology that would come to EOL in 3-4 months.

FireSquid vs. ArrowSquid aren't the same

The ArrowSquid is using a batch processor, that requires totally different data handling as FireSquid. Due to the nature of the ArrowSquid we are able do the processing more effectively but need to rewrite the code from scratch.

TL;DR

Prior this proposal the M1 was aimed as a proof of concept to present the successful delivery by our team. We aim to use latest tech (ArrowSquid) that requires reimplemetation of the current indexer.

semuelle

Thanks for the updates, @yangwao, and sorry for the late reply. Could you add the details of your previous comment regarding ArrowSquid into the application? Currently, it's not mentioned at all. Also, you removed the 5k USD allocation as deliverable, but the total costs remained the same. I would have assumed that it would be reduced correspondingly.

semuelle · 2023-09-25T18:02:06Z

pinging @yangwao & @vikiival

vikiival · 2023-09-25T19:26:12Z

pinging @yangwao & @vikiival

@semuelle is the proposal ready to be merged?

💰 define to use arrowsquid and update final price yangwao/Grants-Program#2

keeganquigley · 2023-09-25T19:41:05Z

Thanks @vikiival the proposal can't be merged until it reaches 3 approvals. However I see your changes haven't been merged yet by @yangwao, I think these changes need to be merged into the application first in order to move forward with it.

yangwao · 2023-09-26T11:01:23Z

Hello @semuelle @keeganquigley please have check

vikiival · 2023-09-26T14:08:23Z

I have read and hereby sign the Contributor License Agreement.

keeganquigley

Thanks @vikiival @yangwao although looks like the application is still missing an Overview section. Can you add this in?

Also, the numbering in the milestone tables is missing 6, therefore it doesn't make sense as is. Can you fix the numbering scheme and perhaps consolidate some of the deliverables so that there aren't so many? For example, 4a. - 4f. could probably be combined into one section, as well as 7a. - 7k. Also section 9 doesn't follow the same letter scheme that the rest of the deliverables have.

📝 unify deliverables per assethub indexer

yangwao · 2023-10-05T12:33:04Z

you removed the 5k USD allocation as deliverable, but the total costs remained the same. I would have assumed that it would be reduced correspondingly.

Hey thanks for your assumption @semuelle, it was supposed to be removed as residues from the first version of the proposal, we've realized that implementation costs are slightly higher in ArrowSquid, than we've anticipated, thus we are taking max allowance on this stage to at least cover our implementation costs.
Hope it's OK for you as we are a trusted delivery partner in the Polkadot ecosystem with 7 indexers [1], [2], [3], [4]. The latest one will be used by Talisman and NovaWallet. So we see it as an ecosystem public good with utility.

Can you add this in?

It's added now.

Can you fix the numbering scheme and perhaps consolidate some of the deliverables so that there aren't so many?

Improved now.

Let us know @keeganquigley when you review it.

keeganquigley

Thanks for the changes @yangwao I think it would be great to have an indexer for AssetHub. Also based on Kodadot's standing in the community I'm willing to go forward with it. I will mark the application as ready for review and ping the rest of the committee.

semuelle

Thanks for the updates, @yangwao. I am also willing to go ahead with it.

github-actions · 2023-10-10T09:19:01Z

Congratulations and welcome to the Web3 Foundation Grants Program! Please refer to our Milestone Delivery repository for instructions on how to submit milestones and invoices, our FAQ for frequently asked questions and the support section of our README for more ways to find answers to your questions.

Before you start, take a moment to read through our announcement guidelines for all communications related to the grant or make them known to the right person in your organisation. In particular, please don't announce the grant publicly before at least the first milestone of your project has been approved. At that point or shortly before, you can get in touch with us at grantsPR@web3.foundation and we'll be happy to collaborate on an announcement about the work you’re doing.

Lastly, please remember to let us know in case you run into any delays or deviate from the deliverables in your application. You can either leave a comment here or directly request to amend your application via PR. We wish you luck with your project! 🚀

* Added KodaDot Grants Ecosystem * Create kodadot_assethub_nft_indexer_statemine_statemint.md * Update markdown * Milestone1 - KodaDot Assets Hub * formatting * removing residues from first version * 💰 define to use arrowsquid and update final price * 📝 unify deliverables per assethub indexer * 📝 numbering per deliverables --------- Co-authored-by: Viki Val <viktorko99@gmail.com>

yangwao added 5 commits May 30, 2023 14:59

Added KodaDot Grants Ecosystem

c7f48f2

Merge pull request #1 from yangwao/yangwao-patch-1

e0ac086

Merge branch 'w3f:master' into master

4c2d480

Create kodadot_assethub_nft_indexer_statemine_statemint.md

57b52e3

Update markdown

85507dc

This was referenced Jul 25, 2023

Make PR - https://wiki.polkadot.network/docs/learn-assets#non-fungible-assets kodadot/nft-gallery#5995

Open

Make PR with KodaDot demo - https://wiki.polkadot.network/docs/learn-nft-pallets kodadot/nft-gallery#5996

Open

semuelle self-assigned this Jul 28, 2023

semuelle requested changes Aug 4, 2023

View reviewed changes

semuelle added the ready for review The project is ready to be reviewed by the committee members. label Aug 4, 2023

Noc2 requested changes Aug 7, 2023

View reviewed changes

semuelle reviewed Aug 9, 2023

View reviewed changes

keeganquigley added changes requested The team needs to clarify a few things first. and removed ready for review The project is ready to be reviewed by the committee members. labels Aug 15, 2023

Milestone1 - KodaDot Assets Hub

7c3cc45

yangwao changed the title ~~KodaDot - AssetsHub NFT indexer~~ KodaDot - AssetsHub NFT indexer - Milestone 1 Sep 6, 2023

formatting

87cb2ef

semuelle requested changes Sep 6, 2023

View reviewed changes

removing residues from first version

c8c41e6

yangwao requested a review from semuelle September 12, 2023 13:52

semuelle reviewed Sep 14, 2023

View reviewed changes

semuelle requested changes Sep 20, 2023

View reviewed changes

💰 define to use arrowsquid and update final price

63b94c2

Merge pull request #2 from vikiival/patch-1

a4a76cb

keeganquigley requested changes Sep 26, 2023

View reviewed changes

vikiival and others added 3 commits September 27, 2023 14:03

📝 unify deliverables per assethub indexer

c3f8b4d

📝 numbering per deliverables

8c6bca3

Merge pull request #3 from vikiival/patch-1

85baead

📝 unify deliverables per assethub indexer

keeganquigley approved these changes Oct 5, 2023

View reviewed changes

keeganquigley added ready for review The project is ready to be reviewed by the committee members. and removed changes requested The team needs to clarify a few things first. labels Oct 5, 2023

semuelle approved these changes Oct 10, 2023

View reviewed changes

Noc2 approved these changes Oct 10, 2023

View reviewed changes

Noc2 merged commit 5259b0d into w3f:master Oct 10, 2023
8 of 9 checks passed

vikiival mentioned this pull request Jan 29, 2024

KodaDot: AssetHub NFT indexer M1 w3f/Grant-Milestone-Delivery#1121

Merged

5 tasks

vikiival mentioned this pull request Apr 3, 2024

KodaDot - Atomic Swap Indexer & API for AssetHub NFTs #2271

Merged

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

KodaDot - AssetsHub NFT indexer - Milestone 1 #1864

KodaDot - AssetsHub NFT indexer - Milestone 1 #1864

yangwao commented Jul 25, 2023 •

edited

CLAassistant commented Jul 25, 2023 •

edited

Polkadot-Forum commented Jul 26, 2023

semuelle left a comment

yangwao commented Aug 4, 2023

Noc2 left a comment

semuelle left a comment

semuelle commented Aug 15, 2023

yangwao commented Aug 15, 2023

keeganquigley commented Aug 15, 2023

semuelle commented Aug 28, 2023

yangwao commented Sep 1, 2023

github-actions bot commented Sep 6, 2023 •

edited

yangwao commented Sep 6, 2023

yangwao commented Sep 6, 2023

semuelle left a comment

semuelle Sep 6, 2023

yangwao Sep 11, 2023

vikiival Sep 11, 2023

semuelle commented Sep 11, 2023

yangwao commented Sep 12, 2023

semuelle left a comment

vikiival commented Sep 14, 2023

semuelle left a comment

semuelle commented Sep 25, 2023

vikiival commented Sep 25, 2023

keeganquigley commented Sep 25, 2023

yangwao commented Sep 26, 2023

vikiival commented Sep 26, 2023

keeganquigley left a comment

yangwao commented Oct 5, 2023

keeganquigley left a comment

semuelle left a comment

github-actions bot commented Oct 10, 2023

KodaDot - AssetsHub NFT indexer - Milestone 1 #1864

KodaDot - AssetsHub NFT indexer - Milestone 1 #1864

Conversation

yangwao commented Jul 25, 2023 • edited

Project Abstract

The key advantages of the AssetHub Indexer include

Grant level

Application Checklist

CLAassistant commented Jul 25, 2023 • edited

Polkadot-Forum commented Jul 26, 2023

semuelle left a comment

Choose a reason for hiding this comment

yangwao commented Aug 4, 2023

Noc2 left a comment

Choose a reason for hiding this comment

semuelle left a comment

Choose a reason for hiding this comment

semuelle commented Aug 15, 2023

yangwao commented Aug 15, 2023

keeganquigley commented Aug 15, 2023

semuelle commented Aug 28, 2023

yangwao commented Sep 1, 2023

github-actions bot commented Sep 6, 2023 • edited

yangwao commented Sep 6, 2023

yangwao commented Sep 6, 2023

Ref

semuelle left a comment

Choose a reason for hiding this comment

semuelle Sep 6, 2023

Choose a reason for hiding this comment

yangwao Sep 11, 2023

Choose a reason for hiding this comment

vikiival Sep 11, 2023

Choose a reason for hiding this comment

semuelle commented Sep 11, 2023

yangwao commented Sep 12, 2023

semuelle left a comment

Choose a reason for hiding this comment

vikiival commented Sep 14, 2023

semuelle left a comment

Choose a reason for hiding this comment

semuelle commented Sep 25, 2023

vikiival commented Sep 25, 2023

keeganquigley commented Sep 25, 2023

yangwao commented Sep 26, 2023

vikiival commented Sep 26, 2023

keeganquigley left a comment

Choose a reason for hiding this comment

yangwao commented Oct 5, 2023

keeganquigley left a comment

Choose a reason for hiding this comment

semuelle left a comment

Choose a reason for hiding this comment

github-actions bot commented Oct 10, 2023

yangwao commented Jul 25, 2023 •

edited

CLAassistant commented Jul 25, 2023 •

edited

github-actions bot commented Sep 6, 2023 •

edited