Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Proposal: nft.storage for NFTHack #62

Merged
merged 8 commits into from
Mar 30, 2021
Merged

Conversation

alanshaw
Copy link
Member

@alanshaw alanshaw commented Mar 2, 2021

This is a proposal for time limited storage of NFT data on IPFS, backed by Filecoin and provided free to NFTHack participants during the hackathon.

Rendered

@github-actions github-actions bot requested a review from mikeal March 2, 2021 11:37
Copy link

@ribasushi ribasushi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!
Probably a few devils will be in the details, but that's to be hashed out during design.

Copy link
Member

@lidel lidel left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This proposal is fully packed! Provides clear onboarding path for NFT developers that:

  • use CIDs identifiers for addressing assets
  • leverage IPFS for hot storage and data retrieval
  • provides a demonstration how Filecoin-backed pinning service could look like
  • promotes future-proof api with no vendor-lock in:
    • makes pinning services api the standard for long term persistence
    • developers can switch from nft.storage to Pinata or something else without changing anything in their code (!)
  • current feature gaps like adding pin.remote commands to js-ipfs are solved as implementation detail of this proposal.


This will either be a proxy to [Pinata](https://pinata.cloud/) (or other pinning service) or use PL's own [pinbot](https://twitter.com/ipfspin).

This gives us space to experiment. Data storage and retrieval is de-risked by the service we pin the data to. It affords us a playground with "_real_" data that we can store on Filecoin.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Super crucial! I think right now a lot of our collabs (e.g Foundation) are looking to us to have well designed solutions that they can drop into - and are looking to us to define how these things shoudl work. Being able to experiment here would be super valuable!


* Create a library that automatically makes deals on Filecoin for the pinned data (use/adapt the dealbot?). This is mentioned in the [AWS Facade proposal](https://github.com/protocol/web3-dev-team/pull/34) and could be re-used there.

* Implement a [deal batching service](https://github.com/protocol/web3-dev-team/pull/60) prototype for increased successful deal probability.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is going to be super important as more folks like Textile/Fleek are enabling IPFS -> Filecoin services. Having a reference implementation would be powerful.

One note - we should apply for DataCap with Andrew ( through https://plus.fil.org so we can be set up to make deals if we need to well in advance!)


* Implement a [deal batching service](https://github.com/protocol/web3-dev-team/pull/60) prototype for increased successful deal probability.

* Run a "[Free Retrieval via IPFS](https://github.com/protocol/web3-dev-team/pull/52) Lotus node" that `nft.storage` can make deals with and that will expose the NFT data to IPFS.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps related to the next two bullet points, but should we consider also trying to get an MVP service for "CID/Deal querying"?

I think one pattern that has emerged with NFTs is we have these end users (you or me) who use an app like foundation who will manage the storage of data as a blackbox-> but we run into the "if X was hit by a bus, do I lose my data".

One answer to that ( which I think @rvagg touched on for his batching proposal) - was potentially exposing the CID/Deal info so a user could grab that externally (and take over retrieval and payment) - is it worth trying to also prototype something lightweight here?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think one pattern that has emerged with NFTs is we have these end users (you or me) who use an app like foundation who will manage the storage of data as a blackbox-> but we run into the "if X was hit by a bus, do I lose my data".

I'm not sure I understand what X is in this context. The storage provider? I would assume that the remote pinning service is doing it's best not to lose my data. Just like I assume Dropbox is doing their best to not lose my data. Perhaps this means multiple redundant copies are stored on Filecoin and I can see value in a library what negotiates deals to achieve this (RAID for Filecoin, except D - Disk is instead Deal 😉).

In IPFS this problem is alleviated a little - anyone that has ever accessed your CID will become a host for it (until they next GC) and can pin it themselves to keep the content alive on the network forever. So even if you only pin to a single provider that gets hit by a bus then there's a chance you'll still be able to get your content from someone else in the network.

One answer to that ( which I think @rvagg touched on for his batching proposal) - was potentially exposing the CID/Deal info so a user could grab that externally (and take over retrieval and payment) - is it worth trying to also prototype something lightweight here?

So, you'll have the CID of the content immediately. I don't think that it would be possible to take over the deal but you could negotiate a deal yourself with a different miner for the same content.

Assuming the content is currently retrievable, there's nothing stopping you from pinning it yourself to a backup pinning service or your own IPFS node.

In the remote pinning service API you can query the status of a pinned CID. I think it would be a good idea to enhance that response to include details of the Filecoin deal it is currently backed up in so that it can still be retrieved even if the pinning service went away.


We may see an uptick in the number of developers working with NFTs that use IPFS for storage.

The component part of actually storing the NFT data on Filecoin has invaluable potential to inform us on real world usage of the API's involved with making a deal. It should enable us to more confidently build solutions knowing they are the right thing to build.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1000

@mikeal
Copy link
Contributor

mikeal commented Mar 2, 2021

This is a fantastic proposal 😍

@olizilla
Copy link
Contributor

olizilla commented Mar 3, 2021

I love it! I would add the work we did on cube as a reference... having a UI where any and all members of the community can chose to contribute to the hosting cost for a given CID, and you can see how many years it has on clock, would be rad. The first n years could be covered automagically.


* Register `nft.storage` (or other cool domain name) and expose a [pinning service API](https://blog.ipfs.io/2021-02-19-go-ipfs-0-8-0/#remote-pinning-services).

This will either be a proxy to [Pinata](https://pinata.cloud/) (or other pinning service) or use PL's own [pinbot](https://twitter.com/ipfspin).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should proxy to Pinata. I don't want the ecosystem to feel like we're creating a tool that eats someone else's business and reduces their ability to serve users in the ecosystem

@momack2 momack2 added this to w3dt: Active in Project Proposals Mar 11, 2021
@mikeal mikeal added this to Needs Owner in Project Pipeline Mar 15, 2021
@mikeal mikeal moved this from Needs Owner to In Progress in Project Pipeline Mar 15, 2021
@mikeal mikeal added the Nitro NFT Free for All label Mar 30, 2021
@mikeal mikeal merged commit d989466 into main Mar 30, 2021
@mikeal mikeal deleted the alanshaw/nft-storage-for-nfthack branch March 30, 2021 23:12
@mikeal mikeal moved this from In Progress to Completed in Project Pipeline Mar 31, 2021
@jacobheun jacobheun moved this from Completed to Shipped in Project Pipeline Jun 9, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Nitro NFT Free for All
Projects
Project Proposals
w3dt: In Progress
Development

Successfully merging this pull request may close these issues.

None yet

7 participants