Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Set up Estuary Storage Provider #855

Open
10d9e opened this issue Jan 10, 2023 · 1 comment
Open

Set up Estuary Storage Provider #855

10d9e opened this issue Jan 10, 2023 · 1 comment
Labels
New Feature Issues that we will work on with people or ourselves

Comments

@10d9e
Copy link
Contributor

10d9e commented Jan 10, 2023

Setting up our own SP is a necessary first step in offloading and decentralizing Estuary's data.

  • dog-fooding our own Storage Provider will allow us to feel the same pain as our current SPs
  • fix and react to changes faster
  • potentially provide our own certified software to SPs, steward our users’ data
  • serve to bootstrap an Estuary/Filecoin Retrieval Market

Once we have determined a root SP (or set of root SPs), one that should be owned by Estuary/Protocol Labs, we have to hard code all deals to flow into it, reducing the total deals by one. It is also very important that the retrieval deals are compensated back to Estuary to demonstrate the economics of the system (and help offset the price of running SP infra).

@10d9e 10d9e added the New Feature Issues that we will work on with people or ourselves label Jan 10, 2023
@10d9e 10d9e added this to the Retrievals Q1 milestone Jan 10, 2023
@jcace
Copy link
Contributor

jcace commented Jan 10, 2023

We need to determine

  • What scale do we want the SP node to be? Seagate makes an excellent storage device in a 2PB size, which could be a good starting point. If we are storing unsealed copies for everything then it'll be closer to ~0.8PiB available storage for deals. Need to land on an architecture where we can scale, if needed.
  • How will we seal? There are high capital costs associated with sealing hardware. As Sealing-As-A-Service / Remote Sealing becomes available, it is a much more flexible option as we can pay per-sector as required, and don't need to own/operate the hardware.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
New Feature Issues that we will work on with people or ourselves
Projects
Status: No status
Development

No branches or pull requests

2 participants