[EPIC] BlobModule

[Wondertan]

Context

So far, celestia-node has developed all the infrastructure and necessary components from the ground up to provide a secure and scalable DA network. Now, the time has come to create that DA module and its API basing, using which any application on top can put data, retrieve and verify its availability. We call this module BlobModule.

BlobModule API

Let's define the API for it. For simplicity and to follow our practice, we define it in Golang.

NOTE: The API is WIP and discussion towards it's finalization happens in #1829

// Blob represents any application-specific binary data that anyone can submit to Celestia.
type Blob struct {
	Namespace namespace.ID
	Data []byte
}

// Commit computes hash commitment of the Blob.
func (b *Blob) Commit() []byte

// BlobModule provides all necessary
type BlobModule interface {
	// Submit sends PFB transaction and reports the height it landed on.
	// Allows sending multiple Blobs atomically synchronously.
	// Uses default wallet registered on the Node.
	Submit(context.Context, Blob...) (height uint64, error)

	// Get retrieves all the blobs for given namespaces at the given height. 
	Get(context.Context, height uint64, namespace.ID...) ([]Blob, error)
	
	// Available checks whether the given hash commitment of a blob is available/included at the given height under a particular namespace.
   // Useful for light clients wishing to check the inclusion of the data.
	Available(context.Context, height uint64, namespace.ID, hash []byte]) (bool, error)
}

Tasks

Beta Give feedback

1. share: GetLeavesByNamespace should allow returning only proofs without data #1836
   area:shares +1
   
2. shrexnd: requesting proofs only on Client and Server #1837
   area:shares +1
3. share: Extend Getter to enable requesting proofs and adjust all the implementations #1838
   area:shares +1
4. shrexnd: requesting data and/or proofs for multiple namespaces #1839
   area:shares +1
5. state: SubmitPayForBlob should support multiple blobs #1840
   area:state +1
6. modblob: scaffold BlobModule and it's API #1841
   4 of 5
   area:blob +1
7. Feature Request: document nmt proof format nmt#248
   documentation +1

[vgonkivs]

• refactoring(share/ipld)!: rework getLeavesByNamespace #1870

[distractedm1nd]

I would suggest moving the rest of the open issues to #2287 and closing this, so that we only have one blob module epic open

[Wondertan]

Agreed. @vgonkivs, mind making the suggestion?

[vgonkivs]

Will move all tasks to the Hardening EPIC

[ramin]

closing as remaining tasks moved to "the hardening" workstream/epic #2287