CODEX-MARKETPLACE: New Raw RFC #36

jimstir · 2024-04-30T01:31:22Z

A raw RFC for Codex marketplace design. Open discussion to create a Codex node RFC which should be referenced in this RFC and future Codex RFCs.

Implementation Reference: nim-codex

jimstir · 2024-05-02T00:32:34Z

codex/marketplace.md

+| --------------- | --------- |
+| Storage Nodes | A Codex node that provides storage services to the marketplace.|
+| Validator Nodes | A Codex node that collects, validates and submits proofs to reward or penalize other storage nodes or validator nodes. |
+| Regular Nodes | The main Codex client that interacts with other nodes to locate and retrieve data. Also considered to be an ephemeral node (light client) |


This is vaguely defined in the write up. I am proposing that the next RFC should be a Codex node raw rfc. The reference implementation: codex-node

This is likely a bit off. Are we defining specs for our current implementation or for the end result of what Codex will be in the future?

Codex node roles are defined in the (WIP) whitepaper and likely could be used here. Will DM the link to you.

Currently defining this spec based on current implementation, past and future designs can be mentioned.

codex/marketplace.md

emizzle

Great first draft, thank you for starting this!

Is this spec meant to represent the current implementation of Codex, or what it will be as a final product?

codex/marketplace.md

Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>

jimstir · 2024-05-16T00:19:17Z

codex/marketplace.md

+
+When storage providers are selected to fill a slot for the request,
+storage providers MUST NOT abandon the slot, unless the request is canceled or complete.
+If canceled, the slot state SHOULD be changed to cancelled or failed.


Is this correct, or is it just cancelled?

Not sure if cancelled is implemented as of now, but we should support it, in that case cancelled and failed are correct states.

jimstir · 2024-05-16T00:21:18Z

codex/marketplace.md

+Each slot represents a chunk of a dataset provided during storage request.
+The first state of a slot is reserved, meaning that the slot is waiting to be reserved by a storage provider.
+The Codex network selects storage providers by using a variation of the Kademila distance function, 
+as described in [CODEX/NODE-DISPERSAL](https://github.com/vacp2p/rfc-index/blob/codex-disperal/codex/slot-dispersal.md).


Proposed a separate RFC for dispersal, currently in draft. Should this stay separate or combine into this rfc? #39

codex/marketplace.md

AuHau

Great start, especially considerating you had to dig up all the details! IMHO it needs a lot of work to have it really in usable state. I will try to help you as much as I can to get it there. I would like to see few things clarified and few to work in.

Can you please elaborate "what are we specing"? Are we specing the smart contract or module implementation that interacts with the smart contracts? This should be clarified and also noted in the document. IMHO this spec should be the second.
Do you have experience writing spec with "external driver" like a smart contract? Very often some state is dictated by the smart contract and this happens "automatically". Quick question, how much are you familiar with smart contracts and how they work/operate? It would be good idea to find maybe some examples from other projects on how to write this sort of spec.
I would like to have some diagram about the states flow of purchasing (eq. client request storage) and sales (eq. storage provider fills the slot) to explain it better to the reader. An example: https://github.com/codex-storage/nim-codex/blob/8b12934fe242381df8a65ac6afd32c6e171dac4a/codex/purchasing/purchase.nim#L6

These are just my initial thoughts, I will do another pass of the review to give more inputs.

AuHau · 2024-05-13T07:24:45Z

codex/marketplace.md

+| Terminology  | Description |
+| --------------- | --------- |
+| Storage Providers | A Codex node that provides storage services to the marketplace. |
+| Validator Nodes | A Codex node that collects, validates and submits missing storage proofs for a reward. |


Suggested change

| Validator Nodes | A Codex node that collects, validates and submits missing storage proofs for a reward. |

| Validator Nodes | A Codex node that check for missing storage proofs and triggers for a reward. |

In the future, there will be Aggregator nodes that will collect and submit proofs on-chain. In the current design, Validators only check if there are missing proofs. If so, they mark them on-chain, which then triggers slashing and eventually leads to removing the failed host and starting the repair process.

AuHau · 2024-05-22T12:15:44Z

codex/marketplace.md

+client nodes MUST split the dataset into data chunks, $(c_1, c_2, c_3, \ldots, c_{20})$.
+Using a erasure coding technique, 
+the data chunks are encoded and placed into separate slots.
+The RECCOMENDED erasure coding technique is a [Reed-Soloman algorithm](https://hackmd.io/FB58eZQoTNm-dnhu0Y1XnA).


I am not so familiar with RFC terminology as this is my first main dive into "specs", but I am wondering about the RECOMMENDED part here. Currently, we support only this erasure coding technique. If somebody writes an alternative implementation of Codex and decides to use different erasure coding, they would be effectively non-compatible with "our" Codex as our implementation would not know how to reconstruct the data from the other implementation. Of course, at some point, we could add a new erasure coding technique, but IMHO then the RFC should be amended with the requirement that all implementations will implement the new erasure coding technique in order to maintain the network compatibility.

Soooo shouldn't this be a MUST?

Sorry I am editing my response from previously, it should be changed to a SHOULD instead of MUST because an implementor can still use a different encoding algorithm and create a slot. Even if the data can not be decoded, do storage providers check the same erasure coding is used before store the data? RECOMMENDED I assumed an implementer could use a different algorithm if desired.

AuHau · 2024-05-22T12:16:25Z

codex/marketplace.md

+Using a erasure coding technique, 
+the data chunks are encoded and placed into separate slots.


I guess the spec for how to do this exactly should be referenced in some other RFC?

I will open this for discussion at a later time. @dryajov

AuHau · 2024-05-22T12:25:03Z

codex/marketplace.md

+The requester MUST provide a `duration` value for the storage request along with the appropriate `reward`,
+payment for request. 


Are we specifying here required parameters for the storage request? Because there is quite much more. Pretty much all the parameters of Request (see here) object needs to be specified (eq. non-zero).

This describes the required values that a client should provide with a storage request transaction. Is it required that an implementor create the same objects as your reference implementation (here)?

AuHau · 2024-05-22T12:27:42Z

codex/marketplace.md

+all slots MUST be filled by storage providers before the request is officially started.
+If the request does not attract enough storage providers after a pre-defined network timeout,
+which is defined by the Codex node as `expiry`,
+the request MUST be canceled.


Wondering about the MUST here. This is done "automatically" by the smart contract, not the node. What needs to happen though, is that the Client node SHOULD withdraw back the remaining funds from the Request and the SPs that filled the slot before expiry SHOULD collect the partial payout for the time they were filling the slot. These actions are not required as the money can be kept in the smart contract indefinitely.

MUST here describes that the implementation must have all slots of a given request filled before the state is changed to Started, the objects RequestState object. I assumed that the Codex node set a predefined value for expiry instead of the requester creating a new value for each request. As for funds associated with the request, the next statement defines, "If canceled, collateral SHOULD be returned to any storage providers and
reward returned to the requester." Should this statement mention partial payout for time waiting for state change to Started?

I assumed that the Codex node set a predefined value for expiry instead of the requester creating a new value for each request.

Currently, the requester defines the timeout. The reasoning is that, the requester should define how long it wants to wait for it's contract to start, the minimum and maximum wait values are unspecified as of now, but its something we might want to consider.

Should this statement mention partial payout for time waiting for state change to Started?

Yes, nodes that have provided storage proofs during the "contract initiation" phase, should be compensated (at least this is the current thinking), and as such this should be captured.

Wondering about the MUST here. This is done "automatically" by the smart contract, not the node.

Regardless of who is initiating this it "MUST" happen.

AuHau · 2024-05-23T09:38:28Z

codex/marketplace.md

+
+Client nodes can create storage requests on the Codex network via the Codex marketplace.
+The marketplace is a a smart contract on a EVM compatiable blockchain.
+The smart contract SHOULD handle storage request, storage slot state, 


What does "SHOULD" mean in this context?

You are correct, this should be MUST.

AuHau · 2024-05-23T09:39:13Z

codex/marketplace.md

+storage provider rewards, storage provider collateral, and storage proof state. 
+
+To create a request to store a dataset on the Codex network,
+client nodes MUST split the dataset into data chunks, $(c_1, c_2, c_3, \ldots, c_{20})$.


Suggested change

client nodes MUST split the dataset into data chunks, $(c_1, c_2, c_3, \ldots, c_{20})$.

client nodes MUST split the dataset into data chunks, $(c_1, c_2, c_3, \ldots, c_{x})$.

We do not split data only into 20 chunks...

AuHau · 2024-05-23T10:00:39Z

codex/marketplace.md

+`reward` returned to the requester.
+The requester MAY create a new request with different values to restart the process.
+
+The storage request SHOULD initiate a new storage contract with the following parameters:


All these parameters are required so most probably it should be MUST?

MAY in this context refers to the requester's choice of creating a new transaction. SHOULD in this context refers to the requester providing the values for the request. In most cases a MUST is used for described the behavior of the application itself not the user because a user may not provide all the values.

codex/marketplace.md

AuHau · 2024-05-23T10:15:58Z

codex/marketplace.md

+The following is related to storage provider who has offered `collateral`
+
+If a storage provider, filling a slot,
+fails to provide enough proofs of storage, the `collateral` MUST be forfeited.


Again there is the question of "what are we specing here":

the collateral MUST be forfeited

It is not like the nodes MUST "forfeit" their collateral by doing some smart contract call. This is handled by the smart contract in the form of slashing. As smart contracts can not "act automatically" (eq. they automatically slash collateral when nodes miss their proof), these actions need to be triggered by somebody submitting transaction (and paying for it), we could potentially talk about who SHOULD/MUST do this triggering. But honestly I would leave that to separate section where we should spec how Validator node should behave.

Suggested change

fails to provide enough proofs of storage, the `collateral` MUST be forfeited.

miss the critical amount of submitted proofs of the storage, the `collateral` will be slashed. If a critical amount of `collateral` is slashed, then the slot is vacated and the repair process kicks in.

I agree forfeited may be the wrong term to use here. From my understanding slashing refers to a user losing a portion of their tokens. This should be mention as well as total lost of collateral when maxNumOfSlashing occurs.

I think forfeited is an appropriate term here, I also think that leaving the mechanics of how this is accomplished open ended is the right approach here. The fact that a contract cannot trigger actions internally and has to be externally initiated is an implementation detail, and a limitation of the underlying execution environment and should be captured in a different spec, possibly by the validator spec.

IMO, this spec should be generic enough, that it can be implemented on top of a different execution environment altogether.

So, I overall agree with the current wording.

dryajov · 2024-05-23T15:00:47Z

codex/marketplace.md

+| --------------- | --------- |
+| storage providers | A Codex node that provides storage services to the marketplace. |
+| validator nodes | A Codex node that collects, validates and submits missing storage proofs for a reward. |
+| client nodes | The most common Codex node that interacts with other nodes to store, locate and retrieve data. Also considered to be an ephemeral node (light client). |


An ephemeral node isn't a light client, in fact we avoid the usage of the term light client al together, due to it's ambiguity.

dryajov · 2024-05-23T15:48:37Z

codex/marketplace.md

+name: Codex Storage Marketplace
+status: raw
+tags: codex
+editor: Dmitriy <dryajov@status.im>


We should list Mark here as well (I can't tag him in this repo, he should be invited - https://github.com/markspanbroek)

@markspanbroek

dryajov · 2024-05-23T16:24:30Z

Great start, lets have Mark review this as well.

markspanbroek

Great work pulling this together @jimstir! Sorry that it took me a while to review it; I've had a lot of other things on my todo list. But here's my review, it looks good overall but I have some suggestions below.

markspanbroek · 2024-05-30T09:00:57Z

codex/raw/marketplace.md

+2. [CIDv1](https://github.com/multiformats/cid#cidv1)
+3. [Proof-of-Data-Possession](https://hackmd.io/2uRBltuIT7yX0CyczJevYg?view)
+4. [Codex market implementation](https://github.com/codex-storage/nim-codex/blob/master/codex/market.nim)
+5. [CODEX/NODE-DISPERSAL](https://github.com/vacp2p/rfc-index/blob/codex-disperal/codex/slot-dispersal.md)


The description of node dispersal is now moved inside this document, so we don't need this reference anymore, I think.

markspanbroek · 2024-05-30T09:01:41Z

codex/raw/marketplace.md

+Each slot represents a chunk of a dataset provided during the storage request.
+The first state of a slot is `reserved`, meaning that the slot is waiting to be reserved by a storage provider.
+The Codex network selects storage providers by using a variation of the Kademila distance function, 
+as described in [CODEX/NODE-DISPERSAL](https://github.com/vacp2p/rfc-index/blob/codex-disperal/codex/slot-dispersal.md).


This should become a reference to the paragraph below, instead of a reference to a separate document.

markspanbroek · 2024-05-30T09:02:10Z

codex/raw/marketplace.md

+tags: codex
+editor: Dmitriy <dryajov@status.im>
+contributors:
+- Mark <mark@status.im>


Suggested change

- Mark <mark@status.im>

- Mark <mark@codex.storage>

markspanbroek · 2024-05-30T09:03:00Z

codex/raw/marketplace.md

+status: raw
+tags: codex
+editor: Dmitriy <dryajov@status.im>
+contributors:


Should you not be in the list of contributors @jimstir?

markspanbroek · 2024-05-30T09:06:22Z

codex/raw/marketplace.md

+| --------------- | --------- |
+| storage providers | A Codex node that provides storage services to the marketplace. |
+| validator nodes | A Codex node that checks for missing storage proofs and triggers for a reward. |
+| client nodes | The most common Codex node that interacts with other nodes to store, locate and retrieve data. Also considered to be an ephemeral node. |


I would remove this last part about the ephemeral module. I don't think it helps to explain what a client node does, yet raises all sorts of questions that are not answered in this document.

markspanbroek · 2024-05-30T10:54:17Z

codex/raw/marketplace.md

+#### Slot Dispersal
+
+Storage providers compete with one another to store data from storage request. 
+Before a storage provider can download the data, they MUST be selected to obtain a reseversation of a slot.


Suggested change

Before a storage provider can download the data, they MUST be selected to obtain a reseversation of a slot.

Before a storage provider can download the data, they MUST obtain a reservation for a slot.

markspanbroek · 2024-05-30T10:55:30Z

codex/raw/marketplace.md

+
+Storage providers compete with one another to store data from storage request. 
+Before a storage provider can download the data, they MUST be selected to obtain a reseversation of a slot.
+The Codex network handles selection by using Kademlia distance function to select eligible storage providers per slot.


Suggested change

The Codex network handles selection by using Kademlia distance function to select eligible storage providers per slot.

The Codex network uses an expanding window based on the Kademlia distance function to select storage providers that are allowed to reserve a slot.

markspanbroek · 2024-05-30T10:56:49Z

codex/raw/marketplace.md

+
+    hash(blockHash, requestId, slotIndex, reservationIndex);
+
+`blockHash`: unique identifier for a specific block


The word "block" is rather ambiguous in our context, so I'd mention specifically that it's the Ethereum blockhash.

markspanbroek · 2024-05-30T10:58:15Z

codex/raw/marketplace.md

+The source address is used along with the storage provider's blockchain address to calculate the Kademlia distance.
+This is represented by:
+
+$$ XOR(A,A_0) $$


It's not clear to me what A and A_0 mean in this context.

markspanbroek · 2024-05-30T10:58:40Z

codex/raw/marketplace.md

+
+$$ XOR(A,A_0) $$
+
+The allowed distance over time $t_1$, can be defined as $2^{256} * F(t_1)$.


We should probably specify what F is here.

dryajov · 2024-05-30T17:51:17Z

codex/raw/marketplace.md

+## Motivation
+The Codex network aims to create a peer-to-peer storage engine with strong data durability, 
+data persistence guarantees and node storage incentives.
+To reach this goal, a data availability and retrieval mechanism is needed.


Not sure where this came from, Codex isn't about data availability, it is strictly about data durability.

I found this in the codex white-paper, data availability is mentioned often. "This makes data availability one of the principal requirements for decentralized storage. To reach that goal, Codex implements strong incentives and a robust data dispersion scheme that allows for extreme flexibility at the moment of accessing a dataset."

dryajov · 2024-05-30T17:52:56Z

codex/raw/marketplace.md

+The Codex network aims to create a peer-to-peer storage engine with strong data durability, 
+data persistence guarantees and node storage incentives.
+To reach this goal, a data availability and retrieval mechanism is needed.
+Support for light clients, like mobile devices should also be embraced.


Don't use light client, better use resource restricted devices.

Suggested change

Support for light clients, like mobile devices should also be embraced.

Support for resource restricted devices, like mobile devices should also be embraced.

dryajov · 2024-05-30T17:54:46Z

codex/raw/marketplace.md

+data persistence guarantees and node storage incentives.
+To reach this goal, a data availability and retrieval mechanism is needed.
+Support for light clients, like mobile devices should also be embraced.
+The protocol should remove complexity to allow for simple implementation and 


I would remove this phrasing. This isn't a concrete goal that can be properly quantified.

dryajov · 2024-05-30T18:05:16Z

codex/raw/marketplace.md

+  address client;
+
+  // content identifier
+  string cid;


I've noticed this before, but forgot to bring it up - we should not store the Cid as a string but as a byte array/string. It should be changed in the contract implementation and should be correctly captured here.

@markspanbroek @emizzle @AuHau

Good point, we should be using the binary format instead of the stringified form. Adam already created an issue for this: codex-storage/codex-contracts-eth#107

dryajov · 2024-05-30T18:07:16Z

codex/raw/marketplace.md

+- MUST be generated by the client node
+
+`reward`
+- total amount is REQUIRED in transaction to initiate storage request.


This wording is a bit weird, I'm not sure what it means.

Added this in response to feedback from mark, here. I'll word differently

jimstir added 5 commits April 25, 2024 20:36

Create marketplace.md

c714e07

Update marketplace.md

3710944

Update marketplace.md

9aa7f04

Update marketplace.md

3a7a30b

Update marketplace.md

1f7cfda

jimstir marked this pull request as draft April 30, 2024 01:31

jimstir added 3 commits April 30, 2024 22:35

Update marketplace.md

5c3d69f

Update marketplace.md

d84c62e

Update marketplace.md

b40b61b

jimstir marked this pull request as ready for review May 2, 2024 00:28

jimstir commented May 2, 2024

View reviewed changes

jimstir requested a review from dryajov May 2, 2024 00:38

jimstir commented May 2, 2024

View reviewed changes

codex/marketplace.md Outdated Show resolved Hide resolved

emizzle reviewed May 2, 2024

View reviewed changes

jimstir and others added 7 commits May 2, 2024 06:07

Update codex/marketplace.md

e13acac

Co-authored-by: Eric <5089238+emizzle@users.noreply.github.com>

Update marketplace.md

115d3ff

Update marketplace.md

8cf4866

Update marketplace.md

fe90d98

Update marketplace.md

b8df748

Update marketplace.md

5497f4d

Update marketplace.md

21394fe

jimstir commented May 16, 2024

View reviewed changes

jimstir requested a review from emizzle May 16, 2024 00:22

AuHau suggested changes May 23, 2024

View reviewed changes

dryajov reviewed May 23, 2024

View reviewed changes

jimstir added 2 commits May 23, 2024 11:24

Update marketplace.md

d7d813f

Update marketplace.md

464205e

dryajov reviewed May 23, 2024

View reviewed changes

Update marketplace.md

023b8f6

jimstir mentioned this pull request May 29, 2024

CODEX-NODE-SLOT-DISPERSAL: New Raw RFC #39

Closed

jimstir added 5 commits May 28, 2024 22:31

Update marketplace.md

b48e7c8

Create request-contract.md

88c2450

Add request diagram

41debdc

Update and rename codex/marketplace.md to codex/raw/marketplace.md

137ee43

Add diagram

822f1a6

markspanbroek reviewed May 30, 2024

View reviewed changes

jimstir and others added 3 commits May 30, 2024 10:30

Update marketplace.md

58e1317

Update marketplace.md

5acdaba

Merge branch 'main' into codex-marketplace

9f0020d

dryajov reviewed May 30, 2024

View reviewed changes

jimstir added 2 commits May 30, 2024 22:36

Update marketplace.md

6a82c7e

Update marketplace.md

0c2fb0c

	\| Validator Nodes \| A Codex node that collects, validates and submits missing storage proofs for a reward. \|
	\| Validator Nodes \| A Codex node that check for missing storage proofs and triggers for a reward. \|

		Using a erasure coding technique,
		the data chunks are encoded and placed into separate slots.

		The requester MUST provide a `duration` value for the storage request along with the appropriate `reward`,
		payment for request.

	client nodes MUST split the dataset into data chunks, $(c_1, c_2, c_3, \ldots, c_{20})$.
	client nodes MUST split the dataset into data chunks, $(c_1, c_2, c_3, \ldots, c_{x})$.

	fails to provide enough proofs of storage, the `collateral` MUST be forfeited.
	miss the critical amount of submitted proofs of the storage, the `collateral` will be slashed. If a critical amount of `collateral` is slashed, then the slot is vacated and the repair process kicks in.

	Before a storage provider can download the data, they MUST be selected to obtain a reseversation of a slot.
	Before a storage provider can download the data, they MUST obtain a reservation for a slot.

	The Codex network handles selection by using Kademlia distance function to select eligible storage providers per slot.
	The Codex network uses an expanding window based on the Kademlia distance function to select storage providers that are allowed to reserve a slot.


		hash(blockHash, requestId, slotIndex, reservationIndex);

		`blockHash`: unique identifier for a specific block


		$$ XOR(A,A_0) $$

		The allowed distance over time $t_1$, can be defined as $2^{256} * F(t_1)$.

	Support for light clients, like mobile devices should also be embraced.
	Support for resource restricted devices, like mobile devices should also be embraced.

CODEX-MARKETPLACE: New Raw RFC #36

Are you sure you want to change the base?

CODEX-MARKETPLACE: New Raw RFC #36

Conversation

jimstir commented Apr 30, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

emizzle left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

AuHau left a comment • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimstir May 23, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimstir May 23, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dryajov May 23, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dryajov commented May 23, 2024

markspanbroek left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dryajov May 30, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jimstir May 31, 2024 • edited

Choose a reason for hiding this comment

AuHau left a comment •

edited

jimstir May 23, 2024 •

edited

jimstir May 23, 2024 •

edited

dryajov May 23, 2024 •

edited

dryajov May 30, 2024 •

edited

jimstir May 31, 2024 •

edited