feat: w3filecoin new client and api #848

vasco-santos · 2023-08-02T13:32:14Z

Implements new client and API based on storacha-network/specs#71

Note that new client is filecoin spec based rather than aggregation only. There are other roles that are not aggregator:

storefront will be implemented by web3.storage/nft.storage via w3up
aggregator will be implemented by w3filecoin
broker will be implemented by spade-proxy
chain will (likely) be implemented by w3filecoin (reading from spade oracle, while we might have an Oracle ourselves later on)

Gozala · 2023-08-07T23:16:32Z

packages/capabilities/src/filecoin.js

+    /**
+     * Storefront requestin piece to be aggregated
+     */
+    storefront: Schema.text(),


I think we should drop this field per spec feedback, but if we don't we should:

Ensure that it does not change in delegations, meaning verified in derives.

Ensure that it equals to with field so that nft.storage can't specify web3.storage.

with might change in follow up invocation, so changing point 1

packages/capabilities/src/filecoin.js

Gozala · 2023-08-07T23:27:09Z

packages/filecoin-api/src/aggregator.js

+ * @param {API.AggregatorServiceContext} context
+ * @returns {Promise<API.UcantoInterface.Result<API.PieceAddSuccess, API.PieceAddFailure> | API.UcantoInterface.JoinBuilder<API.PieceAddSuccess>>}
+ */
+async function queueHandler(piece, storefront, group, context) {


Suggested change

async function queueHandler(piece, storefront, group, context) {

async function accept(piece, storefront, group, context) {

Why not call it an accept since basically that is what we do here ? Or alternatively call it a dequeue ?

packages/filecoin-api/src/aggregator.js

Gozala · 2023-08-07T23:35:04Z

packages/filecoin-api/src/broker.js

+  // Check if self signed to call queue handler
+  if (context.id.did() === capability.with) {
+    return queueHandler(piece, offer, deal, context)
+  }
+
+  return queueAdd(piece, offerCid, deal, offer, context)


I think following would follow the logic more clearly

Suggested change

// Check if self signed to call queue handler

if (context.id.did() === capability.with) {

return queueHandler(piece, offer, deal, context)

}

return queueAdd(piece, offerCid, deal, offer, context)

// Check if self signed to call queue handler

if (context.id.did() === capability.with) {

return queueHandler(piece, offer, deal, context)

}

// If self issued we accept without verification

return context.id.did() === capability.with

? accept(piece, offer, deal, context)

: enqueue(piece, offer, deal, context)

packages/filecoin-api/src/broker.js

Gozala · 2023-08-07T23:41:47Z

packages/filecoin-api/src/storefront.js

+ */
+async function queueHandler(piece, content, context) {
+  // store piece
+  const put = await context.pieceStore.put({


Not sure howe piece propagates to the aggregator here, if there was no user call with filecoin/add first

Yeah, I decoupled things to happen outside of the scope so that each client/api can be used independently. Otherwise, we would get everything requiring everything else. Either they can rely on store events, or insertion with the client right after

🤖 I have created a release *beep* *boop* --- ## [8.0.0](capabilities-v7.0.0...capabilities-v8.0.0) (2023-08-09) ### ⚠ BREAKING CHANGES * update aggregation capabilitites to use height instead of size together with client and api ([#831](#831)) ### Features * w3filecoin new client and api ([#848](#848)) ([7a58fbe](7a58fbe)) ### Bug Fixes * update aggregation capabilitites to use height instead of size together with client and api ([#831](#831)) ([31730f0](31730f0)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

🤖 I have created a release *beep* *boop* --- ## [1.1.0](filecoin-client-v1.0.0...filecoin-client-v1.1.0) (2023-08-09) ### Features * w3filecoin new client and api ([#848](#848)) ([7a58fbe](7a58fbe)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: Vasco Santos <santos.vasco10@gmail.com>

🤖 I have created a release *beep* *boop* --- ## [1.1.0](filecoin-api-v1.0.0...filecoin-api-v1.1.0) (2023-08-09) ### Features * w3filecoin new client and api ([#848](#848)) ([7a58fbe](7a58fbe)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: Vasco Santos <santos.vasco10@gmail.com>

🤖 I have created a release *beep* *boop* --- ## 1.0.0 (2023-08-10) ### Features * w3filecoin new client and api ([#848](#848)) ([7a58fbe](7a58fbe)) ### Bug Fixes * upgrade data segment ([#850](#850)) ([fba281f](fba281f)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

🤖 I have created a release *beep* *boop* --- ## 1.0.0 (2023-08-10) ### Features * w3filecoin new client and api ([#848](#848)) ([7a58fbe](7a58fbe)) ### Bug Fixes * upgrade data segment ([#850](#850)) ([fba281f](fba281f)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: Vasco Santos <santos.vasco10@gmail.com>

🤖 I have created a release *beep* *boop* --- ## [1.2.0](filecoin-api-v1.1.0...filecoin-api-v1.2.0) (2023-08-10) ### Features * w3filecoin new client and api ([#848](#848)) ([7a58fbe](7a58fbe)) ### Bug Fixes * upgrade data segment ([#850](#850)) ([fba281f](fba281f)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

🤖 I have created a release *beep* *boop* --- ## [1.2.0](filecoin-client-v1.1.0...filecoin-client-v1.2.0) (2023-08-10) ### Features * w3filecoin new client and api ([#848](#848)) ([7a58fbe](7a58fbe)) ### Bug Fixes * upgrade data segment ([#850](#850)) ([fba281f](fba281f)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: Vasco Santos <santos.vasco10@gmail.com>

🤖 I have created a release *beep* *boop* --- ## [1.3.0](filecoin-client-v1.2.2...filecoin-client-v1.3.0) (2023-08-30) ### Features * w3filecoin new client and api ([#848](#848)) ([7a58fbe](7a58fbe)) ### Bug Fixes * upgrade data segment ([#850](#850)) ([fba281f](fba281f)) * w3filecoin spec separate capabilities to queue and enqueue ([#856](#856)) ([6bf9142](6bf9142)), closes [#855](#855) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

🤖 I have created a release *beep* *boop* --- ## [1.3.0](filecoin-api-v1.2.2...filecoin-api-v1.3.0) (2023-08-30) ### Features * w3filecoin new client and api ([#848](#848)) ([7a58fbe](7a58fbe)) ### Bug Fixes * upgrade data segment ([#850](#850)) ([fba281f](fba281f)) * w3filecoin spec separate capabilities to queue and enqueue ([#856](#856)) ([6bf9142](6bf9142)), closes [#855](#855) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: Vasco Santos <santos.vasco10@gmail.com>

🤖 I have created a release *beep* *boop* --- ## [1.4.0](filecoin-api-v1.3.1...filecoin-api-v1.4.0) (2023-08-30) ### Features * w3filecoin new client and api ([#848](#848)) ([7a58fbe](7a58fbe)) ### Bug Fixes * types when storefront is not in capability nb ([#886](#886)) ([448a7d1](448a7d1)) * upgrade data segment ([#850](#850)) ([fba281f](fba281f)) * w3filecoin spec separate capabilities to queue and enqueue ([#856](#856)) ([6bf9142](6bf9142)), closes [#855](#855) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

vasco-santos force-pushed the feat/w3filecoin-new-client-and-protocol branch 5 times, most recently from 0b757f9 to 1cdc74e Compare August 2, 2023 20:37

feat: w3filecoin new client and api

d2bc770

vasco-santos force-pushed the feat/w3filecoin-new-client-and-protocol branch 2 times, most recently from b541ff6 to 568618f Compare August 3, 2023 07:59

fix: add space to piece record

5b15429

vasco-santos force-pushed the feat/w3filecoin-new-client-and-protocol branch from 568618f to 5b15429 Compare August 3, 2023 08:25

chore: iterate on interfaces

0e71e7e

vasco-santos force-pushed the feat/w3filecoin-new-client-and-protocol branch from 5dbb012 to 0e71e7e Compare August 3, 2023 20:38

chore: iterate based on feedback in architecture PR

8e34375

vasco-santos marked this pull request as ready for review August 4, 2023 09:36

vasco-santos added 2 commits August 4, 2023 14:08

chore: simplify errors

c833635

chore: simplify errors

9f365aa

vasco-santos force-pushed the feat/w3filecoin-new-client-and-protocol branch from f7ffeab to 9f365aa Compare August 4, 2023 14:41

vasco-santos requested review from Gozala and alanshaw August 4, 2023 14:43

chore: add inserted at to store

8dca776

This was referenced Aug 7, 2023

Implement filecoin storefront protocol in w3up storacha-network/w3infra#214

Closed

Implement filecoin storefront protocol to integrate with w3filecoin web3-storage/web3.storage#2290

Open

Gozala approved these changes Aug 8, 2023

View reviewed changes

vasco-santos added 4 commits August 8, 2023 11:57

chore: updates based on specs pr review

aa260f3

chore: format and more comments addressed

193b4bc

chore: export correct name

8a61fab

fix: types for client receipt

4db685b

vasco-santos merged commit 7a58fbe into main Aug 9, 2023
14 checks passed

vasco-santos deleted the feat/w3filecoin-new-client-and-protocol branch August 9, 2023 08:48

it-dag-house mentioned this pull request Aug 9, 2023

chore(main): release capabilities 8.0.0 #844

Merged

This was referenced Aug 9, 2023

chore(main): release filecoin-api 1.1.0 #852

Merged

chore(main): release filecoin-client 1.1.0 #851

Merged

This was referenced Aug 9, 2023

chore(main): release filecoin-client 1.0.0 #853

Merged

chore(main): release filecoin-api 1.0.0 #854

Merged

This was referenced Aug 10, 2023

chore(main): release filecoin-client 1.2.0 #859

Merged

chore(main): release filecoin-api 1.2.0 #860

Merged

This was referenced Aug 30, 2023

chore(main): release filecoin-client 1.3.0 #884

Merged

chore(main): release filecoin-api 1.3.0 #885

Merged

it-dag-house mentioned this pull request Aug 30, 2023

chore(main): release filecoin-api 1.4.0 #887

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: w3filecoin new client and api #848

feat: w3filecoin new client and api #848

vasco-santos commented Aug 2, 2023 •

edited

Gozala Aug 7, 2023

vasco-santos Aug 8, 2023

Gozala Aug 7, 2023

Gozala Aug 7, 2023

Gozala Aug 7, 2023

vasco-santos Aug 8, 2023

	async function queueHandler(piece, storefront, group, context) {
	async function accept(piece, storefront, group, context) {

feat: w3filecoin new client and api #848

feat: w3filecoin new client and api #848

Conversation

vasco-santos commented Aug 2, 2023 • edited

Gozala Aug 7, 2023

Choose a reason for hiding this comment

vasco-santos Aug 8, 2023

Choose a reason for hiding this comment

Gozala Aug 7, 2023

Choose a reason for hiding this comment

Gozala Aug 7, 2023

Choose a reason for hiding this comment

Gozala Aug 7, 2023

Choose a reason for hiding this comment

vasco-santos Aug 8, 2023

Choose a reason for hiding this comment

vasco-santos commented Aug 2, 2023 •

edited