[DNM] feat(da): support fiber (not via c-node)#3244
[DNM] feat(da): support fiber (not via c-node)#3244julienrbrt wants to merge 35 commits intomainfrom
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
The latest Buf updates on your PR. Results from workflow CI / buf-check (pull_request).
|
|
Claude finished @julienrbrt's task in 3m 12s —— View job Code Review: [DNM] feat(da): support fiber (not via c-node)
This is a well-structured PoC for integrating the Fiber DA protocol. The layered design (interface → mock → adapter → integration tests) is solid and the Critical1. result, err := c.fiber.Upload(context.Background(), namespace[len(namespace)-10:], flat)The passed 2. func (c *fiberDAClient) GetLatestDAHeight(context.Context) (uint64, error) {
panic(fmt.Errorf("p2p should not be enabled"))
}A High3. Subscriber index corruption in The cleanup goroutine captures
A simple fix is to use a 4. blobCh, err := c.fiber.Listen(...)
if err != nil {
c.logger.Error().Err(err).Msg("fiber listen failed")
return // closes out — caller receives closed channel with no error
}The caller receives 5. Namespace slice panics if namespace[len(namespace)-10:] // panics if len(namespace) < 10There is no guard before this slicing in 6. base, err := da.NewFiberClient(...)
if err != nil {
panic(err)
}
Medium7. 8. for i := range results {
results[i] = true // not implemented
}
return results, nilThis bypasses blob integrity checks entirely without signalling to callers. Return 9. func (c *fiberDAClient) Retrieve(ctx context.Context, height uint64, namespace []byte) datypes.ResultRetrieve {
return c.retrieve(ctx, height, namespace, true)
}
func (c *fiberDAClient) RetrieveBlobs(ctx context.Context, height uint64, namespace []byte) datypes.ResultRetrieve {
return c.retrieve(ctx, height, namespace, false)
}The 10. P2P broadcast suppressed for Fiber — undocumented if !e.config.DA.IsFiberEnabled() {
// Broadcast header and data to P2P networkWhen Fiber is enabled, full nodes receive no P2P gossip. This is a significant protocol change that affects sync for non-sequencer nodes and deserves a doc comment explaining the rationale and expected sync path. 11. Listen performs a Download per event for Minor12. Magic number 13. Height: 0, /* TODO */The zero height is returned to callers including the coordinator. Several tests are skipped because of this. Track the root cause (Fiber protocol doesn't confirm celestia block height synchronously) as a GitHub issue rather than leaving silent 14. Hardcoded 15. Comment typo in config constants // FlagConsensusChainID is the Chain ID of the celestia app node
FlagDAFiberConsensusChainID = FlagPrefixEvnode + "da.fiber.consensus_chain_id"The comment says 16. // UploadConcurrency limits the number of concurrent upload connectionsThe comment ends the struct body — there is no actual field. Either add the field or remove the comment. 17. Missing error log in TestingThe unit and mock test coverage is good. A few notes:
|
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #3244 +/- ##
==========================================
+ Coverage 62.33% 63.16% +0.82%
==========================================
Files 122 124 +2
Lines 12873 13258 +385
==========================================
+ Hits 8024 8374 +350
- Misses 3968 3995 +27
- Partials 881 889 +8
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Adds a fibremock package with: - DA interface (Upload/Download/Listen) matching the fibre gRPC service - In-memory MockDA implementation with LRU eviction and configurable retention - Tests covering all paths Migrated from celestiaorg/x402-risotto#16 as-is for integration.
Adds tools/celestia-node-fiber, a new Go sub-module that implements the ev-node fiber.DA interface by delegating Upload, Download and Listen to a celestia-node api/client.Client. Upload and Download run locally against a Celestia consensus node (gRPC) and Fibre Storage Providers (Fibre gRPC) — no bridge-node hop — using celestia-node's self-sufficient client (celestiaorg/celestia-node#4961). Listen subscribes to blob.Subscribe on a bridge node and forwards only share-version-2 blobs, which is how Fibre blobs settle on-chain via MsgPayForFibre. The package lives in its own go.mod, parallel to tools/local-fiber, so ev-node core does not inherit celestia-app / cosmos-sdk replace-directive soup. A FromModules constructor accepts the Fibre and Blob Module interfaces directly so callers can inject mocks or share an existing *api/client.Client. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…#3280) * test(celestia-node-fiber): showcase end-to-end Upload/Listen/Download Adds tools/celestia-node-fiber/testing/, a single-validator in-process showcase that boots a fibre-tagged Celestia chain + in-process Fibre server + celestia-node bridge, registers the validator's FSP via valaddr (with the dns:/// URI scheme the client's gRPC resolver expects), funds an escrow account, and drives the full adapter surface. TestShowcase proves the round-trip: subscribe via Listen, Upload a blob, wait for the share-version-2 BlobEvent that lands after the async MsgPayForFibre commits, assert the BlobID from Listen matches Upload's return, Download and diff the payload bytes. The harness is intentionally single-validator — a 2-validator Docker Compose showcase is planned as a follow-up for exercising real quorum collection. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * test(celestia-node-fiber): scale showcase to 10 blobs, document DataSize gap Upload 10 distinct-payload blobs through adapter.Upload, collect BlobEvents via adapter.Listen until every BlobID is accounted for (order-insensitive, rejects duplicates), then round-trip each blob through adapter.Download to diff bytes. Catches routing bugs (wrong blob returned for a BlobID) and duplicate-event bugs that a single-blob test can't see. Scaling the test also exposed a semantic issue: the v2 share carries only (fibre_blob_version + commitment), so b.DataLen() — what listen.go's fibreBlobToEvent reports today — is always 36, not the original payload length ev-node's fibermock conveys. The adapter can't derive the payload size from the subscription stream alone; surfacing it correctly needs an x/fibre PaymentPromise lookup (tracked as a TODO on fibreBlobToEvent). The test therefore asserts DataSize is non-zero rather than matching len(payload). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…3281) listen.go previously set BlobEvent.DataSize to b.DataLen(), which for a share-version-2 Fibre blob is always the fixed share-data layout (fibre_blob_version + commitment = 36 bytes) — not the original payload length. That diverges from ev-node's fibermock contract and misleads any consumer that uses DataSize to allocate buffers or report progress. The v2 share genuinely doesn't carry the original size, and x/fibre v8 has no chain query to derive it from the commitment. The only accurate path is to Download the blob and measure. Listen now does exactly that before forwarding each event. The cost is one FSP round-trip per v2 blob; can be made opt-out later if it hurts throughput-sensitive use cases. Tests: - Showcase restores the strict DataSize == len(payload) assertion across all 10 blobs. - Unit test TestListen_FiltersFibreOnlyAndEmitsEvent now stubs fakeFibre.Download to return a deterministic payload and asserts DataSize matches its length. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ight subscriptions (#3283) feat(celestia-node-fiber): Listen takes fromHeight for resume subscriptions Threads a fromHeight parameter through the Fibre DA Listen path so a subscriber can rejoin the stream from a past block height without missing blobs. Consumes the matching celestia-node API change landed in celestiaorg/celestia-node#4962, which gave Blob.Subscribe a fromHeight argument backed by a WaitForHeight loop. Changes: - block/internal/da/fiber/types.go: DA.Listen signature now takes fromHeight uint64. fromHeight == 0 preserves "follow from tip" semantics, >0 replays from that block forward. - block/internal/da/fibremock/mock.go: replay matching blobs with height >= fromHeight before attaching the live subscriber. - block/internal/da/fiber_client.go: outer fiberDAClient.Subscribe does not yet expose a starting height (datypes.DA doesn't plumb one), so pass 0 and defer resume-from-height wiring to a future datypes.DA change. - tools/celestia-node-fiber/listen.go: propagate fromHeight to client.Blob.Subscribe on the celestia-node API. - tools/celestia-node-fiber/go.mod: bump celestia-node to the merged pseudo-version (v0.0.0-20260423143400-194cc74ce99c) carrying #4962. - tools/celestia-node-fiber/adapter_test.go: fakeBlob.subscribeFn gets the new fromHeight arg; TestListen_FiltersFibreOnlyAndEmitsEvent asserts that fromHeight=0 is forwarded. - tools/celestia-node-fiber/testing/showcase_test.go: existing TestShowcase passes fromHeight=0. New TestShowcaseResume uploads 3 blobs, discovers their settlement heights via a live Listen, then opens a fresh Listen with fromHeight at the first blob's height and verifies every historical blob is replayed with correct Height and DataSize. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Overview
Support Fiber client (based on https://github.com/celestiaorg/celestia-app/blob/63fbf31cca216fc4e067a9e1b3a3431115c7009b/fibre), but not via celestia node or apex for this PoC
celestiaorg/celestia-node#4892