Skip to content

[DNM] feat(da): support fiber (not via c-node)#3244

Draft
julienrbrt wants to merge 35 commits intomainfrom
julien/fiber
Draft

[DNM] feat(da): support fiber (not via c-node)#3244
julienrbrt wants to merge 35 commits intomainfrom
julien/fiber

Conversation

@julienrbrt
Copy link
Copy Markdown
Member

@julienrbrt julienrbrt commented Apr 13, 2026

Overview

Support Fiber client (based on https://github.com/celestiaorg/celestia-app/blob/63fbf31cca216fc4e067a9e1b3a3431115c7009b/fibre), but not via celestia node or apex for this PoC
celestiaorg/celestia-node#4892

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Apr 13, 2026

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 1867e9d5-178a-4289-a047-e4e3586073ff

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch julien/fiber

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 13, 2026

The latest Buf updates on your PR. Results from workflow CI / buf-check (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed⏩ skipped✅ passed✅ passedApr 24, 2026, 9:38 PM

@claude
Copy link
Copy Markdown
Contributor

claude Bot commented Apr 13, 2026

Claude finished @julienrbrt's task in 3m 12s —— View job


Code Review: [DNM] feat(da): support fiber (not via c-node)

  • Read AGENTS.md for repo-specific guidelines
  • Read all changed files
  • Review core Fiber DA client implementation
  • Review configuration changes
  • Review sequencer integration
  • Review testing infrastructure
  • Review tools/celestia-node-fiber package
  • Post comprehensive review feedback

This is a well-structured PoC for integrating the Fiber DA protocol. The layered design (interface → mock → adapter → integration tests) is solid and the flattenBlobs/splitBlobs encoding is correct and well-tested. Below are the issues found, ordered by severity.


Critical

1. context.Background() in Submit ignores caller cancellation
block/internal/da/fiber_client.go:92

result, err := c.fiber.Upload(context.Background(), namespace[len(namespace)-10:], flat)

The passed ctx is completely discarded — uploads will not respect caller cancellation or deadlines. Change to ctx. Fix this →

2. panic in GetLatestDAHeight will crash the process
block/internal/da/fiber_client.go:303-305

func (c *fiberDAClient) GetLatestDAHeight(context.Context) (uint64, error) {
    panic(fmt.Errorf("p2p should not be enabled"))
}

A panic in an interface method that could be called by any part of the node is not an acceptable sentinel. Return (0, errors.ErrUnsupported) or a domain-specific error instead. Fix this →


High

3. Subscriber index corruption in MockDA.Listen under concurrent close
block/internal/da/fibremock/mock.go:218-228

The cleanup goroutine captures idx at subscription time, but the swap-with-last removal strategy invalidates idx when a subscriber with a lower index is removed first. For example:

  • Start with subscribers [A(idx=0), B(idx=1), C(idx=2)]
  • A's context is cancelled: swap [0] with [2][C, B]. C is now at position 0 but its closure still holds idx=2.
  • C's context is cancelled: last = len-1 = 1, idx (2) <= last (1) → false → no swap, subscribers = subscribers[:1][C]. C is never removed and its channel leaks.

A simple fix is to use a map[int]subscriber with an auto-incrementing key, or accept the O(n) linear scan on removal since this is a mock.

4. Subscribe swallows Listen errors from the goroutine
block/internal/da/fiber_client.go:253-265

blobCh, err := c.fiber.Listen(...)
if err != nil {
    c.logger.Error().Err(err).Msg("fiber listen failed")
    return  // closes out — caller receives closed channel with no error
}

The caller receives (out, nil) and only discovers the failure when out immediately closes. Consider returning the error synchronously (move Listen before launching the goroutine) or signalling via a side-channel. Fix this →

5. Namespace slice panics if namespace is shorter than 10 bytes
fiber_client.go:92, 141, 260

namespace[len(namespace)-10:]  // panics if len(namespace) < 10

There is no guard before this slicing in Submit, retrieve, or Subscribe. Add a length check and return a clear error. Fix this →

6. NewFiberDAClient panics on error in block/public.go
block/public.go:86-93

base, err := da.NewFiberClient(...)
if err != nil {
    panic(err)
}

NewFiberClient can fail (e.g., nil client). Callers like StartNode construct this in the request path — a panic here brings down the whole process. The function should return (FullDAClient, error). Fix this →


Medium

7. KeyringPath has no CLI flag
pkg/config/config.go:303 defines FiberDAConfig.KeyringPath but AddFlags in the same file registers no FlagDAFiberKeyringPath. Users cannot set the keyring directory from the CLI or environment. Either add the flag or remove the field.

8. Validate silently returns true for everything
fiber_client.go:318-326

for i := range results {
    results[i] = true  // not implemented
}
return results, nil

This bypasses blob integrity checks entirely without signalling to callers. Return errors.ErrUnsupported (like GetProofs does) so callers can decide how to handle unimplemented verification.

9. retrieve / RetrieveBlobs are identical
fiber_client.go:129-135

func (c *fiberDAClient) Retrieve(ctx context.Context, height uint64, namespace []byte) datypes.ResultRetrieve {
    return c.retrieve(ctx, height, namespace, true)
}
func (c *fiberDAClient) RetrieveBlobs(ctx context.Context, height uint64, namespace []byte) datypes.ResultRetrieve {
    return c.retrieve(ctx, height, namespace, false)
}

The bool parameter is named _ in retrieve — it has no effect. The two methods produce identical results. Either give the flag meaning or collapse them.

10. P2P broadcast suppressed for Fiber — undocumented
block/internal/executing/executor.go:631

if !e.config.DA.IsFiberEnabled() {
    // Broadcast header and data to P2P network

When Fiber is enabled, full nodes receive no P2P gossip. This is a significant protocol change that affects sync for non-sequencer nodes and deserves a doc comment explaining the rationale and expected sync path.

11. Listen performs a Download per event for DataSize
tools/celestia-node-fiber/listen.go:30-33
The comment acknowledges one FSP round-trip per blob. At high throughput this could saturate the connection. Consider making DataSize optional (0 = unknown) and letting consumers decide whether to resolve it.


Minor

12. Magic number fiberSubscribeChanSize = 42
fiber_client.go:248 — the value 42 carries no semantic meaning. A constant named something like defaultSubscriptionBuffer with a comment explaining the rationale (e.g., matching upstream buffer) would be clearer.

13. TODO blocks in returned ResultSubmit
fiber_client.go:115, 125

Height: 0, /* TODO */

The zero height is returned to callers including the coordinator. Several tests are skipped because of this. Track the root cause (Fiber protocol doesn't confirm celestia block height synchronously) as a GitHub issue rather than leaving silent /* TODO */ in results.

14. Hardcoded blobExpiration = 24 * time.Hour in adapter
tools/celestia-node-fiber/adapter.go:28
The placeholder expiry could mislead callers into assuming blobs are retained longer (or shorter) than the actual protocol. The TODO is noted, but consider returning a zero ExpiresAt (meaning "unknown") until the real value is available.

15. Comment typo in config constants
pkg/config/config.go:102

// FlagConsensusChainID is the Chain ID of the celestia app node
FlagDAFiberConsensusChainID = FlagPrefixEvnode + "da.fiber.consensus_chain_id"

The comment says FlagConsensusChainID but the constant is FlagDAFiberConsensusChainID.

16. UploadConcurrency mentioned but not defined
pkg/config/config.go:306

// UploadConcurrency limits the number of concurrent upload connections

The comment ends the struct body — there is no actual field. Either add the field or remove the comment.

17. Missing error log in forwardFibreBlobs on skip
tools/celestia-node-fiber/listen.go:71-76
When fibreBlobToEvent fails, the event is silently dropped. A logger.Warn() or logger.Debug() would significantly aid debugging in production.


Testing

The unit and mock test coverage is good. A few notes:

  • Multiple Retrieve/Subscribe/Validate tests are skipped pending height tracking (t.Skip("pending Height tracking from fiber DA")). These should be tracked in issues so they don't remain skipped indefinitely.
  • The //go:build fibre integration tests are a clean opt-in pattern.
  • TestMockDA_DeterministicBlobID uploads the same payload twice, verifying the blob ID is content-addressed — this would silently coalesce two different uploads at the same height if they share content. Consider whether this is intentional.

@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 13, 2026

Codecov Report

❌ Patch coverage is 90.65657% with 37 lines in your changes missing coverage. Please review.
✅ Project coverage is 63.16%. Comparing base (2865d6d) to head (4485d91).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
block/public.go 0.00% 12 Missing ⚠️
block/internal/da/fibremock/mock.go 90.90% 5 Missing and 5 partials ⚠️
block/internal/da/fiber_client.go 96.74% 5 Missing and 3 partials ⚠️
pkg/sequencers/solo/sequencer.go 61.53% 5 Missing ⚠️
pkg/config/config.go 75.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3244      +/-   ##
==========================================
+ Coverage   62.33%   63.16%   +0.82%     
==========================================
  Files         122      124       +2     
  Lines       12873    13258     +385     
==========================================
+ Hits         8024     8374     +350     
- Misses       3968     3995      +27     
- Partials      881      889       +8     
Flag Coverage Δ
combined 63.16% <90.65%> (+0.82%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

julienrbrt and others added 7 commits April 14, 2026 15:12
Adds a fibremock package with:
- DA interface (Upload/Download/Listen) matching the fibre gRPC service
- In-memory MockDA implementation with LRU eviction and configurable retention
- Tests covering all paths

Migrated from celestiaorg/x402-risotto#16 as-is for integration.
@julienrbrt julienrbrt changed the title feat(da): support fiber (not via c-node) [DNM] feat(da): support fiber (not via c-node) Apr 20, 2026
julienrbrt and others added 15 commits April 20, 2026 14:46
Adds tools/celestia-node-fiber, a new Go sub-module that implements the
ev-node fiber.DA interface by delegating Upload, Download and Listen to a
celestia-node api/client.Client.

Upload and Download run locally against a Celestia consensus node (gRPC)
and Fibre Storage Providers (Fibre gRPC) — no bridge-node hop — using
celestia-node's self-sufficient client (celestiaorg/celestia-node#4961).
Listen subscribes to blob.Subscribe on a bridge node and forwards only
share-version-2 blobs, which is how Fibre blobs settle on-chain via
MsgPayForFibre.

The package lives in its own go.mod, parallel to tools/local-fiber, so
ev-node core does not inherit celestia-app / cosmos-sdk replace-directive
soup. A FromModules constructor accepts the Fibre and Blob Module
interfaces directly so callers can inject mocks or share an existing
*api/client.Client.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…#3280)

* test(celestia-node-fiber): showcase end-to-end Upload/Listen/Download

Adds tools/celestia-node-fiber/testing/, a single-validator in-process
showcase that boots a fibre-tagged Celestia chain + in-process Fibre
server + celestia-node bridge, registers the validator's FSP via
valaddr (with the dns:/// URI scheme the client's gRPC resolver
expects), funds an escrow account, and drives the full adapter
surface.

TestShowcase proves the round-trip: subscribe via Listen, Upload a
blob, wait for the share-version-2 BlobEvent that lands after the
async MsgPayForFibre commits, assert the BlobID from Listen matches
Upload's return, Download and diff the payload bytes.

The harness is intentionally single-validator — a 2-validator
Docker Compose showcase is planned as a follow-up for exercising real
quorum collection.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(celestia-node-fiber): scale showcase to 10 blobs, document DataSize gap

Upload 10 distinct-payload blobs through adapter.Upload, collect
BlobEvents via adapter.Listen until every BlobID is accounted for
(order-insensitive, rejects duplicates), then round-trip each blob
through adapter.Download to diff bytes. Catches routing bugs (wrong
blob returned for a BlobID) and duplicate-event bugs that a
single-blob test can't see.

Scaling the test also exposed a semantic issue: the v2 share carries
only (fibre_blob_version + commitment), so b.DataLen() — what
listen.go's fibreBlobToEvent reports today — is always 36, not the
original payload length ev-node's fibermock conveys. The adapter
can't derive the payload size from the subscription stream alone;
surfacing it correctly needs an x/fibre PaymentPromise lookup
(tracked as a TODO on fibreBlobToEvent). The test therefore asserts
DataSize is non-zero rather than matching len(payload).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…3281)

listen.go previously set BlobEvent.DataSize to b.DataLen(), which for
a share-version-2 Fibre blob is always the fixed share-data layout
(fibre_blob_version + commitment = 36 bytes) — not the original
payload length. That diverges from ev-node's fibermock contract and
misleads any consumer that uses DataSize to allocate buffers or
report progress.

The v2 share genuinely doesn't carry the original size, and x/fibre
v8 has no chain query to derive it from the commitment. The only
accurate path is to Download the blob and measure. Listen now does
exactly that before forwarding each event. The cost is one FSP
round-trip per v2 blob; can be made opt-out later if it hurts
throughput-sensitive use cases.

Tests:
- Showcase restores the strict DataSize == len(payload) assertion
  across all 10 blobs.
- Unit test TestListen_FiltersFibreOnlyAndEmitsEvent now stubs
  fakeFibre.Download to return a deterministic payload and asserts
  DataSize matches its length.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ight subscriptions (#3283)

feat(celestia-node-fiber): Listen takes fromHeight for resume subscriptions

Threads a fromHeight parameter through the Fibre DA Listen path so a
subscriber can rejoin the stream from a past block height without
missing blobs. Consumes the matching celestia-node API change landed
in celestiaorg/celestia-node#4962, which gave Blob.Subscribe a
fromHeight argument backed by a WaitForHeight loop.

Changes:

- block/internal/da/fiber/types.go: DA.Listen signature now takes
  fromHeight uint64. fromHeight == 0 preserves "follow from tip"
  semantics, >0 replays from that block forward.
- block/internal/da/fibremock/mock.go: replay matching blobs with
  height >= fromHeight before attaching the live subscriber.
- block/internal/da/fiber_client.go: outer fiberDAClient.Subscribe
  does not yet expose a starting height (datypes.DA doesn't plumb
  one), so pass 0 and defer resume-from-height wiring to a future
  datypes.DA change.
- tools/celestia-node-fiber/listen.go: propagate fromHeight to
  client.Blob.Subscribe on the celestia-node API.
- tools/celestia-node-fiber/go.mod: bump celestia-node to the merged
  pseudo-version (v0.0.0-20260423143400-194cc74ce99c) carrying #4962.
- tools/celestia-node-fiber/adapter_test.go: fakeBlob.subscribeFn
  gets the new fromHeight arg; TestListen_FiltersFibreOnlyAndEmitsEvent
  asserts that fromHeight=0 is forwarded.
- tools/celestia-node-fiber/testing/showcase_test.go: existing
  TestShowcase passes fromHeight=0. New TestShowcaseResume uploads 3
  blobs, discovers their settlement heights via a live Listen, then
  opens a fresh Listen with fromHeight at the first blob's height and
  verifies every historical blob is replayed with correct Height and
  DataSize.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants