Skip to content

feat(antd, antd-go): external-signer single-chunk publish#61

Merged
Nic-dorman merged 3 commits into
mainfrom
feat/external-signer-chunks
May 13, 2026
Merged

feat(antd, antd-go): external-signer single-chunk publish#61
Nic-dorman merged 3 commits into
mainfrom
feat/external-signer-chunks

Conversation

@Nic-dorman
Copy link
Copy Markdown
Collaborator

@Nic-dorman Nic-dorman commented May 13, 2026

Summary

Adds external-signer single-chunk publish to both halves of the SDK:

  • antdPOST /v1/chunks/prepare + POST /v1/chunks/finalize. Unlike POST /v1/chunks (which requires AUTONOMI_WALLET_KEY on the daemon), neither new endpoint touches the daemon's wallet. Funds flow through the external signer's payForQuotes() exactly like the existing file/data prepare-and-finalize flow.
  • antd-goPrepareChunkUpload(ctx, content) + FinalizeChunkUpload(ctx, uploadID, txHashes), plus the PrepareChunkResult type.

Why

Today's POST /v1/chunks is daemon-wallet-only. Consumers that run antd wallet-less and pay externally (Indelible is the concrete one) have no way to publish single chunks through their own EVM signer. The existing file/data prepare/finalize endpoints don't help — those self-encrypt the input bytes into a multi-chunk DataMap-of-the-bytes, which is the wrong shape when you want to publish an already-serialized chunk verbatim at its content address.

A concrete trigger: a downstream republish flow (78 pre-existing private DataMaps that need to become public) wants to take serialized DataMap bytes already stored locally, publish them as one chunk each, and pay through its own signer rather than provisioning a hot wallet on antd. After this lands, that becomes a normal SDK call.

Surface

antd REST

POST /v1/chunks/prepare
{ "data": "<base64>" }

→ 200 (new chunk)
{
  "address": "<hex64>",
  "already_stored": false,
  "upload_id": "<hex32>",
  "payment_type": "wave_batch",
  "payments": [
    { "quote_hash": "0x…", "rewards_address": "0x…", "amount": "100" },
    
  ],
  "total_amount": "700",
  "payment_vault_address": "0x…",
  "payment_token_address": "0x…",
  "rpc_url": "https://…"
}

→ 200 (already on network — no payment needed)
{
  "address": "<hex64>",
  "already_stored": true
}
POST /v1/chunks/finalize
{
  "upload_id": "<hex32>",
  "tx_hashes": { "<quote_hash>": "<tx_hash>", }
}

→ 200
{ "address": "<hex64>" }

Wave-batch only. Single-chunk publishes don't need merkle batching — one chunk's worth of quotes is well below the wave-batch threshold.

antd-go

// happy path
res, _ := client.PrepareChunkUpload(ctx, content)
if !res.AlreadyStored {
    // external signer pays via res.Payments → tx_hashes
    addr, _ := client.FinalizeChunkUpload(ctx, res.UploadID, txHashes)
}

Implementation notes

  • state.rs: new pending_chunks: Mutex<HashMap<String, TimestampedChunk>> map alongside the existing pending_uploads, with matching cleanup_stale_chunks. The periodic cleanup task in main.rs now sweeps both.
  • chunk_prepare computes the BLAKE3 address up-front so the already-stored exit can return it without ant-core needing to (ant-core's prepare_chunk_payment short-circuits with Ok(None) in that case).
  • Zero-amount quotes are filtered out of the payments array — they still live in peer_quotes for ProofOfPayment, but the external signer shouldn't pay for them.
  • The chunks REST routes are method-distinct from the existing GET /v1/chunks/{addr} and POST /v1/chunks, so axum routes them correctly without precedence games.
  • New types in types.rs use skip_serializing_if heavily so the already-stored response carries only address — no upload_id, no payment fields. Test coverage locks the wire shape.

ant-core dep pin

antd/Cargo.toml currently pins ant-core to the merge commit of WithAutonomi/ant-client#89 (c0f6a81), which adds the Client::finalize_chunk primitive this PR consumes. The rev pin will be bumped to a tag once ant-core cuts a release containing that commit.

Test plan

  • cargo build clean
  • cargo test — 37 tests pass (6 new type/serialization tests for the chunk surface)
  • go build ./... clean (antd-go)
  • go test ./... — full suite passes (3 new httptest-backed tests for PrepareChunkUpload / FinalizeChunkUpload)
  • Reviewer: end-to-end smoke against a real daemon — quote one chunk via prepare, pay externally, finalize, fetch it back via GET /v1/chunks/{addr} and confirm byte-equality

🤖 Generated with Claude Code

Nic-dorman and others added 3 commits May 6, 2026 18:10
Mirrors PR #31's release.yml fix — without `repo-token`, the action
issues the GitHub release-list API call anonymously and shares the
~60/hr per-IP quota with every other runner on the same egress IP.
release.yml has had the auth since 2026-04-28; ci.yml didn't, and
PR #57's post-merge CI run flaked here on 2026-05-06 (run 25447450352
job 'API contract tests (antd-rust)' — exact `arduino/setup-protoc@v3
... API rate limit exceeded` message reproduced from PR #31's
investigation).

Adds `repo-token: \${{ secrets.GITHUB_TOKEN }}` to both occurrences
in ci.yml — the Check (antd) job at line 33 and the API contract tests
job at line 75. No semantic change to either workflow; just shifts
the action onto our 5,000/hr token quota.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds `POST /v1/chunks/prepare` + `POST /v1/chunks/finalize` to the antd REST
surface and `PrepareChunkUpload` + `FinalizeChunkUpload` to antd-go, giving
external signers a way to publish a single chunk paid through their own EVM
wallet rather than the daemon's.

## antd (Rust)

- `state.rs`: new `pending_chunks: Mutex<HashMap<String, TimestampedChunk>>`
  map alongside `pending_uploads`, with matching `cleanup_stale_chunks`.
  Single periodic cleanup task in `main.rs` now sweeps both.
- `types.rs`: `PrepareChunkRequest/Response` + `FinalizeChunkRequest/Response`.
  Response uses `skip_serializing_if` heavily so an already-on-network reply
  (`already_stored: true`) carries only `address` — no upload_id, no payments.
- `rest/chunks.rs::chunk_prepare`: calls `Client::prepare_chunk_payment`,
  computes the BLAKE3 address up-front (so the already-stored exit can still
  return it), filters zero-amount quotes from the external-signer payment
  list, and stashes the PreparedChunk under a fresh upload_id.
- `rest/chunks.rs::chunk_finalize`: looks up by upload_id, parses tx_hashes,
  calls the new `Client::finalize_chunk`, returns the chunk address.

Unlike `POST /v1/chunks` (which requires `AUTONOMI_WALLET_KEY` on the daemon),
neither new endpoint touches the daemon's wallet — funds flow through the
external signer.

## antd-go

- New `PrepareChunkResult` type mirroring the response shape.
- `PrepareChunkUpload(ctx, content)` + `FinalizeChunkUpload(ctx, uploadID, txHashes)`.
- `boolField` helper joins the existing `str`/`num64`/`unum64` helpers.
- Three new httptest-backed tests: happy-path prepare, already-stored prepare,
  finalize round-trip.

## Dep pin

`antd/Cargo.toml` pins ant-core to the merge commit of WithAutonomi/ant-client#89
(`c0f6a81`) which adds the `Client::finalize_chunk` primitive this PR consumes.
The pin should be bumped to a tag once ant-core cuts a release containing
that commit; the rev pin keeps reproducibility in the meantime.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Nic-dorman
Copy link
Copy Markdown
Collaborator Author

E2E smoke result — PASSED on Arbitrum One mainnet

Drove prepare → payForQuotes → finalize → ChunkGet against a local antd built from b643bc8, paying via an external wallet (the antd was started wallet-less). Random 256-byte payload, full round-trip.

payload     256 bytes, first8=4b970dc57e5ea6d1

--- prepare ---
address           e60b6f72b3451fc0d7a46eb92054547fcdc03ae4f28bdffd2efa4c9d7b727315
already_stored    false
upload_id         abc95292c2342722e180666bbd570295
payment_type      wave_batch
payments          1 entries, total=11718823242187500 atto
evm rpc           https://arb1.arbitrum.io/rpc
evm token         0xa78d8321B20c4Ef90eCd72f2588AA985A4BDb684
evm vault         0x9A3EcAc693b699Fc0B2B6A50B5549e50c2320A26
prepare took      20.999s

--- pay ---
tx_hashes         1 entries
  15d078..47e9 -> e1bab4..114b
pay took          2.945s

--- finalize ---
stored at         e60b6f72b3451fc0d7a46eb92054547fcdc03ae4f28bdffd2efa4c9d7b727315
finalize took     725ms

--- verify (ChunkGet round-trip) ---
fetched           256 bytes
get took          773ms

E2E PASSED — chunk published via external signer round-tripped intact.

Things this confirms:

  • prepare works against a wallet-less antd — the daemon collected one quote for ~0.0117 ANT (1.17e16 atto) without needing AUTONOMI_WALLET_KEY.
  • The address returned at prepare time matches the address returned at finalize time — content addressing holds.
  • The external wallet's payForQuotes(...) (the same shape Indelible already uses for file uploads) was sufficient for the single-chunk path — no special-casing in the signer.
  • ChunkGet after finalize returns the exact bytes that went into prepare — full byte equality on a 256-byte payload.

/v1/chunks/prepare initial quote collection was the slow part at ~21s; the rest of the round-trip (pay + finalize + get) totaled ~4.4s. That's consistent with the file/data prepare flow which also dominates wall-clock on quote collection.

@Nic-dorman Nic-dorman merged commit 428248a into main May 13, 2026
3 checks passed
@Nic-dorman Nic-dorman deleted the feat/external-signer-chunks branch May 13, 2026 12:51
Nic-dorman added a commit that referenced this pull request May 13, 2026
Cuts v0.7.0 atop v0.6.1. Adds external-signer single-chunk publish:

- feat(antd, antd-go): external-signer single-chunk publish (#61)
  - POST /v1/chunks/prepare — wave-batch payment shape for a single
    chunk, no daemon wallet required. Returns already_stored=true with
    just the address when the chunk is already on-network.
  - POST /v1/chunks/finalize — submits the chunk after the external
    signer has paid, returns the network address.
  - antd-go: PrepareChunkUpload(ctx, content) and
    FinalizeChunkUpload(ctx, uploadID, txHashes).
  - Bumps ant-core to a rev containing the new Client::finalize_chunk
    primitive (ant-client#89).

End-to-end smoke verified against Arbitrum One mainnet: random 256-byte
payload round-tripped via prepare → payForQuotes (external wallet) →
finalize → ChunkGet with full byte equality.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant