Skip to content

fix(grpc): convert ChainPoint/BlockRef timestamp from seconds to ms#1003

Merged
scarmuega merged 1 commit into
txpipe:mainfrom
stnly-btcdefi:fix/u5c-timestamp-ms-spec-compliance
May 28, 2026
Merged

fix(grpc): convert ChainPoint/BlockRef timestamp from seconds to ms#1003
scarmuega merged 1 commit into
txpipe:mainfrom
stnly-btcdefi:fix/u5c-timestamp-ms-spec-compliance

Conversation

@stnly-btcdefi
Copy link
Copy Markdown
Contributor

@stnly-btcdefi stnly-btcdefi commented May 28, 2026

While integrating PR #1000 downstream into a Cardano application that consumes utxo-rpc through pogun-chainfollower, we noticed the new block_ref field on SearchUtxos/ReadUtxos came back populated but with a 10-digit timestamp — a UNIX seconds value, not the milliseconds the utxorpc proto declares. Tracing it further, the same unit mismatch exists in four pre-existing sites in Dolos's gRPC layer (ReadTip, ReadTx, both on v1alpha and v1beta), all the way through EraSummary::slot_time and the pallas LedgerContext adapter.

What the proto says

utxorpc/spec declares the timestamp field as milliseconds in four places, consistently:

  • proto/utxorpc/v1alpha/query/query.protoChainPoint.timestamp (// Block ms timestamp)
  • proto/utxorpc/v1beta/query/query.protoChainPoint.timestamp (same)
  • proto/utxorpc/v1alpha/sync/sync.protoBlockRef.timestamp (same)
  • proto/utxorpc/v1beta/sync/sync.protoBlockRef.timestamp (same)

What Dolos actually writes

The root is EraSummary::slot_time (crates/cardano/src/eras.rs:23) which returns seconds. Three confirming signals in-tree:

  1. crates/cardano/src/eras.rs:38 — the sibling helper names its arithmetic let second_delta = slot_delta * self.slot_length;.
  2. crates/cardano/src/forks.rs:81 — Byron's ms-typed slot_duration is divided by 1000 before being stored as slot_length.
  3. crates/cardano/src/forks.rs:106 — Shelley system_start is stored as chrono::DateTime::timestamp() as u64, which is unix seconds.

That seconds value flows directly into proto fields documented as ms at five sites:

RPC Site Status before this PR
QueryService.ReadTx (v1alpha) src/serve/grpc/v1alpha/query.rs:998 pre-existing on main
QueryService.ReadTx (v1beta) src/serve/grpc/v1beta/query.rs:998 pre-existing on main
SyncService.ReadTip (v1alpha) src/serve/grpc/v1alpha/sync.rs:277 pre-existing on main
SyncService.ReadTip (v1beta) src/serve/grpc/v1beta/sync.rs:277 pre-existing on main
QueryService.SearchUtxos & ReadUtxos (via block_ref) src/serve/grpc/block_refs.rs:71 introduced in #1000

The first four route through LedgerContext::get_slot_timestamp (pallas trait, doc'd as seconds — correctly implemented by Dolos's adapter at src/adapters/mod.rs:208). The fifth bypasses the trait and calls chain_summary.slot_time(slot) directly because PR #1000 hoists the era summary per request — also correct in isolation, but seconds.

The fix

Convert at the boundary. Each gRPC site that writes into a // Block ms timestamp proto field multiplies the seconds-typed source value by 1000. Five surgical edits, total diff +21/−5.

The pallas LedgerContext::get_slot_timestamp trait is not changed — its contract is seconds and Dolos correctly implements it. The conversion belongs at the proto write boundary, not at the trait. The internal EraSummary::slot_time is also unchanged for the same reason.

Verification — built and probed a preprod snapshot

dolos bootstrap snapshot --variant full on preprod, dolos daemon, then grpcurl against each endpoint. Before this PR (PR #1000 alone), all five RPCs returned 10-digit timestamp values that decode to the correct date when read as seconds and to January 1970 when read as ms. After this PR, every endpoint returns the same value × 1000 — 13 digits, decoding correctly as ms:

RPC Before After
ReadTip (v1alpha + v1beta) 1777953075 (≈ 1970-01-21 as ms) 1777979745000 (2026-05-05 09:55:45 UTC)
ReadTx (v1alpha + v1beta) 1777567069 1777567069000 (2026-04-30 16:37:49 UTC)
ReadUtxos (v1alpha + v1beta) 1777567069 1777567069000 (2026-04-30 16:37:49 UTC)
SearchUtxos (v1alpha + v1beta) 1777575127 1777575127000 (2026-04-30 18:52:07 UTC)

Slot math sanity check: preprod's byron startTime + byron_slots × 20s + shelley_slots × 1s = slot_time(slot) lines up to the second for every value above. The conversion isn't reinterpreting anything — it's just promoting units.

Stacking

Based on fix/u5c-block-ref-populate. The diff against main will include PR #1000's changes plus this PR's five-site fix; the diff against fix/u5c-block-ref-populate is just the five edits.

Summary by CodeRabbit

  • New Features

    • Added centralized block reference metadata handling to improve consistency across gRPC query services.
  • Improvements

    • Standardized timestamp formatting to milliseconds across all API endpoints for consistent time representation.

Review Change Stack

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 28, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 3baf8393-c64a-4a2a-a4cf-511b41ad7232

📥 Commits

Reviewing files that changed from the base of the PR and between 342ba2b and e931df8.

📒 Files selected for processing (5)
  • src/serve/grpc/block_refs.rs
  • src/serve/grpc/v1alpha/query.rs
  • src/serve/grpc/v1alpha/sync.rs
  • src/serve/grpc/v1beta/query.rs
  • src/serve/grpc/v1beta/sync.rs
✅ Files skipped from review due to trivial changes (1)
  • src/serve/grpc/v1alpha/sync.rs
🚧 Files skipped from review as they are similar to previous changes (2)
  • src/serve/grpc/v1beta/sync.rs
  • src/serve/grpc/block_refs.rs

📝 Walkthrough

Walkthrough

This PR introduces a shared block reference helper (BlockRefData and fetch_block_refs) and systematically scales all gRPC endpoint timestamps from seconds to milliseconds (multiplied by 1000) across v1alpha and v1beta sync and query services.

Changes

Millisecond Timestamp Scaling Across gRPC Endpoints

Layer / File(s) Summary
Block reference helper with millisecond timestamps
src/serve/grpc/block_refs.rs
BlockRefData struct and fetch_block_refs async function deduplicate transactions, load era summary once, query block metadata per unique tx hash, and compute timestamps scaled by 1000 with storage failures mapped to Status::internal and decode errors logged as skipped entries.
v1alpha gRPC timestamp scaling
src/serve/grpc/v1alpha/query.rs, src/serve/grpc/v1alpha/sync.rs
read_tx and read_tip in v1alpha scale slot timestamps by 1000 (seconds to milliseconds) with zero fallback when timestamp is unavailable.
v1beta gRPC timestamp scaling
src/serve/grpc/v1beta/query.rs, src/serve/grpc/v1beta/sync.rs
read_tx and read_tip in v1beta apply the same millisecond scaling (×1000) to slot timestamps with zero fallback, matching v1alpha behavior.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Possibly related PRs

  • txpipe/dolos#1000: Extends fetch_block_refs-based prefetching to populate AnyUtxoData.block_ref with block-by-tx-hash metadata in gRPC responses.
  • txpipe/dolos#813: Modifies shared gRPC sync ReadTip/BlockRef construction to propagate slot timestamp, overlapping with the timestamp plumbing touched in this PR.

Suggested reviewers

  • scarmuega
  • gonzalezzfelipe

Poem

🐰 A rabbit hops through timestamp lands,
Scaling seconds by a thousand's hands,
From sync to query, left to right,
Milliseconds shine, the conversion's tight!
No moment skipped in this careful dance. ✨

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately describes the main change: converting timestamp values from seconds to milliseconds in gRPC ChainPoint and BlockRef fields.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@stnly-btcdefi stnly-btcdefi marked this pull request as ready for review May 28, 2026 10:58
@stnly-btcdefi stnly-btcdefi requested a review from scarmuega as a code owner May 28, 2026 10:58
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 28, 2026

Actionable comments posted: 0

Copy link
Copy Markdown
Contributor

@gonzalezzfelipe gonzalezzfelipe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This includes the changes already merged on #1000 , can you rebase for it to make it cleaner for review?

@stnly-btcdefi stnly-btcdefi force-pushed the fix/u5c-timestamp-ms-spec-compliance branch from 342ba2b to e931df8 Compare May 28, 2026 13:48
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 28, 2026

Actionable comments posted: 0

@scarmuega
Copy link
Copy Markdown
Member

Tests fail but for un-related reasons (a newly added lint check).
We'll merge as it is and fix lints of follow-up.

@scarmuega scarmuega merged commit d0f6c9e into txpipe:main May 28, 2026
9 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants