feat(database): Track transaction Datum

[arepala-uml]

1. Added changes to SetTransaction so every inline output datum and witness-set datum is saved in the metadata DB.
2. Added database.GetDatum and exposed it via LedgerState.Datum where it can be helped by giving other components a clean way to fetch stored datums by hash.

Closes #1015

---

Summary by cubic

Track and persist transaction datums in the metadata DB and add a lookup to fetch them by hash. This makes inline output and witness-set datums accessible via LedgerState.Datum.

• New Features
  • Save inline output and witness-set datums during SetTransaction.
  • Store raw CBOR and Blake2b256 hash in metadata.
  • Add database.GetDatum and expose it via LedgerState.Datum for easy retrieval.

^{Written for commit 1587c41. Summary will update automatically on new commits.}

Summary by CodeRabbit

• New Features

  • Added hash-based datum retrieval so clients can fetch stored datums by hash.
  • Ledger API extended to expose datum read operations.

• Enhancements

  • Transactions now persist inline and witness datums automatically during processing, including safe handling of nil/empty datums.

• Bug Fixes

  • Improved error reporting for missing or invalid datum lookups.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

📝 Walkthrough

Walkthrough

This pull request adds datum tracking across three layers: a new Database method GetDatum(hash []byte, txn *Txn) to retrieve datums by hash (with input validation and Blake2b256 conversion), two new SQLite metadata helpers storeTransactionDatums and storeDatum to persist inline datums from produced UTxOs and transaction witnesses (CBOR marshaling, Blake2b256 hashing, and conditional storage), and a LedgerState method Datum(hash []byte) that delegates to the database retrieval. Datum storage is invoked in SetTransaction after certificate processing and propagates errors upward.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~35 minutes

• Verify CBOR marshaling paths (Cbor vs MarshalCBOR) and correct handling of nil/empty datums in storeDatum.
• Confirm Blake2b256 hashing and hash conversion/format used by GetDatum matches stored keys.
• Review integration point in SetTransaction for ordering and error propagation ("store datums failed: ...").
• Check iterations over produced UTxOs and witnesses for completeness and duplicate handling.

Possibly related PRs

• feat: changes to track transaction metadata #980: Modifies the same SetTransaction implementation in database/plugin/metadata/sqlite/transaction.go; likely related due to overlapping transaction persistence logic.

Pre-merge checks and finishing touches

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'feat(database): Track transaction Datum' clearly summarizes the main change: adding datum tracking functionality to the database layer.
Linked Issues check	✅ Passed	The pull request implements all requirements from issue #1015: tracks transaction datums across different transaction parts and provides lookup functionality via LedgerState.Datum.
Out of Scope Changes check	✅ Passed	All changes are directly related to the stated objectives: datum tracking, storage in metadata DB, and retrieval via GetDatum/LedgerState.Datum methods.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch track_transactio_datum

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between fe78df4 and 1587c41.

📒 Files selected for processing (1)

• database/plugin/metadata/sqlite/transaction.go (1 hunks)

🧰 Additional context used

🧠 Learnings (1)

📚 Learning: 2025-10-24T21:56:15.978Z

Learnt from: wolf31o2
Repo: blinklabs-io/dingo PR: 975
File: database/models/transaction.go:21-26
Timestamp: 2025-10-24T21:56:15.978Z
Learning: In database/models/transaction.go, the Transaction struct uses a dual foreign-key pattern for UTXO associations: Outputs []Utxo uses TransactionID→ID (where UTXOs were created), and Inputs []Utxo uses SpentAtTxId→Hash (where UTXOs were consumed). Explicit GORM tags `gorm:"foreignKey:SpentAtTxId;references:Hash"` for Inputs and `gorm:"foreignKey:TransactionID;references:ID"` for Outputs prevent association conflicts and preserve UTXO provenance.

Applied to files:

• database/plugin/metadata/sqlite/transaction.go

🧬 Code graph analysis (1)

database/plugin/metadata/sqlite/transaction.go (3)

database/plugin/metadata/sqlite/database.go (1)

• MetadataStoreSqlite (37-45)

database/models/transaction.go (2)

• Transaction (20-34)
• Transaction (36-38)

database/models/datum.go (2)

• Datum (17-22)
• Datum (24-26)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)

• GitHub Check: cubic · AI code reviewer
• GitHub Check: nilaway
• GitHub Check: Analyze (go)

🔇 Additional comments (3)

database/plugin/metadata/sqlite/transaction.go (3)

685-687: LGTM! Datum storage correctly integrated.

The datum storage is properly integrated after certificate processing with appropriate error handling. The error context is clear, and since this runs within the same database transaction, consistency is maintained.

---

717-742: LGTM! Datum marshaling and storage logic is sound.

The method correctly handles edge cases:

• Returns early for nil datums (line 723-725)
• Falls back to MarshalCBOR() when Cbor() returns empty (line 727-733)
• Skips storage of empty datums (line 734-736)

The Blake2b256 hashing and subsequent storage via SetDatum are appropriate.

---

692-715: Code is correct—duplicate datum handling is properly implemented.

The verification confirms that SetDatum handles duplicate datums gracefully. It uses GORM's OnConflict clause with UpdateAll: true (line 64 of database/plugin/metadata/sqlite/datum.go), which transparently upserts records on hash collision. When the same datum hash appears multiple times across UTxOs or witnesses, subsequent calls will update the existing record rather than throwing an error. The storeTransactionDatums implementation is therefore correct.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🧹 Nitpick comments (2)

database/datum.go (1)

45-47: Consider a more descriptive error message for invalid input.

The error message "datum not found" for an empty hash is misleading, as this is an invalid input rather than a missing datum. Consider using errors.New("invalid hash: hash cannot be empty") to clarify this is a validation error.

database/plugin/metadata/sqlite/transaction.go (1)

727-736: Consider logging when datum marshaling produces empty data.

The function silently returns nil when a non-nil datum marshals to empty bytes (line 734-736). While this might be intentional for handling optional datums, it could mask unexpected cases where a datum should contain data but doesn't.

Consider adding a debug log:

 	if len(rawDatum) == 0 {
 		var err error
 		rawDatum, err = datum.MarshalCBOR()
 		if err != nil {
 			return fmt.Errorf("marshal datum: %w", err)
 		}
 	}
 	if len(rawDatum) == 0 {
+		d.logger.Debug("skipping empty datum", "slot", slot)
 		return nil
 	}

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 1db1871 and fe78df4.

📒 Files selected for processing (3)

• database/datum.go (2 hunks)
• database/plugin/metadata/sqlite/transaction.go (1 hunks)
• ledger/state.go (1 hunks)

🧰 Additional context used

🧠 Learnings (2)

📚 Learning: 2025-10-24T21:56:15.978Z

Learnt from: wolf31o2
Repo: blinklabs-io/dingo PR: 975
File: database/models/transaction.go:21-26
Timestamp: 2025-10-24T21:56:15.978Z
Learning: In database/models/transaction.go, the Transaction struct uses a dual foreign-key pattern for UTXO associations: Outputs []Utxo uses TransactionID→ID (where UTXOs were created), and Inputs []Utxo uses SpentAtTxId→Hash (where UTXOs were consumed). Explicit GORM tags `gorm:"foreignKey:SpentAtTxId;references:Hash"` for Inputs and `gorm:"foreignKey:TransactionID;references:ID"` for Outputs prevent association conflicts and preserve UTXO provenance.

Applied to files:

• database/plugin/metadata/sqlite/transaction.go

📚 Learning: 2025-10-22T20:19:26.360Z

Learnt from: wolf31o2
Repo: blinklabs-io/dingo PR: 971
File: utxorpc/sync_test.go:91-91
Timestamp: 2025-10-22T20:19:26.360Z
Learning: In utxorpc/sync_test.go, tests should not construct database/models.Block directly for sync server use. Instead, build a models.Block only as a container for CBOR and call models.Block.Decode() to obtain a github.com/blinklabs-io/gouroboros/ledger/common.Block (or the repository’s ledger.Block interface), and use that decoded value in the test.

Applied to files:

• database/datum.go

🧬 Code graph analysis (3)

ledger/state.go (1)

database/models/datum.go (2)

• Datum (17-22)
• Datum (24-26)

database/plugin/metadata/sqlite/transaction.go (3)

database/plugin/metadata/sqlite/database.go (1)

• MetadataStoreSqlite (37-45)

database/models/transaction.go (2)

• Transaction (20-34)
• Transaction (36-38)

database/models/datum.go (2)

• Datum (17-22)
• Datum (24-26)

database/datum.go (4)

database/database.go (2)

• Database (45-50)
• New (118-154)

database/txn.go (1)

• Txn (27-34)

database/models/datum.go (2)

• Datum (17-22)
• Datum (24-26)

database/plugin/metadata/sqlite/database.go (1)

• New (48-120)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

• GitHub Check: cubic · AI code reviewer

🔇 Additional comments (4)

database/datum.go (1)

18-22: LGTM!

The added imports support the new GetDatum functionality appropriately.

ledger/state.go (1)

198-201: LGTM!

The Datum method provides a clean public API for datum retrieval, properly delegating to the database layer. The comment clearly indicates its purpose for implementing issue #741.

database/plugin/metadata/sqlite/transaction.go (2)

685-690: LGTM!

The integration of datum storage into SetTransaction is well-placed after certificate processing, with proper error handling and propagation.

---

692-715: LGTM!

The function correctly handles both inline datums from transaction outputs and witness-set datums. The copy at line 709 is good practice to avoid loop variable reuse issues when taking addresses.

[cubic-dev-ai]

No issues found across 3 files