Give ways to retrieve full datums associated to datum hashes

[KtorZ]

Describe your idea, in simple words.

Kupo should give ways for application to retrieve datum associated with specific datum hashes. This could be achieved in two ways; either:

• By storing datums found in transactions while syncing the chain; and then by providing an API entrypoint to inspect those datums.
• Or, by resolving datum hashes on-demand using the chain-sync protocol from the chain-provider.

The trade-off between the two approaches is mainly about when the cost is paid. With the former, we pay the cost for every transaction of every block, since we'd now need the consumer process to also look for and store datums. This is possibly a minor cost addition though and need to be checked.

In the latter however, the cost only occurs when requests are made to resolve a specific datum hash, and would likely be a matter of a few milliseconds per datum hash. In principle, this also sounds fine because, resolving datums would likely only happen for submitting transactions which is ultimately capped anyway by the L1 throughput.

Why is it a good idea?

It seems to be a crucial but still missing pieces for making kupo a truly useful chain-index for DApps developers. Rightfully so.

Are you willing to work on it yourself?

Yes.

[waalge]

🤔 I thought I'd left a comment here before...Anyway.

Just to point at https://github.com/waalge/kupo/tree/waalge/add-datums as a stop gap attempt while this is in the works.

[nielstron]

Just to give some perspective on my use case:
I am usually querying the UTxO set of an address. If that address is a contract, I then separately query the datums of each encountered datum hash.

I am more than happy to just continue using that setup, namely that would just mean I would want to use an endpoint /v1/script_datum/{datum_hash}that returns the datum corresponding to that hash, if available.

It would also be handy to have the datum directly in v1/matches but I suggest resolving datums be optionally turned on via a flag like ?resolve_datum_hash or the likes and not having them stored permanently nor resolved in every query.

[KtorZ]

@nielstron what you're describing is exactly how I started (and planned finishing) to implementing this. Namely, store the datums separately and give ways to query them by hashes, but allows a "deep fetch" of the matches results through a query flag to also resolve datum hashes.