GET /liquid/api/address/<address>/utxo returns "Too many history entries" without any possiblity of paging

[thomaseizinger]

Here is an address I just grabbed of the UI where this happens: https://blockstream.info/liquid/api/address/QLFdUboUPJnUzvsXKu83hUtrQ1DuxyggRg/utxo

❯ curl https://blockstream.info/liquid/api/address/QLFdUboUPJnUzvsXKu83hUtrQ1DuxyggRg/utxo
Too many history entries%                                                                                                                                                                                                                                      
❯

Is there a way of how I can still access the UTXOs?

[shesek]

I'm afraid that this accessing the UTXO set of "too popular" addresses is currently not possible. This might be revisited in the future.

The only thing you could do today is get the transaction history, which doesn't have a "too popular" limit, and figure out the current UTXO set based on that. One way to do that is with two passes over the transaction list, one to populate a set of (txid,vout,amount) for every funded output, and a second pass to remove the outputs spent by the inputs.

(If tx A spends tx B and they both confirm in the same block, its possible for the spending tx A to appear in the transaction history list before the parent tx B that creates the spent output. Doing two passes is a simple way to deal with this, though not the most efficient one.)

[thomaseizinger]

What is the limit at which the UTXOs set is becoming inaccessible?

This particular address has 478865 transactions 😅. I don't think we'll be hitting this number anytime soon in our application but knowing where limit is would be nice!

[shesek]

It's 500 by default, can be increased with --utxos-limit <N>.

[php22032]

Can't support paging?

like:
GET /block/:hash/txs[/:start_index]

[shesek]

Not easily I'm afraid, the way the utxo database is currently structured is incompatible with (efficient) paging and doesn't work well when the per-address utxo set is huge. Making this work would require some re-designing, which might be done some day, but is not currently prioritized.