Indexer Support for Chained Transactions

[zhangyouxin]

Indexer Support Chained Transactions

This issue outlines an SDK solution for implementing chained transactions and explores the possible challenges of integrating this feature into ckb indexer.

Motivation

I am a Typescript user, to support chained transactions, I have created my own SDK to manage created-pending-cells and dead-cells on my App, thanks to Java SDK, GO SDK and JavaScript SDK, I have been inspired by them and implemented my own SDK that supports ckb-indexer-like cell fetcher. It supports get_cells(): Paginated<Cell> method.

And I have noticed that ckb indexer get_cells method may already supports filtering dead-cells: https://github.com/nervosnetwork/ckb/blob/develop/util/indexer/src/pool.rs#L10 , look like there are some techniq obstacles to implent returning created-pending-cells in get_cells method? If this feature could be integrated, downstream tooling would benefit significantly from the ability to use the ckb-indexer in different languages instead of having to implement numerous SDKs.

SDK Design

As described in the picture, I have a storage component storing dead-cells and created-pending-cells, just like indexer pool, my sdk will firstly fetch live cells from CKB node, if cells not enough for a page, then get more cells from storage, both indexer and storage side can use the cursor to remember the pagination position.

declare const encodeCursor(queryType, codeHash, hashType, args, blockNumber, txIndex, outputIndex, txHash): string;

function getCells({ limit, searchKey }){
   const { objects, cursor } = await fetchFromCkbIndexer(searchKey, limit);
   
   const liveCells = storage.filterDeadCells(objects);
   
   if(liveCells.length >= limit){
     const resultCells = liveCells.slice(0, limit);
   } else {
     // if liveCells not enough，append created pending cells
     cosnt resultCells = liveCells.concat(createdPendingCells).slice(0, limit)
   }
   
   const lastCell = resultCells[-1];
   const lastCursor = encodeCursor("lock", lastCell.codeHash, lastCell.hashType, lastCell.args, lastCell.blockNumber, lastCell.txIndex, lastCell.outputIndex, lastCell.txHash);
   return { objects: resultCells, cursor: lastCursor };
}

In the design above, it may be hard for the created-pending-cells to generate a coherent last-cursor, because the transacitons are not confirmed, we don't know the blockNumber and transactionIndex, but may be it does not matter, I just set blockNumber to Math.MaxInteger, transactionIndex to 0x0, then use txHash and outputIndex to locate the created-pending-cells.

[zhangyouxin]

Where should we implement it

TLDR; we decide to implement it in SDK

After some investigation, I have collected some pros and cons for implementing chained transaction support in ckb-indexer and SDKs:

• 💻 Option 1: Implement in the ckb-indexer

  • ✅ Can use pending cells across applications/users/wallets
  • ✅ Downstream (go/java/rust/JavaScript) SDKs do not need to implement CKB indexer logic
  • ❗️ Cursor encoding is incompatible.
  • ❗️ Ckb-indexer and ckb are asynchronous, and there may be delays to use pending cells.

• 💻 Option 2: Implement in the SDK

  • ✅ The pending cells are managed locally so that users can use pending cells faster.
  • ❗️It is necessary to implement the ckb filter protocol.
  • ❗️Pending cells cannot be utilized in different DApps/wallets/devices.

Table view:

	Implement in the indexer	Implement in the SDK
can use pending cell across applications/users/wallets	✅
SDKs does not need to implement CKB indexer logic	✅
indexer cursor encoding is compatible		✅
can use pending cell just in time		✅

The benefit of implementing in ckb-indexer is that each language does not need to implement the logic of ckb-indexer separately, and it supports sharing pending cells between different DApps and wallets. However, since ckb-indexer and ckb are not synchronized in real time, downstream SDKs may need to poll ckb-indexer to obtain pending cells. Additionally, since pending cells do not have block number information, the encoding rules of the cursor may change.

As of now, we could pick option 2 as our solution, here are some reasons:

• Option 1 is hard to implement due to some compatibility issue
• the two issues generated by implementing in the SDK can be solved. Issue 1 can be solved by development work and testing. Issue 2 is a rare scenario, and sharing pending cells between different Dapps may not be a real requirement, as chained transactions are often assembled in the same application.