feat: support Concise Transaction Identifier (CTID) (XLS-37) #4418

RichardAH · 2023-02-13T13:51:04Z

High Level Overview of Change

RPC support for CTID

CTID is a network-aware short alternative to using transaction hashes that is based on the ledger sequence, transaction index and network ID of a validated transaction in a closed ledger.

This PR only adds CTID in the tx response

This implements XLS-37

Discussion: 0037 XLS-37d: Improved Concise Transaction Identifier (CTID) XRPL-Standards#91
Spec PR: XLS-37d: Improved Concise Transaction Identifier (CTID) XRPL-Standards#111

Type of Change

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Refactor (non-breaking change that only restructures code)
Tests (You added tests for code that already exists, or your new feature included in this PR)
Documentation Updates
Release

…rk id > 1024

Co-authored-by: Rome Reginelli <mduo13@gmail.com>

intelliot · 2023-04-25T16:49:39Z

@RichardAH says this is ready to be reviewed. At this time, he does not plan to do more for this. However, he will check with Denis to see if he has done some work on it.

dangell7 · 2023-04-27T19:45:43Z

@intelliot There is another commit that I have here. Basically if you add the network config, all the env functions wont work. Like fund, pay etc. So this is basically a workaround autofill.

This PR will likely fail because this is not included. I can cherry pick this commit in if you'd like or I can rewrite the tests to manually inject the networkID

intelliot · 2023-04-27T23:16:30Z

That's up to you, but I would recommend proposing a PR that, as far as you know, would be ready to merge into develop (if it passes code review and testing). That could look like additional commits/merges into this PR (RichardAH:ctim), or closing this PR and opening a new one from a fork/branch that you create. (The latter is worth considering since this PR's author is @RichardAH rather than @dangell7; but in the end, what really matters is that there is a PR that is complete and ready to go.)

dangell7 · 2023-04-28T15:02:42Z

@intelliot This is now ready for review.

RichardAH · 2023-05-16T09:45:20Z

This should be merged prior to any parallel / side chains being launched

src/ripple/rpc/handlers/Tx.cpp

Builds/CMake/RippledCore.cmake

src/ripple/app/tx/impl/Transactor.cpp

src/ripple/app/ledger/impl/LedgerMaster.cpp

src/ripple/rpc/handlers/Tx.cpp

src/ripple/app/ledger/impl/LedgerMaster.cpp

fix bad seperator clang-format

RichardAH · 2023-07-20T22:03:57Z

I believe I was initially brought in to review why tests were failing. That led to my diagnosis. I further noticed that Nik's last comment seemed to be unresolved, and so I reviewed that part of the code too. I can confirm now that the tests pass, and that Nik's objection is resolved, and so I add my approval, but with the disclosure that I have not reviewed all of the code here, leaving that up to the ones who came before me.

Thanks john

intelliot · 2023-07-20T23:12:38Z

@RichardAH from your perspective, is this PR now ready to merge, or would you prefer to have @nbougalis take a look first?

Also, it is suggested that this PR be squashed into 1 commit when it is merged, with the following commit message. If this message isn't ideal, please feel free to propose an alternative.

feat: support Concise Transaction Identifier (CTID) (XLS-37) (#4418)

The XLS-37 CTID (Concise Transaction ID) is a network-aware tx
identifier which provides a way to efficiently locate a specific
transaction without relying on transaction hashes. A CTID encodes the
sequence number of the ledger that includes the tx, the transaction's
index in that ledger, and the network ID. With the CTID, users can
identify transactions and confirm their results. This applies even for
transactions on sidechains, which may be difficult to find with only a
transaction hash.

Additionally, CTIDs require less storage space than transaction hashes,
which can be beneficial for databases storing millions of transactions.

The XLS-37 specification can be found at:
  https://github.com/XRPLF/XRPL-Standards/pull/111

Add support for running a node on a different network. There is a new
error code, `rpcWRONG_NETWORK`, returned when the requested CTID's
network ID does not match the node's network ID. The error message looks
like:

  Wrong network. You should submit this request to a node running on
  NetworkID: <net_id>

* Add RPC support for the CTID format
  * tx - you can specify "ctid", which is the CTID (16 hex digits, in a
    string, always starting with "C")
  * When retrieving a tx, the "ctid" may be returned
* Add support for encoding, decoding, and validating CTIDs
* Add tests

---------

Co-authored-by: Rome Reginelli <mduo13@gmail.com>
Co-authored-by: Denis Angell <dangell@transia.co>

Note: I updated XLS037 to XLS-37 to match the typical formatting used when other XLS specs have been referenced.

RichardAH · 2023-07-21T07:11:52Z

I will do a final review shortly. It’s been a long time since I wrote this code and I need to squash and review to see what ended up in it after other people’s changes

RichardAH · 2023-07-21T07:28:29Z

LGTM

intelliot · 2023-07-26T17:45:02Z

LGTM

I take this to mean that this PR is ready to merge. @manojsdoshi - please consider this PR for the next release, and squash with the commit message in #4418 (comment)

The XLS-37 CTID (Concise Transaction ID) is a network-aware tx identifier which provides a way to efficiently locate a specific transaction without relying on transaction hashes. A CTID encodes the sequence number of the ledger that includes the tx, the transaction's index in that ledger, and the network ID. With the CTID, users can identify transactions and confirm their results. This applies even for transactions on sidechains, which may be difficult to find with only a transaction hash. Additionally, CTIDs require less storage space than transaction hashes, which can be beneficial for databases storing millions of transactions. The XLS-37 specification can be found at: XRPLF/XRPL-Standards#111 Add support for running a node on a different network. There is a new error code, `rpcWRONG_NETWORK`, returned when the requested CTID's network ID does not match the node's network ID. The error message looks like: Wrong network. You should submit this request to a node running on NetworkID: <net_id> * Add RPC support for the CTID format * tx - you can specify "ctid", which is the CTID (16 hex digits, in a string, always starting with "C") * When retrieving a tx, the "ctid" may be returned * Add support for encoding, decoding, and validating CTIDs * Add tests --------- Co-authored-by: Rome Reginelli <mduo13@gmail.com> Co-authored-by: Denis Angell <dangell@transia.co>

) The XLS-37 CTID (Concise Transaction ID) is a network-aware tx identifier which provides a way to efficiently locate a specific transaction without relying on transaction hashes. A CTID encodes the sequence number of the ledger that includes the tx, the transaction's index in that ledger, and the network ID. With the CTID, users can identify transactions and confirm their results. This applies even for transactions on sidechains, which may be difficult to find with only a transaction hash. Additionally, CTIDs require less storage space than transaction hashes, which can be beneficial for databases storing millions of transactions. The XLS-37 specification can be found at: XRPLF/XRPL-Standards#111 Add support for running a node on a different network. There is a new error code, `rpcWRONG_NETWORK`, returned when the requested CTID's network ID does not match the node's network ID. The error message looks like: Wrong network. You should submit this request to a node running on NetworkID: <net_id> * Add RPC support for the CTID format * tx - you can specify "ctid", which is the CTID (16 hex digits, in a string, always starting with "C") * When retrieving a tx, the "ctid" may be returned * Add support for encoding, decoding, and validating CTIDs * Add tests --------- Co-authored-by: Rome Reginelli <mduo13@gmail.com> Co-authored-by: Denis Angell <dangell@transia.co> --------- Co-authored-by: Rome Reginelli <mduo13@gmail.com> Co-authored-by: Elliot Lee <github.public@intelliot.com> Co-authored-by: Denis Angell <dangell@transia.co>

(XLS-37) (XRPLF#4418) The XLS-37 CTID (Concise Transaction ID) is a network-aware tx identifier which provides a way to efficiently locate a specific transaction without relying on transaction hashes. A CTID encodes the sequence number of the ledger that includes the tx, the transaction's index in that ledger, and the network ID. With the CTID, users can identify transactions and confirm their results. This applies even for transactions on sidechains, which may be difficult to find with only a transaction hash. Additionally, CTIDs require less storage space than transaction hashes, which can be beneficial for databases storing millions of transactions. The XLS-37 specification can be found at: XRPLF/XRPL-Standards#111 Add support for running a node on a different network. There is a new error code, `rpcWRONG_NETWORK`, returned when the requested CTID's network ID does not match the node's network ID. The error message looks like: Wrong network. You should submit this request to a node running on NetworkID: <net_id> * Add RPC support for the CTID format * tx - you can specify "ctid", which is the CTID (16 hex digits, in a string, always starting with "C") * When retrieving a tx, the "ctid" may be returned * Add support for encoding, decoding, and validating CTIDs * Add tests --------- Co-authored-by: Rome Reginelli <mduo13@gmail.com> Co-authored-by: Denis Angell <dangell@transia.co> --------- Co-authored-by: Rome Reginelli <mduo13@gmail.com> Co-authored-by: Elliot Lee <github.public@intelliot.com> Co-authored-by: Denis Angell <dangell@transia.co>

(XLS-37) (XRPLF#4418) The XLS-37 CTID (Concise Transaction ID) is a network-aware tx identifier which provides a way to efficiently locate a specific transaction without relying on transaction hashes. A CTID encodes the sequence number of the ledger that includes the tx, the transaction's index in that ledger, and the network ID. With the CTID, users can identify transactions and confirm their results. This applies even for transactions on sidechains, which may be difficult to find with only a transaction hash. Additionally, CTIDs require less storage space than transaction hashes, which can be beneficial for databases storing millions of transactions. The XLS-37 specification can be found at: XRPLF/XRPL-Standards#111 Add support for running a node on a different network. There is a new error code, `rpcWRONG_NETWORK`, returned when the requested CTID's network ID does not match the node's network ID. The error message looks like: Wrong network. You should submit this request to a node running on NetworkID: <net_id> * Add RPC support for the CTID format * tx - you can specify "ctid", which is the CTID (16 hex digits, in a string, always starting with "C") * When retrieving a tx, the "ctid" may be returned * Add support for encoding, decoding, and validating CTIDs * Add tests --------- Co-authored-by: Rome Reginelli <mduo13@gmail.com> Co-authored-by: Denis Angell <dangell@transia.co> --------- Co-authored-by: Rome Reginelli <mduo13@gmail.com> Co-authored-by: Elliot Lee <github.public@intelliot.com> Co-authored-by: Denis Angell <dangell@transia.co> Signed-off-by: Manoj Doshi <mdoshi@ripple.com>

(XLS-37) (XRPLF#4418) The XLS-37 CTID (Concise Transaction ID) is a network-aware tx identifier which provides a way to efficiently locate a specific transaction without relying on transaction hashes. A CTID encodes the sequence number of the ledger that includes the tx, the transaction's index in that ledger, and the network ID. With the CTID, users can identify transactions and confirm their results. This applies even for transactions on sidechains, which may be difficult to find with only a transaction hash. Additionally, CTIDs require less storage space than transaction hashes, which can be beneficial for databases storing millions of transactions. The XLS-37 specification can be found at: XRPLF/XRPL-Standards#111 Add support for running a node on a different network. There is a new error code, `rpcWRONG_NETWORK`, returned when the requested CTID's network ID does not match the node's network ID. The error message looks like: Wrong network. You should submit this request to a node running on NetworkID: <net_id> * Add RPC support for the CTID format * tx - you can specify "ctid", which is the CTID (16 hex digits, in a string, always starting with "C") * When retrieving a tx, the "ctid" may be returned * Add support for encoding, decoding, and validating CTIDs * Add tests --------- Co-authored-by: Rome Reginelli <mduo13@gmail.com> Co-authored-by: Denis Angell <dangell@transia.co> Signed-off-by: Manoj Doshi <mdoshi@ripple.com>

(XLS-37) (XRPLF#4418) The XLS-37 CTID (Concise Transaction ID) is a network-aware tx identifier which provides a way to efficiently locate a specific transaction without relying on transaction hashes. A CTID encodes the sequence number of the ledger that includes the tx, the transaction's index in that ledger, and the network ID. With the CTID, users can identify transactions and confirm their results. This applies even for transactions on sidechains, which may be difficult to find with only a transaction hash. Additionally, CTIDs require less storage space than transaction hashes, which can be beneficial for databases storing millions of transactions. The XLS-37 specification can be found at: XRPLF/XRPL-Standards#111 Add support for running a node on a different network. There is a new error code, `rpcWRONG_NETWORK`, returned when the requested CTID's network ID does not match the node's network ID. The error message looks like: Wrong network. You should submit this request to a node running on NetworkID: <net_id> * Add RPC support for the CTID format * tx - you can specify "ctid", which is the CTID (16 hex digits, in a string, always starting with "C") * When retrieving a tx, the "ctid" may be returned * Add support for encoding, decoding, and validating CTIDs * Add tests --------- Co-authored-by: Rome Reginelli <mduo13@gmail.com> Co-authored-by: Denis Angell <dangell@transia.co>

(XLS-37) (XRPLF#4418) The XLS-37 CTID (Concise Transaction ID) is a network-aware tx identifier which provides a way to efficiently locate a specific transaction without relying on transaction hashes. A CTID encodes the sequence number of the ledger that includes the tx, the transaction's index in that ledger, and the network ID. With the CTID, users can identify transactions and confirm their results. This applies even for transactions on sidechains, which may be difficult to find with only a transaction hash. Additionally, CTIDs require less storage space than transaction hashes, which can be beneficial for databases storing millions of transactions. The XLS-37 specification can be found at: XRPLF/XRPL-Standards#111 Add support for running a node on a different network. There is a new error code, `rpcWRONG_NETWORK`, returned when the requested CTID's network ID does not match the node's network ID. The error message looks like: Wrong network. You should submit this request to a node running on NetworkID: <net_id> * Add RPC support for the CTID format * tx - you can specify "ctid", which is the CTID (16 hex digits, in a string, always starting with "C") * When retrieving a tx, the "ctid" may be returned * Add support for encoding, decoding, and validating CTIDs * Add tests --------- Co-authored-by: Rome Reginelli <mduo13@gmail.com> Co-authored-by: Denis Angell <dangell@transia.co> Signed-off-by: Manoj Doshi <mdoshi@ripple.com>

) * add CTIM to tx rpc --------- Co-authored-by: Rome Reginelli <mduo13@gmail.com> Co-authored-by: Elliot Lee <github.public@intelliot.com> Co-authored-by: Denis Angell <dangell@transia.co>

cindyyan317 · 2023-09-27T12:45:33Z

src/ripple/protocol/ErrorCodes.h

@@ -47,8 +47,8 @@ enum error_code_i {
    rpcJSON_RPC = 2,
    rpcFORBIDDEN = 3,

+    rpcWRONG_NETWORK = 4,


Do we need to add "rpcWRONG_NETWORK" in unorderedErrorInfos ( ErrorCodes.cpp )

sublimator · 2023-09-28T03:47:04Z

src/ripple/rpc/CTID.h

+
+    uint64_t ctidValue =
+        ((0xC000'0000ULL + static_cast<uint64_t>(ledger_seq)) << 32) +
+        (static_cast<uint64_t>(txn_index) << 16) + network_id;


Bit late :)

But yeah {network_id, ledger_seq, txn_index}
https://github.com/XRPLF/XRPL-Standards/discussions/91\#discussioncomment-5230494 https://github.com/XRPLF/XRPL-Standards/discussions/91\#discussioncomment-5079139

RichardAH and others added 9 commits February 12, 2023 18:15

add CTIM to tx rpc

5e4f8ea

Add NetworkID field to Transaction common fields, enforced when netwo…

1daf5f7

…rk id > 1024

Update src/ripple/app/tx/impl/Transactor.cpp

91b0aae

Co-authored-by: Rome Reginelli <mduo13@gmail.com>

[FOLD] add const, missing include

2752108

[FOLD] fix: apply clang-format patch

3782ab7

preliminary support for xls37 (Improved Concise Transaction Identifier)

1aaac47

fix txn hash lookup from index

04be722

rename CTIM to CTID

fec9fb7

ctid tests

281f20b

intelliot requested review from pwang200, nbougalis, HowardHinnant and mDuo13 April 25, 2023 16:49

dangell7 added 3 commits April 28, 2023 08:44

add autofill + refactor

2dad662

Merge branch 'develop' into ctim

f39a839

clang-format

ccea8c9

mvadari mentioned this pull request May 17, 2023

Add CTID support to the Explorer ripple/explorer#707

Closed

intelliot assigned HowardHinnant and mDuo13 May 19, 2023

intelliot added the Sidechain label May 22, 2023

intelliot changed the title ~~Add Improved Concise Transaction Identifier support~~ Add Improved Concise Transaction Identifier support (CTID) May 26, 2023

RichardAH changed the title ~~Add Improved Concise Transaction Identifier support (CTID)~~ Ensure USERS can FIND their TXN on SIDECHAINS-- Concise Transaction Identifier (CTID) May 26, 2023

intelliot requested a review from seelabs May 26, 2023 18:56

HowardHinnant suggested changes May 26, 2023

View reviewed changes

add ' seperator to hex constants

3a96000

fix bad seperator clang-format

intelliot unassigned thejohnfreeman Jul 20, 2023

intelliot changed the title ~~feat: support Concise Transaction Identifier (CTID) (XLS037)~~ feat: support Concise Transaction Identifier (CTID) (XLS-37) Jul 20, 2023

Merge branch 'develop' into ctim

61d2f5b

intelliot added the Passed Passed code review & PR owner thinks it's ready to merge. Perf sign-off may still be required. label Jul 26, 2023

This was referenced Jul 26, 2023

Add CTID support XRPLF/xrpl.js#2409

Closed

Add CTID support XRPLF/xrpl-py#626

Open

manojsdoshi merged commit 5530a0b into XRPLF:develop Aug 18, 2023
15 checks passed

manojsdoshi mentioned this pull request Aug 18, 2023

Proposed 1.12-b3 #4664

Closed

intelliot added the Clio Reviewed label Aug 22, 2023

JST5000 mentioned this pull request Sep 13, 2023

feat: Add optional CTID field to Tx XRPLF/xrpl.js#2477

Merged

2 tasks

mDuo13 mentioned this pull request Sep 22, 2023

Document ctid field XRPLF/xrpl-dev-portal#2157

Closed

dangell7 mentioned this pull request Sep 26, 2023

CTID sync rippled Xahau/xahaud#114

Merged

cindyyan317 reviewed Sep 27, 2023

View reviewed changes

sublimator reviewed Sep 28, 2023

View reviewed changes

mDuo13 mentioned this pull request Oct 18, 2023

CTID in tx command returns invalidParams on lowercase hex #4776

Open

intelliot mentioned this pull request Oct 24, 2023

Patch CTID #4738

Open

12 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: support Concise Transaction Identifier (CTID) (XLS-37) #4418

feat: support Concise Transaction Identifier (CTID) (XLS-37) #4418

RichardAH commented Feb 13, 2023 •

edited by intelliot

Loading

intelliot commented Apr 25, 2023

dangell7 commented Apr 27, 2023

intelliot commented Apr 27, 2023

dangell7 commented Apr 28, 2023

RichardAH commented May 16, 2023

RichardAH commented Jul 20, 2023

intelliot commented Jul 20, 2023

RichardAH commented Jul 21, 2023

RichardAH commented Jul 21, 2023

intelliot commented Jul 26, 2023 •

edited

Loading

cindyyan317 Sep 27, 2023

sublimator Sep 28, 2023

feat: support Concise Transaction Identifier (CTID) (XLS-37) #4418

feat: support Concise Transaction Identifier (CTID) (XLS-37) #4418

Conversation

RichardAH commented Feb 13, 2023 • edited by intelliot Loading

High Level Overview of Change

This implements XLS-37

Type of Change

intelliot commented Apr 25, 2023

dangell7 commented Apr 27, 2023

intelliot commented Apr 27, 2023

dangell7 commented Apr 28, 2023

RichardAH commented May 16, 2023

RichardAH commented Jul 20, 2023

intelliot commented Jul 20, 2023

RichardAH commented Jul 21, 2023

RichardAH commented Jul 21, 2023

intelliot commented Jul 26, 2023 • edited Loading

cindyyan317 Sep 27, 2023

Choose a reason for hiding this comment

sublimator Sep 28, 2023

Choose a reason for hiding this comment

RichardAH commented Feb 13, 2023 •

edited by intelliot

Loading

intelliot commented Jul 26, 2023 •

edited

Loading