Add RPC for server_definitions to rippled

[mvadari]

High Level Overview of Change

Add a new RPC / WS call: server_definitions, which returns an SDK-compatible definitions.json generated from the rippled instance currently running. This will alleviate definitions juggling, especially with side chains which may have exotic features and serialized fields. Software will be able to query server_definitions against a node from the network they want to connect to and immediately know how to speak that node's language, even if new features are added to it in the future (as long as there are no new serialized types that the software doesn't know how to serialize/deserialize).

Example:

> {"command": "server_definitions"}
< {
    "result": {
        "FIELDS": [
            [
                "Generic",
                {
                    "isSerialized": false,
                    "isSigningField": false,
                    "isVLEncoded": false,
                    "nth": 0,
                    "type": "Unknown"
                }
            ],
            [
                "Invalid",
                {
                    "isSerialized": false,
                    "isSigningField": false,
                    "isVLEncoded": false,
                    "nth": -1,
                    "type": "Unknown"
                }
            ],
            [
                "ObjectEndMarker",
                {
                    "isSerialized": false,
                    "isSigningField": true,
                    "isVLEncoded": false,
                    "nth": 1,
                    "type": "STObject"
                }
            ],
        ...

Context of Change

I took #4402 and modified it to avoid using magic_enum. Closes #3657.

Type of Change

• New feature (non-breaking change which adds functionality)

Test Plan

I haven't written tests yet - I wanted to get this out first, because I'm sure there will be plenty of other issues.

[nbougalis]

Yuck. Consider using Boost.Describe at least.

[mvadari]

I can't figure out how to properly use Boost.Describe - I keep getting a ton of errors. The documentation is very unclear, since the API has changed a lot in the past few versions. Even ChatGPT/Bard isn't able to provide much help.

[nbougalis]

Not needed: you can directly compare an std::optional<T> with a T and it will do the right thing: return hash == defsHash; is enough.

Although, as I said before, it's unclear why defsHash is an std::optional or even a variable at all. You could just implement this as:

return to_string(hash) == defs[jss::value].asString()

[mvadari]

I removed the std::optional in 5fc2ee5, IMO it's a bit clearer and slightly more performant to have the separate variable.

[nbougalis]

Minor nit, but I don't know that you actually need an instance of a class for this: you may as well just have something like inline const Json::Value definitions = []() { ... }();

[mvadari]

IMO things like the hashMatches method are a bit neater with a class as opposed to something inline. If you think I should refactor to remove the class though, I can.

[sublimator]

Nice. Been wanting this for a while:
#1972

[ckeshava]

The PR looks nice. I've still left some suggestions, but they are my subjective opinions and shouldn't stop the PR from going into production

[mvadari]

@intelliot this is ready to merge 👍

[mvadari]

I don't have the powers to retry actions but it looks like something failed with the container setup in the dependency test.

[intelliot]

1. I clicked the button to retry the CI job
2. I've asked @nbougalis to confirm that he doesn't have any remaining blockers (i.e. comments/concerns that still must be addressed)

"If others have signed off, please use their sign off instead."

[intelliot]

@ckeshava has approved the latest code in this PR (aside from the merge commit)

[intelliot]

Suggested commit message:

feat(rpc): add `server_definitions` method (#4703)

Add a new RPC / WS call for `server_definitions`, which returns an
SDK-compatible `definitions.json` (binary enum definitions) generated by
the server. This enables clients/libraries to dynamically work with new
fields and features, such as ones that may become available on side
chains. Clients query `server_definitions` on a node from the network
they want to work with, and immediately know how to speak that node's
binary "language", even if new features are added to it in the future
(as long as there are no new serialized types that the software doesn't
know how to serialize/deserialize).

Example:

```js
> {"command": "server_definitions"}
< {
    "result": {
        "FIELDS": [
            [
                "Generic",
                {
                    "isSerialized": false,
                    "isSigningField": false,
                    "isVLEncoded": false,
                    "nth": 0,
                    "type": "Unknown"
                }
            ],
            [
                "Invalid",
                {
                    "isSerialized": false,
                    "isSigningField": false,
                    "isVLEncoded": false,
                    "nth": -1,
                    "type": "Unknown"
                }
            ],
            [
                "ObjectEndMarker",
                {
                    "isSerialized": false,
                    "isSigningField": true,
                    "isVLEncoded": false,
                    "nth": 1,
                    "type": "STObject"
                }
            ],
        ...
\```

Close #3657

---------

Co-authored-by: Richard Holland <richard.holland@starstone.co.nz>

[kennyzlei]

For Clio, we are considering adding this support in 2.2 release as it needs some time to investigate support for this API. This means for s1/s2, rippled 2.0 release would mean we would need to wait for Clio 2.2 to have this RPC accessible

XRPLF/clio#942

[mvadari]

For Clio, we are considering adding this support in 2.2 release as it needs some time to investigate support for this API. This means for s1/s2, rippled 2.0 release would mean we would need to wait for Clio 2.2 to have this RPC accessible

XRPLF/clio#942

@kennyzlei is it possible to just forward to rippled in 2.1? It should be pretty performant as-is and shouldn't need to be called too often.

[kennyzlei]

@mvadari I believe this will depend on the timing of our Clio releases. We are starting to release more frequently with 2.1 slotted to release in 2 weeks, so 2.2 should not be delayed by much. We need the investigation time to decide if Clio should forward these requests or do something, which may be tight to complete within 2 weeks

[SaxenaKaustubh]

Worked on this with @mvadari on this change. After a few iterations, we have this working the way we want. This concludes the manual testing.