Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support external RPC servers via IPC #1434

Merged
merged 15 commits into from Jan 28, 2019

Conversation

@cryptocode
Copy link
Collaborator

commented Dec 8, 2018

Remaining

  • Add tests (write an IPC client class for this)
  • Document the framing format in the code
  • Add ipc logging option with upgrade path
  • Config upgrade path (merge #1416 first)
    After reviewing/testing, will also add this:
  • Replace boost::bind with lambdas
  • Utilize the new nano::timer class
  • Introduce session id for improved logging

This adds support for external RPC servers that communicate with the node via IPC (currently domain sockets and tcp).

Using external web servers can benefit service scalability and security. IPC connections are persistent, which should reduce time_wait (file descriptor exhaustion) issues on busy servers.

The internal RPC server can thus be disabled in the config, yet have the functionality exposed via IPC. The JSON format is exactly the same as the internal server - clients only need to change URL.

A proof-of-concept REST server in Go is available at https://github.com/nanocurrency/rpc-go.

The IPC gateway allows for other message payloads like protobuf (there's a PR for protobuf which would need some work to work with this PR)

Technically, the IPC gateway defines a simple framing format with an encoding flag. In this case, the encoding is 1 (json rpc) which expects a payload of 32-bit big-endian size + json, which is also the case for the response. The json payload is forwarded to the node's existing RPC action handler. More info will be added to the wiki.

Configuration

To test the rpc-go server, enable either tcp or local ipc in the node config. Do the corresponding change in rpc-go's config.json (which uses domain sockets by default)

The rpc_enable flag can still be active, to access the current RPC server.

@cryptocode cryptocode force-pushed the cryptocode:feature/rpc-externalization branch from 93e5ca9 to b7c4cca Dec 8, 2018

@cryptocode cryptocode force-pushed the cryptocode:feature/rpc-externalization branch 2 times, most recently from e894a49 to d697d70 Dec 9, 2018

@cryptocode

This comment has been minimized.

Copy link
Collaborator Author

commented Dec 28, 2018

@meltingice is working on an IPC lib and gRPC interface based on this PR

https://github.com/meltingice/nano-ipc-js
https://github.com/meltingice/nano-grpc

@rkeene rkeene force-pushed the cryptocode:feature/rpc-externalization branch from d697d70 to 151f565 Dec 28, 2018

Show resolved Hide resolved nano/nano_node/daemon.cpp Outdated
@meltingice

This comment has been minimized.

Copy link

commented Dec 29, 2018

FYI if anyone else runs into this issue... after updating to the latest commit on this PR, the node could no longer deserialize the config. It was missing the node.logging.timing config option, which didn't get added by a config upgrade. Works fine once added. Default is false.

@cryptocode

This comment has been minimized.

Copy link
Collaborator Author

commented Dec 29, 2018

Yeah, the timing upgrade was recently added to master

@meltingice

This comment has been minimized.

Copy link

commented Dec 29, 2018

I am getting an exception that's crashing the node when attempting to call account_representative_set via IPC on an unopened account. The crash does not happen when attempting the same call via the HTTP RPC.

The call I'm making is:

{
  "action": "account_representative_set",
  "wallet": "523325FC5B61CB450035B32AAE11FE7F02BEB5D3F736EF59FC3C7B0FAD35C46C",
  "account": "xrb_118fd6yj4mzc17k3q9ugygdqwj8cxqia8hgrxijb1efahsmnwzk8qs3foxnt",
  "representative": "xrb_1x7biz69cem95oo7gxkrw6kzhfywq4x5dupw4z1bdzkb74dk9kpxwzjbdhhs"
}

And the crash message is:

libc++abi.dylib: terminating with uncaught exception of type std::__1::bad_weak_ptr: bad_weak_ptr
[1]    11184 abort      ./nano_node --daemon --data_path=/Users/ryanlefevre/RaiBlocks/

I'm running this node on macOS 10.14.2.

@cryptocode

This comment has been minimized.

Copy link
Collaborator Author

commented Dec 29, 2018

@meltingice thanks for the report. This seems to be related to action handlers with asynchronous responses (wallet->change_async in this case). I'll look into it.

@meltingice

This comment has been minimized.

Copy link

commented Dec 29, 2018

Just tested and appears to be fixed now. Thanks @cryptocode!

@zhyatt zhyatt added this to the V18.0 milestone Dec 31, 2018

@zhyatt zhyatt added this to Unscheduled in V18 Dec 31, 2018

@zhyatt zhyatt moved this from Unscheduled to CP 1 (2018-01-09) in V18 Dec 31, 2018

@cryptocode cryptocode moved this from CP 1 (2018-01-09) to CP 3 (2018-01-23) in V18 Jan 4, 2019

@zhyatt zhyatt requested review from wezrule and clemahieu Jan 7, 2019

@cryptocode cryptocode force-pushed the cryptocode:feature/rpc-externalization branch from e225eac to 65d55a3 Jan 10, 2019

@cryptocode cryptocode moved this from CP 3 (2018-01-23) to CP 2 (2018-01-16) in V18 Jan 10, 2019

@cryptocode cryptocode force-pushed the cryptocode:feature/rpc-externalization branch from 5ea4099 to 7bd3404 Jan 17, 2019

@cryptocode cryptocode force-pushed the cryptocode:feature/rpc-externalization branch from 7bd3404 to bf83204 Jan 17, 2019

@clemahieu
Copy link
Collaborator

left a comment

Looks great!

@cryptocode cryptocode force-pushed the cryptocode:feature/rpc-externalization branch from 9b68c4d to 13659df Jan 23, 2019

Show resolved Hide resolved nano/node/nodeconfig.hpp Outdated
Show resolved Hide resolved nano/node/ipc.cpp Outdated
Show resolved Hide resolved nano/node/ipc.cpp Outdated
Show resolved Hide resolved nano/node/ipc.cpp Outdated
Show resolved Hide resolved nano/node/ipc.cpp Outdated
Show resolved Hide resolved nano/node/ipc.hpp Outdated
Show resolved Hide resolved nano/node/ipc.hpp Outdated
Show resolved Hide resolved nano/node/ipc.cpp Outdated
Show resolved Hide resolved nano/node/ipc.hpp
Show resolved Hide resolved nano/node/ipc.cpp Outdated

cryptocode added some commits Jan 24, 2019

@rkeene

rkeene approved these changes Jan 28, 2019

@cryptocode cryptocode merged commit f492cfc into nanocurrency:master Jan 28, 2019

2 checks passed

continuous-integration/appveyor/pr AppVeyor build succeeded
Details
continuous-integration/travis-ci/pr The Travis CI build passed
Details
@meltingice

This comment has been minimized.

Copy link

commented Jan 28, 2019

The JS client library has been updated to match the preamble changes. https://www.npmjs.com/package/nano-ipc

@zhyatt zhyatt referenced this pull request Apr 4, 2019

Open

JSON-RPC #589

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.