p2p: Bare minimum to support UNIX sockets #9979

laanwj · 2017-03-12T12:42:02Z

Two commit on top of #9919 (first adds the implementation, second adds a test).
Make it possible to listen and connect to UNIX sockets for P2P. There are two main use cases for this:

Testing without possibility of port collisions
TOR hidden service listening with less risk of privacy leaks by not opening any internet ports

Usage: specify :unix:<path> for bind, whitebind, connect, addnode.

Add functionality for RPC over UNIX sockets, on platforms that support UNIX sockets. This is specified with `:unix:/path/to/socket` to `-rpcbind` and `-rpcconnect`. By default the socket path is relative to the data directory.

evhttp has no direct support for UNIX sockets thus is not able to recognize UNIX peers. Add custom code in HTTPRequest::GetPeer and evunix to recognize these connections as coming from localhost. The previous solution did not work on some libevent versions.

Make it possible to listen and connect to UNIX sockets for P2P. There are two main use cases for this: - Testing without possibility of port collisions - TOR hidden service listening with less risk of privacy leaks by not opening any internet ports Usage: specify `:unix:<path>` for `bind`, `whitebind`, `connect`, `addnode`.

JeremyRubin

Not sure I'm familiar enough with libevent to give more than a concept ack. I read through it & it looks great. Concept Ack.

JeremyRubin · 2017-04-05T14:47:56Z

src/httpserver.cpp

@@ -331,6 +337,8 @@ static bool HTTPBindAddresses(struct evhttp* http)
    } else if (mapMultiArgs.count("-rpcbind")) { // Specific bind address
        const std::vector<std::string>& vbind = mapMultiArgs.at("-rpcbind");
        for (std::vector<std::string>::const_iterator i = vbind.begin(); i != vbind.end(); ++i) {
+            if (boost::starts_with(*i, RPC_ADDR_PREFIX_UNIX)) // Skip UNIX sockets here


an easy way to replicate this without a new boost dep is:

if (!i->rfind(RPC_ADD_PREFIX_UNIX, 0))

I like boost::starts_with. It's python idom (and many other languages) so easy to read. If we want to do without that, I think we should simply implement our own in terms of what you write.

JeremyRubin · 2017-04-05T14:53:38Z

src/bitcoin-cli.cpp

-    raii_evhttp_connection evcon = obtain_evhttp_connection_base(base.get(), host, port);
+    raii_evhttp_connection evcon;
+    if (boost::starts_with(host, RPC_ADDR_PREFIX_UNIX)) {
+#if defined(HAVE_SOCKADDR_UN) && defined(LIBEVENT_EXPERIMENTAL)


Might be better to have the ifdef define two different functions HandleUnixSocket.

I'm always nervous with compiler directives inside functions, unless the code switched is ~1 line. Large ifdefs are in a couple places in this PR.

Indeed, factoring this out to a function makes sense

JeremyRubin · 2017-04-05T15:18:41Z

src/httpserver.cpp

@@ -505,6 +547,12 @@ void StopHTTPServer()
        event_base_free(eventBase);
        eventBase = 0;
    }
+#ifdef HAVE_SOCKADDR_UN
+    //! Clean up UNIX sockets on shutdown
+    for (const auto& path: cleanupUNIXSockets) {


slight preference for something with ownership guarantees of

std::vector<std::unique_ptr<void>> cleanupUNIXSockets; // must be non null value cleanupUNIXSockets.emplace_back(1, [=](void*) { evunix_remove_socket(path); }); cleanupUNIXSockets.clear();

so that sockets aren't double removed.

Maybe wrap the evunix stuff in a class and just keep a vector of those around?

Ok, makes sense

JeremyRubin · 2017-04-05T15:31:02Z

src/httpserver.cpp

@@ -619,6 +667,12 @@ CService HTTPRequest::GetPeer()
        const char* address = "";
        uint16_t port = 0;
        evhttp_connection_get_peer(con, (char**)&address, &port);
+        if (!strcmp(address, "localhost")) {
+            /* Special: will get here for UNIX sockets. As we have no way to indicate that,


recommend comment language (took me a minute to understand your comment because of ambiguity on here and that):

We will get an address of localhost either via a unix socket or via a localhost peer. There is no sane way to return an address for a socket, so we always return localhost's IPv6.

ryanofsky

utACK 9abf3c9, though many files are out of date.

I'm wondering why this is labeled "bare minimum to support UNIX sockets" and when support seems pretty complete. Are there other p2p features over unix sockets that you'd want to support?

ryanofsky · 2017-05-08T18:03:39Z

src/net.cpp

+    // Listen for incoming connections
+    if (listen(hListenSocket, SOMAXCONN) == SOCKET_ERROR)
+    {
+        strError = strprintf(_("Error: Listening for incoming connections failed (listen returned error %s)"), NetworkErrorString(WSAGetLastError()));


In commit "p2p: Bare minimum to support UNIX sockets"

Any reason this error is translated while others are not?

ryanofsky · 2017-05-08T18:04:29Z

src/net.cpp

+        return false;
+    }
+    SOCKET hListenSocket = (SOCKET)fd;
+    LogPrintf("P2P bound to %s\n", path.string());


In commit "p2p: Bare minimum to support UNIX sockets"

Maybe mention path is unix socket path

laanwj · 2017-06-01T14:03:34Z

Thanks for reviewing @JeremyRubin @jnewbery

though many files are out of date.

I somewhat dread having to rebase this, but hope to do so in the near future.

I'm wondering why this is labeled "bare minimum to support UNIX sockets" and when support seems pretty complete. Are there other p2p features over unix sockets that you'd want to support?

There's no support in RPC calls yet, for example. Unix sockets can only be passed on the command line.

jnewbery · 2017-06-01T14:29:36Z

I somewhat dread having to rebase this, but hope to do so in the near future.

Yes - rebase looks like it's going to be a big job :(

Let me know if I can help in any way - I'm definitely interested in getting Unix socket support for RPC.

ryanofsky · 2017-06-01T20:49:57Z

I somewhat dread having to rebase this, but hope to do so in the near future.

I didn't try rebasing, but the merge with master doesn't look bad at all. The only C++ code conflicts are mapMultiArgs -> gArgs replacements. There is a huge scary looking conflict in the test framework util.py, but it just comes from the initialize_chain function moving to another file, and the changes there are minor.

laanwj · 2017-06-24T11:31:40Z

Closing this for now. I'll probably pick it up again for 0.16.

laanwj added the P2P label Mar 12, 2017

laanwj added 6 commits March 12, 2017 13:59

rpc: UNIX sockets support

45fad46

Add functionality for RPC over UNIX sockets, on platforms that support UNIX sockets. This is specified with `:unix:/path/to/socket` to `-rpcbind` and `-rpcconnect`. By default the socket path is relative to the data directory.

tests: Run RPC tests optionally over UNIX socket

6218b11

tests: Add environment option to force RPC over TCP for tests

f5acc05

p2p: Add test for P2P UNIX socket connection

9abf3c9

laanwj force-pushed the 2017_03_unix_socket_p2p branch from cd59f8e to 9abf3c9 Compare March 12, 2017 15:32

laanwj mentioned this pull request Mar 18, 2017

Unix Domain Sockets #5029

Open

JeremyRubin reviewed Apr 5, 2017

View reviewed changes

JeremyRubin mentioned this pull request Apr 5, 2017

UNIX sockets support for RPC #9919

Closed

ryanofsky reviewed May 8, 2017

View reviewed changes

laanwj closed this Jun 24, 2017

icota mentioned this pull request Jun 18, 2020

RFC: Introducing AltNet, a pluggable framework for alternative transports #18988

Closed

jamesob mentioned this pull request Nov 8, 2021

make Python bindings externally usable #23468

Closed

bitcoin locked as resolved and limited conversation to collaborators Feb 15, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

p2p: Bare minimum to support UNIX sockets #9979

p2p: Bare minimum to support UNIX sockets #9979

laanwj commented Mar 12, 2017 •

edited

Loading

JeremyRubin left a comment

JeremyRubin Apr 5, 2017

laanwj Apr 5, 2017 •

edited

Loading

JeremyRubin Apr 5, 2017

laanwj Apr 5, 2017

JeremyRubin Apr 5, 2017

laanwj Apr 5, 2017

JeremyRubin Apr 5, 2017

ryanofsky left a comment

ryanofsky May 8, 2017

ryanofsky May 8, 2017

laanwj commented Jun 1, 2017 •

edited

Loading

jnewbery commented Jun 1, 2017

ryanofsky commented Jun 1, 2017

laanwj commented Jun 24, 2017

p2p: Bare minimum to support UNIX sockets #9979

p2p: Bare minimum to support UNIX sockets #9979

Conversation

laanwj commented Mar 12, 2017 • edited Loading

JeremyRubin left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

laanwj Apr 5, 2017 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

ryanofsky left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

laanwj commented Jun 1, 2017 • edited Loading

jnewbery commented Jun 1, 2017

ryanofsky commented Jun 1, 2017

laanwj commented Jun 24, 2017

laanwj commented Mar 12, 2017 •

edited

Loading

laanwj Apr 5, 2017 •

edited

Loading

laanwj commented Jun 1, 2017 •

edited

Loading