[RPC] Just an attemt to save some bandwidth

[CryptoManiac]

Simple gzip compression can be very useful when you're dealing with a lot of getblocktemplate or getrawmempool requests. It decreases GBT latency and allows you to save a lot of bandwidth.

Just for example, I have few instances of bitcoind which are being used as template sources for eloipool. Each getblocktemplate response has a size of ~2MB. Being gzipped it shrinks down to ~800KB, which is a fairly reasonable size. :)

[jameshilliard]

@CryptoManiac why not just put a reverse proxy like nginx in front of the RPC interface?

[luke-jr]

You really shouldn't use RPC over a network...

[CryptoManiac]

@jameshilliard Well, this also may be a viable solution. Unfortunately, some people are limited to Windows operating system. To handle connections, the NT version of nginx is currently using only select(), so it's very slow. :(

[CryptoManiac]

@luke-jr Why not? If you have a walletless daemon, which listening at the private interface, then I'd say that nothing bad could happen.

However, I understand the main idea. Closing for now.

[laanwj]

Just for example, I have few instances of bitcoind which are being used as template sources for eloipool. Each getblocktemplate response has a size of ~2MB. Being gzipped it shrinks down to ~800KB, which is a fairly reasonable size. :)

This does increase the latency by bit, though, so on faster networks wouldn't want to use this.

In any case I agree that this belongs externally. RPC is in the first place for local communication, for doing anything like compression, encryption put it behind a tunnel or nginx instance.

[CryptoManiac]

@laanwj It depends on your TCP/IP stack settings. In FreeBSD, for example, sometimes it's recommended to set net.inet.tcp.recvspace parameter to 8k. With such receive buffer size you can get a significant improvement even for a local instance of bitcoind, speed up looks like a linear function of compression ratio.

[jonasschnelli]

If you want compress data, why not reverse-proxy over apache and enable mod_deflate?

[CryptoManiac]

@jonasschnelli Because sometimes it doesn't help. Nginx needs to get the data from the upstream, so it can be affected by tcp/ip settings. Unfortunately, bitcoin doesn't support RPC over unix domain sockets for now.

[laanwj]

Unfortunately, bitcoin doesn't support RPC over unix domain sockets for now.

Now that would be a welcome addition. But apparently it needs changes to evhttp. There's a similar ticket open for Transmission, which also uses evhttp.

[laanwj]

I just realized that it can be implemented with the current evhttp just fine - do the listening manually, set the socket to non-blocking, then pass the acceptfd to evhttp_accept_socket_with_handle, similarly as I did here for cloudabi: https://github.com/laanwj/bitcoin/blob/2017_03_cabi_fs/src/httpserver.cpp#L360.

[laanwj]

@CryptoManiac See #9919

[CryptoManiac]

@laanwj Nice one, thanks.