pruneblockchain (RPC): Supported ?

[iangregsondev]

Hi,

I was hoping to use a pruneblockchain rpc command ( https://bitcoincore.org/en/doc/0.21.0/rpc/blockchain/pruneblockchain/ )

I was wondering if electrs supports this ?

This is pruning using the RPC command and not adding anything into bitcoin.conf, so I would need a full synchronised blockchain and then after that I could issue the pruneblockchain - passing in a height I wish to prune up to.

As the product I am developing is only valid from a certain block height then I think this would work for me BUT I am unsure if this is going to cause problems with electrs, will electrs detected that its pruned (no setting in bitcoin.conf) - is this something I can get away with :-) ?

I know that electrs doesn't work with pruned node but is this classed the same as having the settings set in the bitcoin.conf.

Any ideas?

Thanks in advance

[romanz]

As of today, electrs doesn't support pruned nodes.

electrs/src/daemon.rs

Line 343 in 8fb4d7c

bail!("pruned node is not supported (use '-prune=0' bitcoind flag)".to_owned())

[Kixunil]

@iangregsondev take a look at Bitcoin RPC Proxy, it can fake non-pruned node by fetching the missing blocks from peers. If you put it between bitcoind and electrs, it should work. Disclaimer: I didn't test it yet.

[gcomte]

@Kixunil's link doesn't work. Looks like he want's to point us to Bitcoin RPC Proxy

[Kixunil]

@gcomte sorry and thanks for heads up! I didn't prefix it with https:// so GitHub thought it's relative

[ejose19]

Hello @Kixunil, I'm interested in running electrs along bitcoin-rpc-proxy, but so far haven't got it working. First I had to comment:

electrs/src/daemon.rs

Lines 140 to 143 in a552a2b

	let blockchain_info: BlockchainInfo = rpc.call("getblockchaininfo", &[])?;
	if blockchain_info.pruned {
	bail!("electrs requires non-pruned bitcoind node");
	}

in order to ignore pruned error, however when starting I receive this:

Config { network: Bitcoin, db_path: "/home/user/electrs/data/bitcoin", daemon_dir: "/home/user/.bitcoin", daemon_auth: UserPass("public", "<sensitive>"), daemon_rpc_addr: V4(127.0.0.1:10000), daemon_p2p_addr: V4(127.0.0.1:8333), electrum_rpc_addr: V4(127.0.0.1:50001), monitoring_addr: V4(127.0.0.1:4224), wait_duration: 10s, jsonrpc_timeout: 15s, index_batch_size: 10, index_lookup_limit: Some(200), auto_reindex: true, ignore_mempool: false, sync_once: false, disable_electrum_rpc: false, server_banner: "Welcome to electrs 0.9.0 (Electrum Rust Server)!", args: [] }
[2021-10-10T22:41:10.298Z INFO  electrs::metrics::metrics_impl] serving Prometheus metrics on 127.0.0.1:4224
[2021-10-10T22:41:10.298Z DEBUG tiny_http] Server listening on 127.0.0.1:4224
[2021-10-10T22:41:10.298Z DEBUG tiny_http] Running accept thread
[2021-10-10T22:41:10.325Z INFO  electrs::db] "/home/user/electrs/data/bitcoin": 11 SST files, 0.000008855 GB, 0.000000011 Grows
[2021-10-10T22:41:10.325Z DEBUG electrs::db] DB Some(Config { compacted: false, format: 0 })
[2021-10-10T22:41:10.329Z DEBUG bitcoincore_rpc::client] JSON-RPC request: getblockchaininfo []
[2021-10-10T22:41:10.331Z DEBUG bitcoincore_rpc::client] JSON-RPC request: getnetworkinfo []
[2021-10-10T22:41:10.331Z DEBUG bitcoincore_rpc::client] JSON-RPC request: getblockchaininfo []
[2021-10-10T22:41:10.332Z DEBUG electrs::p2p] peer version: VersionMessage { version: 70016, services: ServiceFlags(1032), timestamp: 1633905670, receiver: Address {services: ServiceFlags(NONE), address: 0.0.0.0, port: 0}, sender: Address {services: ServiceFlags(WITNESS|NETWORK_LIMITED), address: 0.0.0.0, port: 0}, nonce: 5370067419933503180, user_agent: "/Satoshi:0.21.1/", start_height: 704439, relay: false }
[2021-10-10T22:41:10.337Z DEBUG electrs::p2p] got 2000 new headers
[2021-10-10T22:41:10.340Z INFO  electrs::index] indexing 2000 blocks: [1..2000]
[2021-10-10T22:41:10.340Z DEBUG electrs::p2p] loading 10 blocks
[2021-10-10T22:41:10.391Z DEBUG electrs::p2p] closing p2p_recv thread: connection closed
[2021-10-10T22:41:10.391Z DEBUG electrs::p2p] closing p2p_loop thread: peer has disconnected
[2021-10-10T22:41:10.391Z DEBUG electrs::p2p] closing p2p_send thread: no more messages to send
[2021-10-10T22:41:10.391Z INFO  electrs::db] closing DB at /home/user/electrs/data/bitcoin
Error: electrs failed

Caused by:
    0: initial sync failed
    1: failed to get block 00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048
    2: receiving on an empty and disconnected channel

for which I assume the method no longer works with 0.9.0 due the p2p requirements (was the previous jsonrpc_import completely removed from codebase or it can be re-enabled through an option? @romanz)

So I downgraded to 0.8.x branch but after starting electrs it just hangs waiting for a reply from bitcoin-rpc-proxy, there's nothing logged on neither electrs nor bitcoin-rpc-proxy. After some manual debugging I found that it hangs here:

electrs/src/daemon.rs

Lines 217 to 232 in 1997ddc

	for line in iter {
	let line = line.chain_err(|| ErrorKind::Connection("failed to read".to_owned()))?;
	if line.is_empty() {
	in_header = false; // next line should contain the actual response.
	} else if in_header {
	let parts: Vec<&str> = line.splitn(2, ": ").collect();
	if parts.len() == 2 {
	headers.insert(parts[0].to_owned(), parts[1].to_owned());
	} else {
	warn!("invalid header: {:?}", line);
	}
	} else {
	contents = Some(line);
	break;
	}
	}

If I place debug!("line: {:?}", line); right at the start of the loop, I can see this:

DEBUG - line: Ok("content-length: 685")
DEBUG - line: Ok("date: Sun, 10 Oct 2021 23:47:19 GMT")
DEBUG - line: Ok("")

and it's not until I SIGTERM the bitcoin-rpc-process that I receive the response that allows the iteration to finish:

DEBUG - line: Ok("[{\"id\":1,\"error\":null,\"result\":{\"connections\":10,\"connections_in\":0,\"connections_out\":10,\"incrementalfee\":0.00001,\"localaddresses\":[],\"localrelay\":false,\"localservices\":\"0000000000000408\",\"localservicesnames\":[\"WITNESS\",\"NETWORK_LIMITED\"],\"networkactive\":true,\"networks\":[{\"limited\":false,\"name\":\"ipv4\",\"proxy\":\"\",\"proxy_randomize_credentials\":false,\"reachable\":true},{\"limited\":false,\"name\":\"ipv6\",\"proxy\":\"\",\"proxy_randomize_credentials\":false,\"reachable\":true},{\"limited\":true,\"name\":\"onion\",\"proxy\":\"\",\"proxy_randomize_credentials\":false,\"reachable\":false}],\"protocolversion\":70016,\"relayfee\":0.00001,\"subversion\":\"/Satoshi:0.21.1/\",\"timeoffset\":0,\"version\":210100,\"warnings\":\"\"}}]")

so I'm not sure if the problem is on electrs side or bitcoin-rpc-process side.

[romanz]

was the previous jsonrpc_import completely removed from codebase or it can be re-enabled through an option?

Yes, it was removed since it was inefficient and caused OOMs in bitcoind: #373

[romanz]

I think that a possible solution is to allow electrs to fetch blocks from other peers - in case the local peer is a pruned node.

[Kixunil]

Yeah, fetching from other peers should be fairly simple but we have to keep in mind to not trust them and verify hashes. btc-rpc-proxy has P2P code too, so it could also proxy P2P connections.