Entry point

README.md

@@ -10,41 +10,133 @@
 256mb ram & 500mb hdd should be fairly enough to keep it up & running.
 <br />
 
-A lightweight bitcoin client, at this very moment it supports only bitcoin mainnet.
-Testnet support will come soon before beta release.<br /><br />
-So, run spruned on your low end system and you have the APIs listed below as you have a 200gigs Bitcoind installation.
-<br /><br />
+At this very moment it supports only bitcoin mainnet. Testnet support will come soon before beta release.<br /><br />
+It's a replacement for bitcoind on lightweight systems (It's proven to work on a Raspberry Zero, along with CLightning), it provides an interface for bitcoin-cli. <br />
+<br />
 
 #### How it works?
 
 spruned downloads and store the bitcoin blocks on demand, when you need them, directly from the Peer2Peer Bitcoin Network.<br/>
-There's a "pruning" functionality emulation, to keep the last ~200 (default settings) blocks already saved, because 
+there's a "bootstrap" functionality, to keep the last ~200 (default settings) blocks on the local storage, because 
 fetch blocks may require also up to 10 seconds with slow connections, and this "bootstrap mode" reduces latencies on usage.<br />
 
-You can use bitcoin-cli, or any other RPC client, as if you had bitcoind up & running.<br /><br />
+You can use bitcoin-cli, or any other RPC client, as if you had bitcoind up & running.<br />
 For the transactions related APIs and utxo tracking, spruned uses the electrum network.
 
 #### Documentation
 
-* [Installation and usage on Raspberry Pi B (1 Core - 512 MB)](docs/rpi-b-2012.md)
+* [Installation and usage on Raspberry Pi (Raspbian 9.3)](docs/rpi-b-2012.md)
 
 #### Dependencies
 
-spruned works with Python 3.5.2 and Python 3.6, as it uses asyncio. atm it should work only on Linux systems.<br />
+spruned works with Python >= 3.5.3. Right now it should work only on Linux systems.<br />
 <br />
 It make intensive usage of connectrum, pybitcointools and pycoinnet libraries. Thanks to mantainers & contributors! <br />
-Especially at this stage of development (but it would be better always), it is recommended to use virtualenv to run spruned
+Especially at this stage of development (but it would be better always), it is recommended to use virtualenv to run spruned.
+
+
+#### Usage
+Developers: I hope code is self explaining enough, if you're familiar with asyncio.<br />
+
+Everyone else: You can get inspiration on how to install spruned taking a look at setup.sh but, if you're lucky enough, setup.sh itself will 
+create a virtual environment and install spruned into it. 
+
+Well, try this:
+```
+$ cd ~/src
+$ sudo apt-get install livleveldb-dev python3-dev git virtualenv
+$ git clone https://github.com/gdassori/spruned.git
+$ cd spruned
+$ ./setup.sh
+$ venv/bin/python spruned.py --help
+
+$ venv/bin/python spruned.py --help
+
+usage: spruned.py [-h] [--rpcuser RPCUSER] [--rpcpassword RPCPASSWORD]
+                  [--rpcport RPCPORT] [--rpcbind RPCBIND] [--datadir DATADIR]
+                  [--daemon] [--keep-blocks KEEP_BLOCKS]
+                  [--network {bitcoin.mainnet}] [--debug]
+                  [--cache-size CACHE_SIZE]
+
+A Bitcoin Lightweight Pseudonode
+
+optional arguments:
+  -h, --help            show this help message and exit
+  --rpcuser RPCUSER     Username for JSON-RPC connections (default: rpcuser)
+  --rpcpassword RPCPASSWORD
+                        Password for JSON-RPC connections (default:
+                        rpcpassword)
+  --rpcport RPCPORT     Listen for JSON-RPC connections on <port> (default:
+                        8332 or testnet: 18332) (default: 8332)
+  --rpcbind RPCBIND     Bind to given address to listen for JSON-RPC
+                        connections. (default: 127.0.0.1)
+  --datadir DATADIR     Specify data directory (default: $HOME/.spruned)
+  --daemon              Run in the background as a daemon and accept commands
+                        (default: False)
+  --keep-blocks KEEP_BLOCKS
+  --network {bitcoin.mainnet}
+  --debug               Enable debug mode (default: False)
+  --cache-size CACHE_SIZE
+                        Cache size (in megabytes) (default: False)
+
+```
 
-#### Usage.
-Code should be pretty self explaining if you're familiar with asyncio.<br />
-**For the non-developers:** a fungible entry point is **not ready** yet.<br />
+And, once you run spruned:
+
+```
+$ tail -f ~/.spruned/spruned.log # to see what's going on!
+```
+
+or check the status*:
 
-However, this is how things are going to be:
 ```
-$ spruned --daemon
 $ bitcoin-cli getblockchaininfo
+{
+  "mediantime": 1523387051,
+  "blocks": 517579,
+  "headers": 517579,
+  "verificationprogress": 100,
+  "chain": "main",
+  "chainwork": null,
+  "difficulty": null,
+  "bestblockhash": "00000000000000000018e502dec1f93d32521674019a45d7d095cbd390279dff",
+  "warning": "spruned v0.0.1. emulating bitcoind v0.16",
+  "pruned": false
+}
+```
+
+Download a block:
+```
+$ bitcoin-cli getblock `bitcoin-cli getblockhash 1`
+{
+  "bits": 486604799,
+  "mediantime": 1231469665,
+  "nextblockhash": "000000006a625f06636b8bb6ac7b960a8d03705d1ace08b1a19da3fdcc99ddbd",
+  "tx": [
+    "0e3e2357e806b6cdb1f70b54c3a3a17b6714ee1f0e68bebb44a74b1efd512098"
+  ],
+  "previousblockhash": "000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f",
+  "version": 1,
+  "chainwork": "Not Implemented Yet",
+  "nonce": 2573394689,
+  "time": 1231469665,
+  "height": 1,
+  "hash": "00000000839a8e6886ab5951d76f411475428afc90947ee320161bbf18eb6048",
+  "versionHex": "Not Implemented Yet",
+  "merkleroot": "0e3e2357e806b6cdb1f70b54c3a3a17b6714ee1f0e68bebb44a74b1efd512098",
+  "difficulty": "Not Implemented Yet"
+}
+```
+
+Or a transaction:
 ```
-Pretty easy.
+$ bitcoin-cli getrawtransaction 0e3e2357e806b6cdb1f70b54c3a3a17b6714ee1f0e68bebb44a74b1efd512098
+01000000010000000000000000000000000000000000000000000000000000000000000000ffffffff0704ffff
+001d0104ffffffff0100f2052a0100000043410496b538e853519c726a2c91e61ec11600ae1390813a627c66fb
+8be7947be63c52da7589379515d4e0a604f8141781e62294721166bf621e73a82cbf2342c858eeac00000000
+
+```
+######* bitcoin-cli is not included
 <br /><br />
 
 ##### Emulated APIs as in bitcoind 0.16:
@@ -79,15 +171,22 @@ Pretty easy.
 
 #### Limitations
 
-- May reduce privacy: if you have the entire blockchain you don't have to tell no one what you're going to search.
+- May reduce privacy: if you have the entire blockchain on your own, you have to tell no one what you're looking for.
 - Not fast as a full node: internet download is slower than a read from disk.
 - Doesn't relay and partecipate to the network (this may change).
+- Very unstable!
 
 
 #### Future development
 
-- Pluggable currencies specs
 - Full Tor support
 - Mempool emulation
 - Zeromq emulation
 - Maintenance UI
+
+
+##### Have fun with sPRUNED
+
+Spruned is **plenty** of bugs, because I care of you and I don't what you to get bored.
+To reduce the entropy and help to put some interim order into the universe, you can have fun
+with spruned by fixing bugs and enhancing it with new features. You can bet this will help.

requirements.txt

@@ -5,6 +5,7 @@ aiohttp==3.1.1
 jsonrpcserver==3.5.3
 sqlalchemy==1.2.6
 coverage==4.5.1
--e git+https://github.com/gdassori/pycoinnet.git@spruned-support#egg=pycoinnet
+-e git+https://github.com/gdassori/pycoinnet.git@spruned-support#egg=pycoinnet-0.19-spruned-02
 -e git+https://github.com/gdassori/pycoin.git@spruned-support#egg=pycoin-0.90a
-plyvel==0.9.0
+plyvel==0.9.0
+daemonize==2.4.7

spruned.py

@@ -1,43 +1,88 @@
-import asyncio
-
-from spruned.application import tools
-tools.load_config()
-
-from spruned.application.tools import async_delayed_task
-from spruned.application.logging_factory import Logger
-from spruned.builder import blocks_reactor, headers_reactor, jsonrpc_server, repository, cache
-
-
-async def main_task(loop):
-    try:
-        Logger.leveldb.info('Ensuring integrity of the storage, and tracking missing items')
-        try:
-            await loop_check_integrity(loop)
-        except asyncio.TimeoutError:
-            Logger.cache.error('There must be an error in storage, 30 seconds to check are too many')
-        Logger.leveldb.info('Checking cache limits')
-        try:
-            asyncio.wait_for(asyncio.gather(cache.check()), timeout=30)
-        except asyncio.TimeoutError:
-            Logger.cache.error('There must be an error in cache, 30 seconds to check are too many')
-        headers_reactor.add_on_best_height_hit_callbacks(blocks_reactor.start())
-        headers_reactor.add_on_best_height_hit_callbacks(blocks_reactor.bootstrap_blocks())
-        loop.create_task(headers_reactor.start())
-        loop.create_task(jsonrpc_server.start())
-        loop.create_task(cache.lurk())
-    finally:
-        pass
+#!/usr/bin/env python3
+# Copyright (C) 2018 Guido Dassori <guido.dassori@gmail.com>
+#
 
+import argparse
+import asyncio
+from daemonize import Daemonize
+from spruned.application.context import ctx
 
-async def loop_check_integrity(l):
-    """
-    this task also prune blocks
-    """
-    await repository.ensure_integrity()
-    l.create_task(async_delayed_task(loop_check_integrity(l), 3600))
+parser = argparse.ArgumentParser(
+    description="A Bitcoin Lightweight Pseudonode",
+    formatter_class=argparse.ArgumentDefaultsHelpFormatter
+)
+parser.add_argument(
+    '--rpcuser',
+    action='store', dest='rpcuser', default=ctx.rpcuser,
+    help='Username for JSON-RPC connections'
+)
+parser.add_argument(
+    '--rpcpassword',
+    action='store', dest='rpcpassword', default=ctx.rpcpassword,
+    help='Password for JSON-RPC connections'
+)
+parser.add_argument(
+    '--rpcport',
+    action='store', dest='rpcport', default=ctx.rpcport,
+    help='Listen for JSON-RPC connections on <port> (default: 8332 or testnet: 18332)'
+)
+parser.add_argument(
+    '--rpcbind',
+    action='store', dest='rpcbind', default=ctx.rpcbind,
+    help='Bind to given address to listen for JSON-RPC connections.'
+)
+parser.add_argument(
+    '--datadir',
+    action='store', dest='datadir', default=ctx.datadir,
+    help='Specify data directory'
+)
+parser.add_argument(
+    '--daemon',
+    action='store_true', dest='daemonize', default=bool(ctx.daemonize),
+    help='Run in the background as a daemon and accept commands'
+)
+parser.add_argument(
+    '--keep-blocks',
+    action='store', dest='keep_blocks', default=int(ctx.keep_blocks),
+    help='', type=int
+)
+parser.add_argument(
+    '--network',
+    action='store', dest='network', default=ctx.network,
+    choices=[
+        'bitcoin.mainnet',
+        #'bitcoin.testnet',
+        #'bitcoin.regtest'
+    ],
+    help=''
+)
+parser.add_argument(
+    '--debug',
+    action='store_true', dest='debug', default=ctx.debug,
+    help='Enable debug mode'
+)
+parser.add_argument(
+    '--cache-size',
+    action='store', dest='cache_size', default=int(ctx.cache_size),
+    help='Cache size (in megabytes)'
+)
 
 
-if __name__ == '__main__':  # pragma: no cover
+def start():
+    ctx.load_args(args)
+    create_directory()
     main_loop = asyncio.get_event_loop()
+    from spruned.main import main_task
     main_loop.create_task(main_task(main_loop))
     main_loop.run_forever()
+
+
+if __name__ == '__main__':  # pragma: no cover
+    args = parser.parse_args()
+    from spruned.application.tools import create_directory
+    if args.daemonize:
+        pid = ctx.datadir + 'spruned.pid'
+        daemon = Daemonize(app="spruned", pid=pid, action=start)
+        daemon.start()
+    else:
+        start()

spruned/application/cache.py

@@ -120,7 +120,7 @@ async def lurk(self):
             await self.check()
         finally:
             self.lock.release()
-            self.loop.create_task(self.delayer(self.lurk(), 30))
+            self.loop.create_task(self.delayer(self.lurk(), 600))
 
     def get_index(self):
         if not self.index: