Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: one-click installer - cli edition #3534

Merged
merged 6 commits into from
Nov 11, 2021
Merged

feat: one-click installer - cli edition #3534

merged 6 commits into from
Nov 11, 2021

Conversation

CjS77
Copy link
Collaborator

@CjS77 CjS77 commented Nov 3, 2021

Tari Launchpad - CLI edition

Code changes

  • Updated configuration for gRPC endpoints to accept MultAddr format in addition to SocketAddr. Changes should be 100% backwards compatible.

a.k.a. Tari one-click miner.

Currently this only works on MacOS and Linux.
Contributions for quick_start.bat are welcomed.

Prerequisites

  1. Docker
  2. Set up some environment variables:
  • DATA_FOLDER (required). The path to store the Tari configuration files and logs. You can create multiple
    network setups, each in their own sandbox.

  • TARI_NETWORK (optional). Default=weatherwax. Specify the Tari network to connect to.

  • START_TOR (optional). Default=1. Whether we should start a Tor instance

  • START_BASE_NODE (optional). Default=1. Whether we should start a base node instance

  • START_WALLET (optional). Default=1. Whether we should start a wallet instance (GRPC only)

  • START_MINER (optional). Default=1. Whether we should fire up a SHA_3 solo miner.

  • USE_OWN_MODEROD (optional). Default=0. Whether we should start and sync our own Monero node.

  • START_MONERO_MM (optional). Default=1. Whether we should start merge-mining on Monero.

  • TARI_WALLET_PASSWORD (optional). Highly recommended. Default: tari. The password for your Tari wallet.

  • TARI_WALLET__WEATHERWAX__TOR_CONTROL_AUTH (optional). Default: tari. The password for the Tor control auth.

  • TARI_MONERO_WALLET_ADDRESS (optional). Though you'll donate funds to some random address if you don't set it.

Quick Start

./quick-start.sh

And you should be golden.

Interacting with the wallet and base node.

Any Tari client that supports gRPC can be connected to the node setup. With the default configuration, the base node
will be at http://127.0.0.1:18142 and the wallet will be at http://127.0.0.1:18143.

Kreya is a fairly user-friendly gRPC client that can give you a rudimentary UI to the docker rig
out of the box:

Getting block information

node_blocks

Wallet - whoami

wallet_id

Wallet - send transactions

wallet_send

Viewing logs and configuration files

Logs and configuration files are stored in {DATA_FOLDER}/{app}/log/*.log.

So assuming your data folder is `/tmp/tari1/, the generated file structure looks like:

/tmp/tari1$ ll -R
drwxr-xr-x user user 128 B Wed Nov  3 15:56:53 2021   base_node/ 
drwxr-xr-x user user  96 B Wed Nov  3 15:56:53 2021   config/    
drwxr-xr-x user user 128 B Mon Nov  1 12:55:50 2021   mm_proxy/  
drwxr-xr-x user user 128 B Mon Nov  1 12:55:58 2021   sha3_miner/
drwxr-xr-x user user  64 B Wed Nov  3 15:56:53 2021   tor/       
drwxr-xr-x user user 192 B Mon Nov  1 12:56:17 2021   wallet/    
drwxr-xr-x user user  96 B Mon Nov  1 12:20:31 2021   xmrig/     

./base_node:
drwxr-xr-x user user 128 B Fri Oct 29 17:44:56 2021   config/
drwxr-xr-x user user 320 B Wed Nov  3 15:40:55 2021   log/   

./base_node/config:
.rw-r--r-- user user 312 B Mon Nov  1 20:09:38 2021   base_node_id.json 
.rw-r--r-- user user 211 B Mon Nov  1 20:09:38 2021   base_node_tor.json

./base_node/log:
.rw-r--r-- user user  3.3 MB Wed Nov  3 15:54:20 2021   core.log   
.rw-r--r-- user user    1 MB Wed Nov  3 15:53:10 2021   network.log
.rw-r--r-- user user    0 B  Mon Nov  1 13:01:33 2021   other.log  

./config:
.rw-r--r-- user user 4.1 KB Mon Nov  1 20:12:10 2021   log4rs.yml

./mm_proxy:
drwxr-xr-x user user 128 B Mon Nov  1 12:53:33 2021   config/
drwxr-xr-x user user 352 B Wed Nov  3 14:10:04 2021   log/   

./mm_proxy/config:
.rw-r--r-- user user 8.7 KB Mon Nov  1 12:53:33 2021   config.toml
.rw-r--r-- user user 1.1 KB Mon Nov  1 12:53:33 2021   log4rs.yml 

./mm_proxy/log:
.rw-r--r-- user user 8.4 MB Wed Nov  3 15:53:53 2021   core.log   
.rw-r--r-- user user   0 B  Mon Nov  1 13:01:38 2021   network.log
.rw-r--r-- user user   0 B  Mon Nov  1 13:01:38 2021   other.log  

./sha3_miner:
drwxr-xr-x user user 192 B Mon Nov  1 13:01:36 2021   log/   

./sha3_miner/log:
.rw-r--r-- user user 677.7 KB Wed Nov  3 15:49:31 2021   core.log    
.rw-r--r-- user user     0 B  Mon Nov  1 13:01:36 2021   network.log 
.rw-r--r-- user user     0 B  Mon Nov  1 13:01:36 2021   other.log   

./wallet:
drwxr-xr-x user user 192 B Mon Nov  1 12:56:17 2021   ./         
drwxr-xr-x user user 288 B Wed Nov  3 15:56:53 2021   ../        
drwxr-xr-x user user 256 B Wed Nov  3 15:23:19 2021   log/       
drwxr-xr-x user user 128 B Wed Nov  3 15:54:10 2021   weatherwax/

./wallet/log:
.rw-r--r-- user user 564.2 KB Wed Nov  3 15:54:20 2021   core.log   
.rw-r--r-- user user 280.3 KB Wed Nov  3 15:54:08 2021   network.log
.rw-r--r-- user user     0 B  Mon Nov  1 13:01:24 2021   other.log  

./xmrig:
drwxr-xr-x user user    96 B  Mon Nov  1 12:20:31 2021   ./       
drwxr-xr-x user user   288 B  Wed Nov  3 15:56:53 2021   ../      
.rw-r--r-- user user 212.9 KB Wed Nov  3 15:53:53 2021   xmrig.log

You can edit the network configuration (requires a container restart) and log configuration (takes effect within 30s)
by editing the files in the top-level config folder.

Current shortcomings and TODOs

  • It's a CLI only. Next step is to build a nice UI wrapper around this.

  • TODO - Windows support.

  • The blockchain data is stores in docker volumes, and not on the host machine directly. This is due to crippling performance
    limitations one suffers when mounting host file system from Windows or MacOS into docker containers.
    This isn't a big drawback, since you seldom want or need to access the raw blockchain database files anyway. They're
    still accessible. But ensure that you reserve enough space to store the Tari, and optionally,
    Monero blockchains inside the Docker VM
    .

  • TODO - Multi-network sandboxing. Sandboxing currently doesn't work across different networks (e.g. weatherwax and mainnet). This should be fixed soon.

  • Local MoneroD support. The merge-miner proxy doesn't actually use the local MoneroD container if it is running. This
    is left as a TODO for the UI version.

  • Tor control password. For now, you set this manually. In the UI edition, this will be ephemeral and configured automatically.

  • GRPC addresses. The code only supports IPv4 addresses as gRPC configuration parameters. This makes the docker networking
    fragile since the IP addresses must be guessed. TODO - accept DNS names in gRPC configs to make this more robust.

Accessing blockchain data

The blockchain data is stored in a docker volume for performance reasons. If you need to back up or access the LMDB
a blockchain data, you can use something like this to extract it to the host filesystem:

docker run --rm -v $(pwd):/backup -v blockchain:/blockchain ubuntu tar czvf /backup/backup.tar.gz /blockchain

@CjS77 CjS77 changed the title Installer feat: one-click installer - cli edition Nov 3, 2021
@CjS77 CjS77 marked this pull request as draft November 4, 2021 16:32
@CjS77
Copy link
Collaborator Author

CjS77 commented Nov 4, 2021

Picked up a few issues when running on a clean Linux build. Moving to draft while I sort them out

@CjS77 CjS77 marked this pull request as ready for review November 10, 2021 10:27
This seemingly innoucous update required several changes due to changes
in how the compiler interprets dead code.

* A few variables were renamed _x where it looked like x might be used
  in upcoming PRs. YAGNI maybe, but I gave the benefit of the doubt in
this cases.
* Some crates that had LOADS of dead code (ffi crate I'm looking at
  you), I just changed the `deny(dead_code)` to `warn(dead_code)`
* And in some places, I just nuked the dead code, which tidied up some
  heavy code in the wallet in particular. Since the tests pass, I
presume this is fine and there isn't a weird new compioler bug that is
missing where the code is needed.

Add more docker files to installer

Add docker build files for xmrig, mm_proxy, monerod.
Add a quick_start.sh script that demonstrates one-click miner capability
Tidy up docker compose and add services for mining

Update base images to buster

Force cargo to http 1.1 in docker

Move docker specs to launchpad folder

Add script to push latest image bundle

- `push_bundle.sh` examines the `:latest` images of the docker rig and
  pushes them to quay.io, tagging them with both the `latest` tag and
their individual version numbers.
a.k.a. _Tari one-click miner_.

Currently this only works on MacOS and Linux.
Contributions for  `quick_start.bat` are welcomed.

`./quick-start.sh`

and you fire up a tor instance, a base node, wallet, sha3 miner, merge_miner proxy and
xmrig.

All networking, configuration and setup is taken care of via environment
variables and sane defaults.

Any Tari client that supports gRPC can be connected to the node setup.
With the default configuration, the base node
will be at `http://127.0.0.1:18142` and the wallet will be at
`http://127.0.0.1:18143`.

--  Current shortcomings and TODOs

* It's a CLI only. Next step is to build a nice UI wrapper around this.

* TODO - Windows support.

* The blockchain data is stores in docker volumes, and not on the host
  machine directly. This is due to crippling performance
limitations one suffers when mounting host file system from Windows or
MacOS into docker containers.
This isn't a big drawback, since you seldom want or need to access the
raw blockchain database files anyway. Are they're
[still accessible](#accessing-blockchain-data). But **ensure that you
reserve enough space to store the Tari, and optionally,
Monero blockchains inside the Docker VM**.

* TODO - Multi-network sandboxing. Sandboxing currently doesn't work
  across different networks (e.g. weatherwax and mainnet). This should
be fixed soon.

* Local MoneroD support. The merge-miner proxy doesn't actually use the
  local MoneroD container if it is running. This
  is left as a TODO for the UI version.

* Tor control password. For now, you set this manually. In the UI
  edition, this will be ephemeral and configured automatically.

* GRPC addresses. The code only supports IPv4 addresses as gRPC
  configuration parameters. This makes the docker networking
  fragile since the IP addresses must be guessed. TODO - accept DNS
names in gRPC configs to make this more robust.

Resolve file permissions on internal docker volumes
Robust networking support in docker configurations requires us to be able to specify the gRPC address of
things like the node and wallet in terms of domains instead of raw IP addresses.

This adds support for MultiAddr specification of these addresses while still being 100%
backwards compatible.
aviator-app bot pushed a commit that referenced this pull request Nov 10, 2021
Motivation and Context
---

Robust networking support in docker configurations requires us to be able to specify the gRPC address of
things like the node and wallet in terms of domains instead of raw IP addresses.

This adds support for MultiAddr specification of these addresses while still being 100%
backwards compatible.

Also included in #3534 

How Has This Been Tested?
---
No new tests added.
Description
---
- Fixed broken cucumber tests involving wallet startup - the promise was not returned when the wallet started up.
- **Edit** Fixed "ambiguous" error in `Scenario: Massive multiple reorg`

Motivation and Context
---
See above.

How Has This Been Tested?
---
Running cucumber locally with `npm test -- --tags "not @long-running and not @broken"`
Copy link
Collaborator

@stringhandler stringhandler left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Untested, but files are mainly additions and the other changes should have little impact. Will test shortly

@aviator-app aviator-app bot merged commit ec67798 into development Nov 11, 2021
@CjS77 CjS77 deleted the installer branch November 15, 2021 15:34
sdbondi added a commit to sdbondi/tari that referenced this pull request Nov 18, 2021
* development: (32 commits)
  feat: add atomic swap refund transaction handling (tari-project#3573)
  feat: improve wallet connectivity status for console wallet (tari-project#3577)
  v0.21.1
  feat: add error codes to LibWallet for CipherSeed errors (tari-project#3578)
  ci: split cucumber job into two (tari-project#3583)
  feat(wallet): import utxo’s as EncumberedToBeReceived rather than Unspent (tari-project#3575)
  docs: rfc 0250_Covenants (tari-project#3574)
  feat: get fee for transactions for stratum transcoder (tari-project#3571)
  test: make monerod stagenet usage resilient (tari-project#3572)
  feat: add atomic swap htlc sending and claiming (tari-project#3552)
  feat: implement prometheus metrics for base node (tari-project#3563)
  feat: implement multiple read single write for sqlite (tari-project#3568)
  feat: trigger time lock balance update when block received (tari-project#3567)
  test: reduce cucumber ci to critical only (tari-project#3566)
  test: fix cucumber console wallet startup (tari-project#3564)
  chore: add node id/public key to log mdc (tari-project#3559)
  fix: avoid implicit using of the time crate (tari-project#3562)
  feat: one-click installer - cli edition (tari-project#3534)
  ci: add workflow dispatch to libwallet build action (tari-project#3556)
  fix: stop leak of value of recovered output (tari-project#3558)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants