Guide to Mining Zcash on Amazon EC2

Austin Williams edited this page Aug 16, 2017 · 43 revisions

Mining Zcash on Amazon EC2

This guide is no longer applicable. You should use third-party GPU mining software and mine to a pool now. This wiki will remain here for historical purposes.

This guide is for mining Zcash on Amazon EC2. If you are looking to solo mine on a home computer, you'll want to use this other mining guide instead.

Mining with EC2 is an easy way to control many miners without a large up-front capital investment. This guide will only cover solo mining using the zcashd native miner -- though all of your miners will mine to the same keypool (if you configure it that way. See the 'Key Management' section below).

This is an unofficial guide to mining Zcash on Amazon EC2. I put this together for some folks who asked for it on Reddit. I'm not affiliated with the Zcash project or developers in any way, and this guide is based solely on what I've learned while floundering with the software. Use this guide at your own risk.

The first part of this guide is taken from the Public-Sprout-Guide. In particular, the Getting Started, Compiling, and Testing sections are copied here almost verbatim. If you struggle with any of these sections, see the Public Sprout Guide.

Disclaimer: This is beta software and it changes rapidly. I make no promises that I'll keep this guide up-to-date. You are responsible for your own security. Use at your own risk.

Disclaimer: It is almost certainly not profitable to solo mine on EC2. Pool mining is a little better because it can lower your variance significantly. But even with pool mining your expected value will be the same as if you solo mine. This is not a "get rich quick" scheme. Mining is very competitive. Continue at your own risk.

Big Picture Strategy

This is the overall strategy we're taking here:

  1. Set up a miner on an EC2 instance and configure it well
  2. Take an image of the instance
  3. Reuse the image on as many EC2 instances as we can afford

The key is to configure your EC2 instance correctly during step (1) so that things go well for you during step (3). Most importantly, you need to implement a key-management solution so you don't you lose any coins you've mined.

You'll also want to configure your miners to start automatically on boot up, and to randomize your initial nonce so that your miners don't waste time searching the same subset of the nonce space.

When we're finished, you'll have a completely automated, scalable, hands-off mining configuration for mining Zcash on Amazon EC2.

Prerequisites

This guide assumes that you already know how to get set up with Amazon EC2 and how to launch and connect to EC2 linux instances.

You should also have basic linux command line skills. If you struggle with anything, just remember that google is your friend, and so are most of the folks in the Zcash forums. We'll be using Ubuntu Server 14.04.

Getting Started

Log into Amazon EC2 and launch an Ubuntu Server 14.04 instance with instance type m4.large. (You can choose another instance type instead, but I recommend that it have at least 8 GB of RAM for this installation step.)

I recommend creating a new security group called "zcash" that you'll use with all of your instances. To that security group, add an inbound rule with type: Custom TCP Rule, Protocol: TCP, Port Range: 18232, and Source: Anywhere (0.0.0.0/0). This will allow other nodes to connect to your node.

Installing Zcash

Now SSH into your instance and run updates:

$ sudo apt-get update

Then install the dependencies:

$ sudo apt-get install build-essential pkg-config libgtest-dev libc6-dev m4 g++-multilib autoconf libtool ncurses-dev unzip git python zlib1g-dev wget bsdmainutils automake

Then fetch the zcash repository with git and run fetch-params.sh like so:

$ git clone https://github.com/zcash/zcash.git
$ cd zcash/
$ git checkout origin
$ ./zcutil/fetch-params.sh

This will fetch the most recent version of zcash proving and verifying keys and place them into ~/.zcash-params/. These keys are, together, nearly 1.5GB in size, so it may take some time to download them.

Compiling

Ensure you have successfully installed all system package dependencies as described above. Then run the build:

./zcutil/build.sh -j$(nproc)

This should compile the dependencies and build zcashd.

Testing

The tests take a while to run but I encourage you to do them anyway. For the first set of tests run:

$ ./qa/zcash/full-test-suite.sh

If you notice any failed tests, please inform the zcash devs.

To run the next set of tests run:

$ ./qa/pull-tester/rpc-tests.sh

Again, if you get any failures, please inform the Zcash devs.

Configuring the Miner

Now that Zcash is installed, we're going to configure the miner. This is a little more complicated than for solo mining on a home machine. So be patient.

Initial Configuration

We need to set up a config file, so first we make a directory for it:

$ mkdir -p ~/.zcash

Next we need to create a config file and give it some configurations. First we create and edit the file:

$ nano ~/.zcash/zcash.conf

Enter the following in the file:

addnode=mainnet.z.cash
rpcuser=rpcusername
rpcpassword=rpcpassword
gen=1
genproclimit=4
equihashsolver=tromp

Naturally, you'll want to use a random password. You'll also want to set the genproclimit to the number of cores on which you want to mine.

Then save the file (within nano you do this via CTRL+X then Y).

Next, start zcashd via the command:

$ ~/zcash/./src/zcashd -externalip=`wget --quiet -O- http://169.254.169.254/latest/meta-data/public-ipv4` -daemon

Give your server a few seconds to finish loading. You can check to see if it's working via:

$ ~/zcash/./src/zcash-cli getinfo

Give your node time to catch up with the blockchain. You can see how many blocks your node has received/checked by running the following command and looking at the line labelled blocks:

$ ~/zcash/./src/zcash-cli getinfo

That's it! You've got the basic solo mining configuration set up. If you were mining on your own at home, you'd be ready to go. But to mine on EC2 using our 'Big Picture Strategy', we need to tweak some things -- or else you could lose coins, waste hashing power, etc.

Choose a Key Management Option

When you first start up Zcashd it creates a keypool containing 100 addresses. The private keys for these addresses are stored in your wallet.dat file. If you use the same image on multiple EC2 instances, all of your instances will begin with the same 100 addresses in their keypool. And that's okay. But things can get ugly quickly, and here's why:

Once an address in your keypool receives coins (either by mining or receiving coins from another address) that address is removed from your keypool. This means that, eventually, your keypool (which was originally filled with 100 addresses) will become empty. When that happens, Zcashd will randomly create new addresses as needed, and store the private keys in your wallet.dat file. This is problematic given our mining strategy, because it means that -- once the initial pool of 100 addresses is used up -- each of your miners will be mining to addresses from different keypools. And unless you manually log into each of your miners and back up each of those wallet.dat files, you could very easily lose the private keys to some of your mined coins.

There are three ways to deal with this problem, each with their own pros and cons. You need to choose one of them. I'll explain the differences and you decide whichever one is right for you. I personally recommend Option 2.

Option 1: Use a Hierarchical Deterministic (HD) Wallet

Using a hierarchical deterministic is the best possible solution here, because it guarantees that all of your miners will be mining to the same of pool addresses forever. The keypools will never diverge, even though your are mining to a different address every time.

The 'con' here is that Zcashd doesn't currently implement HD wallets. However, there is hope. Bitcoin Core very recently merged HD wallets into the core client. As a result, it's possible that we could a native HD wallet in Zcash soon. Until then, I'd advise you to choose another option.

Option 2: Start With a Huge Key Pool (Recommended)

Another approach is to start with a huge keypool (say, 30,000 addresses) before you take your image. That way, even if you have dozens of miners, you'd be unlikely to run your common keypool dry. This method is easy to implement. Be aware, however, that your keypool is still finite. You will (in principle) still run out of common keys eventually. Of course, by then you will likely be using a different wallet and image anyway. I recommend this option for key management until HD wallets are implemented in the core client.

If you want to go this route, here's how to do it:

SSH into your instance and make sure Zcashd is running:

$ ~/zcash/./src/zcashd -daemon

You can see the current size of your keypool by running:

$ ~/zcash/./src/zcash-cli getinfo

and looking for "keypoolsize". That's how many unused addresses you have in your keypool. We want to make that number a lot bigger.

Start by issuing the command:

$ ~/zcash/./src/zcash-cli keypoolrefill 30000

It will take a while to generate those 30,000 addresses so be patient.

You can check to see if it worked correctly by using:

$ ~/zcash/./src/zcash-cli getinfo and checking the "keypoolsize" again. It should be much larger now.

Option 3: Mine to the Same Address Every Time

Rather than mining to a newly chosen address every time, you can opt to have every miner mine to the same address every time. One way to accomplish this is by preventing keys from being removed from the keypool after they've been used to receive coins.

While this will guarantee that your keypools will never diverge, it also has a negative effect on your privacy. It allows everyone to know how many coins you've mined. If you choose this option be sure you understand the consequences and use at your own risk.

If you want to go this route, here's one way to do it:

SSH into your instance and make sure zcashd is stopped:

$ ~/zcash/./src/zcash-cli stop

First we need to edit the miner.cpp file:

$ nano ~/zcash/src/miner.cpp

We want to remove a from this file. To find that line quickly, press CTRL+W to enter search mode, then paste in reservekey.KeepKey(); and press enter.

Once there, delete the line reservekey.KeepKey();. This will prevent the miner from removing addresses from the keypool once coins have been successfully mined to an address. Save the file via CTRL+X then Y.

Next, we need to edit the wallet.cpp file:

$ nano ~/zcash/src/wallet/wallet.cpp

We want to remove a line from this file. To find that line quickly, press CTRL+W' to enter search mode, then paste inreservekey.KeepKey();` and press enter.

Delete the line reservekey.KeepKey();. This will prevent wallet.cpp from removing an address from the keypool when that address receives funds. Save the file via CTRL+X then Y.

Next, recompile the source code with:

$ ~/zcash/./zcutil/build.sh -j4

And finally, start zcashd:

$ ~/zcash/./src/zcashd -daemon

Now your miners will always mine to the same address (the first address in your keypool).

Backup Wallet.dat

You'll want to copy your wallet.dat file from your remote EC2 instance to your local machine so you have a backup. If you don't know how to copy files from an EC2 instance to your local machine, see the very bottom of this page for instructions.

The path to your wallet.dat file is: ~/.zcash/wallet.dat.

!IMPORTANT NOTE!

At this point your keypool does NOT contain any protected addresses. So if you generate a protected address, that address will not be coming from your existing keypool. So every time you generate a protected address you need to backup your wallet.dat file again.

Make the Miner Start Automatically on Boot

You'll probably want zcashd to start mining automatically after booting up -- without you having to log in. To accomplish this, start with the command:

$ crontab -e

Choose option 2, then add this line to the file and save it:

@reboot ~/zcash/./src/zcashd -externalip=`wget --quiet -O- http://169.254.169.254/latest/meta-data/public-ipv4` -daemon

Easy as that. Your miners should start automatically now without you having to log in.

Take an Image of Your Instance

At this point you're ready to create an image. You can do that from the EC2 console by choosing your instance and choosing the 'Actions' button. Consult Amazon's documentation if you have trouble.

Set Up Spot Requests

You can choose any instance types you want when setting up your spot instances -- so long as you have enough RAM to solve Equihash with the dev's chosen parameters (~ 1 GB). Remember, the bottleneck isn't memory, it's memory bandwidth.

Choosing an instance type with a lot of RAM, like an r3.8xlarge, is a waste of money. More memory won't help you mine more coins. Having more instances running will.

All costs being equal, it would be better to have 2 m3.large instances than 1 r3.large instance. While both configurations have the same amount of (total) memory, the former has higher memory bandwidth.

Final Thoughts

When the spot price drops low enough to trigger your requests, your instances will boot up and automatically start mining without any interaction needed from you. When the spot price rises above your max bid, your miners will shut down (and all your mined coins will remain safe in the wallet you backed up to your local computer). This is a completely hands-off configuration.

Best of luck, and happy mining!

Acknowledgments

Special thanks to @aniemerg on the Zcash forum for the -externalip suggestion which lets your node connect to other nodes on the network.

Donations

Donations are appreciated but not expected. If you think I've helped you by putting this guide together can show me some love here:

QRC

BTC: 1AwkkQrKPNPPKkyALd9KfnzCu6Q1CWniPZ

Thank you!

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.