Compile sgminer 5.5.5 in Ubuntu 16.04.1 LTS
Compiling programs from source code under Linux is "easy when you know how", but can seem a bit daunting for new users. This page aims to explain how to compile sgminer for Ubuntu 16.04.1 LTS with the amdgpu-pro driver, and endeavours to explain what we're doing rather than simply to provide a list of commands, on the principle of "give a man a fish".
First, let's explain a couple of prerequisites that we assume as a starting point, as this guide cannot cater to all possible eventualities.
sgminer is software for OpenCL mining on suitable graphics cards. Suitable is relative, of course, but as a working minimum you'll want to be doing this on a machine with
- 64-bit CPU (doesn't need to be high powered, a Celeron G1840 works well)
- 2GB RAM
- one or more modern GPUs suited to OpenCL mining
- this guide does not cover modifying and flashing a custom video BIOS, but typically you'll want to do so
- a power supply adequate to the GPUs you plan to run
- the safe working assumption for AMD RX470 / RX480 series GPUs is a PSU capacity of ~200W per card
For practical intents and purposes, this typically implies current AMD GPUs with 4GB or 8GB of video memory (if you want to be able to mine Ethereum). It should work with nVidia or other GPUs, though performance may not be optimal. Please don't waste your time trying to mine on integrated GPUs, anything with <=2GB of RAM or anything older than a year or two. If it works at all, your mining output will not break even for electricity costs. Any reasonably modern system can suffice, subject to adequate power supply and PCIe slots.
We do not cover the installation of Ubuntu itself; information and instructions are readily available. As a starting point, we assume that you have a working installation of Ubuntu 16.04.1, that you have updated it to the current version of the 16.04.1 LTS distribution (apt update && apt upgrade) and that you have configured what you require as regards system startup, use of on-board graphics, network access and so on.
We now want to install the AMD APP SDK and the amdgpu-pro Linux driver first, as the latter pulls in a number of compiler dependencies for us. Later instructions for installing other dependencies assume that you have a working amdgpu-pro driver and thus a working C compiler.
- Download the AMD APP SDK v3.0
- Follow the instructions in the installation notes
amdgpu-pro version 16.50 contains a newer OpenCL compiler which causes problems with current sgminer versions. Possible solutions include using version 16.40, or installing the 16.50 driver, then overwriting the OpenCL ICD files with the versions from 16.40. This hybrid approach, whilst perhaps less than ideal, gains you the stability of the newer driver whilst maintaining the ability to compile working OpenCL kernels.
- Download the 16.40 driver
- Extract the driver archive with
tar xvJf amdgpu-pro-16.40-348864.tar.xz cd amdgpu-pro-16.40-348864dpkg -i opencl-amdgpu-pro-icd_16.40-348864_*.deb
This will overwrite the OpenCL ICD, but overwriting files like this leaves the package manager unclear about what we want and by default, it will try to reinstall the half-installed packages from 16.50. We can't just remove those as other parts of the amdgpu-pro driver depend on them, so, instruct the package manager to hold the packages we just installed with:
- echo "opencl-amdgpu-pro-icd hold" | dpkg --set-selections
- echo "opencl-amdgpu-pro-icd:i386 hold" | dpkg --set-selections
This will hold these OpenCL package versions so you should be able to apt update and apt upgrade in the usual fashion. Your sgminer should then be able to compile OpenCL binaries properly. Make sure to delete any old .bin files compiled by the 16.50 driver, to force sgminer to rebuild kernels at run time.
-
apt install git autoconf automake libtool build-essentialwill give you a compile environment with GNU autotools -
apt install libncurses5-dev libcurl4-gnutls-devprovides the curses libraries and curl libraries linked by sgminer
That should be everything you need to compile and install sgminer. Depending upon how you intend to run sgminer,
you may also wish to install screen which allows you to run sgminer in the background and attach a terminal to it at will, and may want to consider providing network access, for instance via SSH.
git clone https://github.com/genesismining/sgminer-gm.git
cd sgminer-gm
git submodule init
git submodule update
autoreconf -fi
sgminer's configuration scripts are perhaps due an update. Meanwhile, you'll need to instruct the configure script with information on where to find the OpenCL libraries from amdgpu-pro and the headers you want to use with them, and to disable a couple of previously default behaviours that are no longer relevant to building sgminer against the current amdgpu-pro drivers.
You can also define, if you wish, a specific prefix under which to install sgminer. This is optional but recommended; if you arrange to run sgminer from within its own directory so that OpenCL kernels compiled by any particular version of sgminer live in the same directory, it becomes trivial to install multiple instances alongside each other, which can be very helpful for testing new versions and/or rolling back to older ones. My typical invocation of configure looks like this:
CFLAGS="-Os -Wall -march=native -I/opt/AMDAPPSDK-3.0/include" LDFLAGS="-L/opt/amdgpu-pro/lib/x86_64-linux-gnu" ./configure --disable-git-version --disable-adl --prefix=/opt/sgminer-5.5.5
and what this does is:
- firstly, set the environment variable CFLAGS to pass some instructions to the C compiler
- "-Os" - optimise for small binary size and memory footprint
- "-Wall" - show compiler warnings about (non-critical) issues in the source code
- "-march=native" - use all available optimisations for this machine, don't try to be portable
- "-I/opt/AMDAPPSDK-3.0/include" - look here for include files, picks up OpenCL headers from AMDAPPSDK-3.0
- then, set the environment variable LDFLAGS to pass some instructions to the linker
- "-L/opt/amdgpu-pro/lib/x86_64-linux-gnu" - look here for libraries, picks up OpenCL libs from amdgpu-pro
- finally, run the
configurescript located in the current working directory and- disable attempts to use ADL, as it doesn't work with amdgpu-pro
- disable setting the software version from git tags that may or may not exist
- configure the software to install into the path /opt/sgminer-5.5.5
The configure script will examine the system environment (producing a lot of output), write out the files necessary to instruct the C compiler, and display a summary of the configuration. The last few lines of output should look something like this:
------------------------------------------------------------------------
sgminer 5.5.5-gm
------------------------------------------------------------------------
Configuration Options Summary:
Use git version......: no
libcurl(GBT+getwork).: Enabled: -lcurl
curses.TUI...........: FOUND: -lncurses
OpenCL...............: FOUND. GPU mining support enabled
ADL..................: Detection overrided. GPU monitoring support DISABLED
Compilation............: make (or gmake)
CPPFLAGS.............:
CFLAGS...............: -Os -Wall -march=native -I/opt/AMDAPPSDK-3.0/include
LDFLAGS..............: -L/opt/amdgpu-pro/lib/x86_64-linux-gnu -lpthread
LDADD................: -ldl -lcurl submodules/jansson/src/.libs/libjansson.a -lpthread -L/opt/AMDAPPSDK-3.0/lib/x86_64 -lOpenCL -lm -lrt
Installation...........: make install (as root if needed, with 'su' or 'sudo')
prefix...............: /opt/sgminer-5.5.5
As can be seen in the output, this reflects the instructions we gave above, plus the various shared libraries we shall require. Provided that we have found curl, curses and OpenCL (and, if we want to use amdgpu-pro, the correct path), we should be ready to compile sgminer.
In its simplest form, the command to build the sources is just make. You can speed things up using multiple parallel compile threads; an optimal number is typically number of CPU cores plus one, i.e.
-
make -j3on a 2-core CPU -
make -j5on a 4-core CPU
If you'll be compiling sgminer on a whole farm of rigs, or if you just like Bash one-liners, this will run make and opportunistically spit out the appropriate -jN switch if it can make that call based on sysfs, or just run plain make if not. Quite unnecessary for a one-off compile, but nice to know.
make $(if $(THREADS="-j$(($(tail -c 2 /sys/devices/system/node/node0/cpulist 2>/dev/null)+2))"); then echo $THREADS; fi)
Whichever way, make will now try to compile the various source code and link it together to form the sgminer binary. It will produce various warnings and advisories along the way, which are non-critical unless something fails. If your compile is successful, you should end up with an executable file sgminer in the same directory. You can verify this with
root@raptor:~/sgminer-gm# ./sgminer --version
sgminer 5.5.5-gm
and / or:
root@raptor:~/sgminer-gm# ./sgminer --ndevs
[13:24:24] CL Platform vendor: Advanced Micro Devices, Inc.
[13:24:24] CL Platform name: AMD Accelerated Parallel Processing
[13:24:24] CL Platform version: OpenCL 2.0 AMD-APP (2117.10)
[13:24:24] Platform devices: 6
[13:24:24] 0 Ellesmere
[13:24:24] 1 Ellesmere
[13:24:24] 2 Ellesmere
[13:24:24] 3 Ellesmere
[13:24:24] 4 Ellesmere
[13:24:24] 5 Ellesmere
[13:24:24] 6 GPU devices max detected
The first of these will fail if any libraries are missing, so if we get a version number then the compiled binary duly executes on our system. The second checks for OpenCL GPU devices on the default OpenCL platform, which will be AMD APP SDK unless other OpenCL platforms already exist on the system. You may in such cases need to use
./sgminer --gpu-platform 1 --ndevs
in order to locate the correct OpenCL platform. If both commands work without error and the latter indicates the correct OpenCL platform, you're well on the way to a working installation.
As simple as make install; you'll need to be root or use sudo. This will install sgminer to the path we specified with the --prefix option earlier on, or to /usr/local if you did not specify a prefix. Installing sgminer is not compulsory; it can be configured and used from within the source directory, but installing to a separate working directory with the --prefix option makes it easier over time to handle multiple coexisting instances of sgminer.
The range of options for configuring sgminer is substantial, and this HOWTO does not seek to cover all possible options. Rather, we will look at a minimal set of configuration parameters adequate to starting sgminer; beyond that, documentation on optimal configurations per algorithm can gladly be linked from here, but not much of it has been written yet.
We need to set some environment variables immediately prior to running sgminer, in order to configure the underlying OpenCL platform for the application we intend to run. Thus, the usual way of starting sgminer involves creating a script to set the required variables and launch sgminer, which can then take its runtime configuration settings from any, or some combination of:
- the command line arguments given to sgminer at launch
- a JSON format configuration file on the local disk
- a central configuration file loaded over the network
- commands given over the sgminer API interface (if so configured)
- commands given via the curses terminal interface (if so configured)
A basic startup script that simply sets the variables and runs sgminer might look something like:
#!/bin/bash
export GPU_FORCE_64BIT_PTR=1
export GPU_USE_SYNC_OBJECTS=1
export GPU_MAX_ALLOC_PERCENT=100
export GPU_SINGLE_ALLOC_PERCENT=100
export GPU_MAX_HEAP_SIZE=100
[sgminer command]
where the nature of [sgminer command] varies according to whether we are relying on a configuration file or
specifying arguments at command line. If you have followed the earlier suggestion to install sgminer under its own custom prefix, you may want to add in a cd /opt/sgminer-5.5.0 to change into that working directory before launching sgminer; this has the advantage of keeping any OpenCL binaries compiled by sgminer in the same directory.
At its simplest, you can have sgminer behave much like any other command line OpenCL miner, but you will need to give it a few extra command line arguments.
Here's an example of the most basic startup script for Ethereum pool mining on a single server, assuming our sgminer installation is in /opt/sgminer-5.5.5:
#!/bin/bash
export GPU_FORCE_64BIT_PTR=1
export GPU_USE_SYNC_OBJECTS=1
export GPU_MAX_ALLOC_PERCENT=100
export GPU_SINGLE_ALLOC_PERCENT=100
export GPU_MAX_HEAP_SIZE=100
cd /opt/sgminer-5.5.5
./sgminer -T -I21 -o stratum+tcp://pool.alpereum.ch:3004 -u 0xc91B3f6a889a9963c7eaE2d5BD5Ca88a6Ae2d2EA -p x -k ethash
A brief explanation:
- "-T" output as text, do not load the curses terminal interface
- "-I21" set GPU work intensity to 2^21 (equals raw intensity of 2097152)
- sets the number of parallel scanning threads per GPU
- varies widely by algorithm and GPU memory footprint as to how many threads will fit
- "-o stratum+tcp://pool.alpereum.ch:3004" sets a pool server and port
- "-u" username, in this case an Ethereum address
- "-p" password, for some reason you still need it even for pools that don't require it
- "-k ethash" specifies the algorithm to use with this pool
Make sure that the script is executable with chmod 755 /path/to/script. The available options to sgminer should largely be self-explanatory; you can find a summary of available options with sgminer --help and you will want to use some of them.
You can also place some or part of the configuration into JSON formatted configuration files; see 'example.conf' supplied with the sgminer sources for a template. This may be helpful if you want to make the same central configuration available to multiple rigs, or to keep your pool configuration separate from your rig startup, or to select a configuration file for a particular rig during startup.
Be aware that sgminer's default behaviour, when not instructed with a config file, is to attempt to load its default configuration file from a number of locations, including ~/.sgminer/sgminer.conf and /etc/sgminer.conf, in addition to any arguments received on the command line. To disable that behaviour, do one of the following:
- make sure that no unwanted configuration files are available (will use command line arguments), and/or
- specify a preferred configuration file with the -c switch, and/or
- specify a default configuration file with the --default-config switch, and/or
- specify the default configuration file as --default-config=/dev/null to ignore default config files
- this will produce an error "Fatal JSON error in config file", although not fatal if sgminer is configured elsewhere
The author's own approach is to avoid JSON configuration files and give the required arguments in his startup script, but this is for a fairly small operation. See below for the reason why.
Also note that sgminer's ability to save configuration files tends to lag behind its feature set. For example, sgminer supports load balancing with quota per pool (and very good it is, too) but the built-in configuration file writer does not support the necessary syntax to preserve quota settings, it remaining an "experimental" feature.
Reading such configuration files works, but can lead to the situation of sgminer corrupting its own config as it partially saves changes made via the curses interface or the API. This can be worked around by reading a config file to which there is no write permission, or copying a config file into place at startup. Or just not using a configuration file and supplying the requisite configuration at startup.
It is possible to write more elaborate startup scripts to generate the desired sgminer command line incantations on the fly at startup. This is broadly outside the scope of this HOWTO (the possibilities are endless) but worth brief mention that it's feasible. For instance, this might be used to set pools as enabled or disabled (see the --state switch) at run time, or to select a configuration based on the GPUs or software versions in use.
FIXME: link an example or two
In much the same vein, configuring and using the sgminer API is outside the realms of a "getting started" HOWTO; more information can be found in the sgminer documentation and will eventually be incorporated into the Wiki. Mentioned here for awareness of the option; with a number of pools configured, the API can be used to instruct a running sgminer instance as to what you want it to do.
FIXME: link an example or two
When we have a working startup script, launching sgminer should be as simple as running that script, but there are one or two considerations to bear in mind.
It is assumed and suggested that sgminer be run as root; this ensures it has the necessary permissions to write to sysfs to control the GPU settings, and where so configured, to issue a reboot if needed. It is possible to launch sgminer as an unprivileged user (make sure that user is a member of group 'video'), but there's little to be gained by it, at least on a dedicated rig. If you're mining on a desktop, it's perhaps still easiest to run sgminer as root, but you will need to set up su or use sudo if you want to be able to start and stop it as the unprivileged user running Xorg.
sgminer includes an interactive terminal interface based on curses which (when everything is working) is more flexible and more informative than the output in --text-only mode. However, the curses interface requires that it be connected to a terminal for input and output; without external help (i.e. screen) one cannot launch a curses interface in the background, and running from SSH connections is vulnerable to the controlling terminal being lost when the connection drops.
Therefore, to be able to run sgminer on system startup (i.e. in the background and without a terminal connected) your options are
- launch into the background in text-only mode, control is still possible via the sgminer API
- wrap the curses mode in a console window manager such as 'screen' and put that into the background
The latter approach using screen is perhaps the nicer of the two, as besides providing the ability to launch sgminer into the background and connect or disconnect from it at will, it provides the option to run an arbitrary assortment of processes alongside the miner and multiplex between them in a single terminal session.
GNU screen has a fine manual of its own, so we will not cover all possible use cases here. Instead, here's that startup script modified to use the curses interface and to work with screen:
#!/bin/bash
export GPU_FORCE_64BIT_PTR=1
export GPU_USE_SYNC_OBJECTS=1
export GPU_MAX_ALLOC_PERCENT=100
export GPU_SINGLE_ALLOC_PERCENT=100
export GPU_MAX_HEAP_SIZE=100
cd /opt/sgminer-5.5.5
`which screen` -AdmS sgminer ./sgminer -I22 -o stratum+tcp://pool.alpereum.ch:3004 -u 0xc91B3f6a889a9963c7eaE2d5BD5Ca88a6Ae2d2EA -p x -k ethash
`which screen` -dr -S sgminer -p sgminer || /bin/true
The first line launches sgminer in a backgrounded screen session using the same command line as above, minus the -T (--text-only) switch so that we get the curses interface. We name the screen session as 'sgminer' so it can later be identified in screen -list or when reconnecting to it.
The second line tries to connect us to the screen session we created (and preselects the tab containing sgminer, in case we add multiple tabs later); this will succeed where it is run from a terminal and fail gracefully when run on system startup. The || /bin/true on the end prevents the failure of this last from being considered as an error, so our startup script will return without error when called e.g. from /etc/rc.local.
There are a variety of ways to launch a program on startup; Ubuntu 16.04.1 uses systemd to manage its startup files, so in theory we could create and install the necessary systemd service files to launch sgminer from systemd. The time-honoured method of adding local extras to your system startup, though, is simply to call the script from /etc/rc.local and this works fine for our startup script above.
You can connect to the resulting screen session with (as root) screen -dr.
There are a few considerations that we have not covered in this HOWTO, but that may need attention in the course of configuring sgminer for automated startup. Headings added here for relevant links later.