Plot-CPU is a tool utilizing multi-core CPUs to boost MassDB plotting process.
Build
Make sure your Golang version is newer than go1.13.3
and GO111MODULE
is enabled.
go build -o bin/plot-cpu main.go
to build an executable.
Run
./plot-cpu <output_dir> <ordinal> <public_key> <bit_length>
to plot a MassDB.
For example: ./plot-cpu /path/to/root/dir 0 0372a265421441050884d204292775565b9e7d16dd574a47e64cefff0ec1829ad3 24
.
You can also import massdb.cpu
into your own project:
import (
"github.com/massnetorg/Plot-CPU/massdb.cpu"
"massnet.org/mass/poc/engine/massdb"
)
// It is your responsible to allocate enough memory before plotting
if allocated, err := massdb_cpu.AllocateMemory(bl); err != nil {
panic(err)
}
defer massdb_cpu.ReleaseMemory()
mdb, err := massdb.CreateDB(massdb_cpu.TypeMassDBCPU, outputDir, ordinal, publicKey, bitLength)
if err != nil {
panic(err)
}
result := mdb.Plot()
if err := <-result; err != nil {
panic(err)
}
MASS_MAX_PLOT_MEM
specify the max usable memory for plotting (in MiB).MASS_TMPDIR
specify the directory storing temp files while plotting (should be a SSD with at least 64 GiB free space).
This implementation provides four different plotting strategies for different size of Memory and SSD.
Plotting strategies are mainly decided by the available(free) memory.
Here defines:
BL = bit lenght of a massdb,
FREE_MEM = max available free memory,
MASS_TMPDIR = fast SSD directory to store temp files while plotting,
MAP_A_MEM = required memory for HashMapA = (2 ^ BL) * [(BL + 7) / 8]
MAP_B_MEM = required memory for HashMapB = MAP_A_MEM * 2
Given the parameters above, select the proper plotting strategy:
-
MAP_A_MEM + MAP_B_MEM <= FREE_MEM:
Calculate and store HashmapA in memory => Calculate and store HashmapB in memory => Write HashmapB to file. (Done)
-
MAP_B_MEM <= FREE_MEM < MAP_A_MEM + MAP_B_MEM:
Calculate and store HashmapA in memory => Write HashmapA to file ($MASS_TMPDIR) => Calculate and store HashmapB in memory => Write HashmapB to file => Remove HashmapA file. (Done)
-
MAP_A_MEM <= FREE_MEM < MAP_B_MEM:
Calculate and store HashmapA in memory => Write HashmapA to file ($MASS_TMPDIR) => Calculate and flush HashmapB window to file (2 rounds) => Remove HashmapA file. (Done)
-
FREE_MEM < MAP_A_MEM:
Use calcWorker and rwWorker to generate HashmapA file (MASS_TMPDIR) => Remove temp files (MASS_TMPDIR) from HashMapA => Use calcWorker and rwWorker to generate HashmapB file => Remove HashmapA file => Remove temp files (MASS_TMPDIR) from HashMapB. (Done)
Disk requirements (in Bytes) for these strategies are:
| Strategy | 1 | 2 | 3 | 4 |
| MASS_TMPDIR | 0 | MAP_A_MEM | MAP_A_MEM | MAP_B_MEM * 2 |
| OUTPUT_DIR | MAP_B_MEM | MAP_B_MEM | MAP_B_MEM | MAP_B_MEM |
For more details, see massdb.cpu/plot.go
.
Plot-CPU
is licensed under the terms of the MIT license. See LICENSE for more information or see https://opensource.org/licenses/MIT.