chaincfg: Avoid block 1 subsidy codegen explosion.

[jrick]

chaincfg/v2 modified the Params type to contain the required payouts
created by the miner of the block at height 1. However, it was
discovered that, due to code generation issues of the Go compiler,
this caused the package's weight (measured by
github.com/jondot/goweight) to explode to 7.7 MB. This appears to be
due to an enormous amount of function calls, all of which could panic,
when building the large slice literal.

This improves the compiled size of the package by using code
generation techniques to optimize the memory layout and code
transformation of the subsidy values each time a params function is
called. A single string contains all scripts for subsidy payouts
concatenated together, in hex encoding, and a slice contains the index
in the script data and the required payout amount. This is iterated
in a loop to create a []TokenPayout each time it is needed.

The new size of the compiled package using Go 1.12.7 on OpenBSD/amd64
is 379 kB, a reduction of 7.32 MB, and about 5% of the original size.

[matheusd]

Tested regenerating subsidytables.go and the resulting block one ledger.

Strangely, I'm not sure whether the results of go weight are OS-dependent or if I'm using it wrong but I didn't notice any size differences before/after this PR.

[jrick]

I see the same issue on darwin/amd64.

$ mkdir -p /tmp/chaincfgtest && cd /tmp/chaincfgtest
$ go mod init chaincfgtest
go: creating new go.mod: module chaincfgtest
$ cat > main.go << EOF
> package main
> 
> import _ "github.com/decred/dcrd/chaincfg/v2"
> 
> func main() {}
> EOF
$ goweight | head
  7.7 MB github.com/decred/dcrd/chaincfg/v2
  3.3 MB runtime
  1.8 MB net
  1.3 MB reflect
  940 kB math/big
  813 kB syscall
  769 kB github.com/decred/dcrd/wire
  452 kB time
  436 kB fmt
  412 kB internal/x/net/dns/dnsmessage
$ go mod edit -replace=github.com/decred/dcrd/chaincfg/v2=$HOME/src/dcrd/chaincfg
$ goweight | grep chaincfg/v2
  687 kB github.com/decred/dcrd/chaincfg/v2
$ uname -a
Darwin mini.local 18.6.0 Darwin Kernel Version 18.6.0: Thu Apr 25 23:16:27 PDT 2019; root:xnu-4903.261.4~2/RELEASE_X86_64 x86_64

It needs to be linked as part of a larger build for the goweight tool to work. You could presumably also find the compiled .a file in the go build cache but became more difficult to find these since the cache changes in 1.12.

[jrick]

You can also see the size increase in a compiled binary:

$ go build # still with replace
$ ls -lh chaincfgtest 
-rwxr-xr-x  1 jrick  staff   3.1M Jul 26 09:13 chaincfgtest
$ go mod edit -dropreplace=github.com/decred/dcrd/chaincfg/v2
$ go build
$ ls -lh chaincfgtest 
-rwxr-xr-x  1 jrick  staff   9.4M Jul 26 09:13 chaincfgtest

[davecgh]

Since this involves touching definitions that affect consensus, I wrote some quick and dirty code to ensure the definitions are the same before and after by importing an earlier version of the module as well as the changes introduced by this PR and comparing the programmatically comparing results.

go.mod:

module comparedefs

go 1.12

replace github.com/decred/dcrd/chaincfg/v2 => ../

require (
        github.com/decred/dcrd/chaincfg v1.5.1
        github.com/decred/dcrd/chaincfg/v2 v2.0.2
        github.com/decred/dcrd/dcrutil v1.4.0
        github.com/decred/dcrd/txscript v1.1.0
)

main.go:

package main

import (
	"bytes"
	"fmt"

	v1 "github.com/decred/dcrd/chaincfg"
	"github.com/decred/dcrd/chaincfg/v2"
	"github.com/decred/dcrd/dcrutil"
	"github.com/decred/dcrd/txscript"
)

func compareLedgers(name string, v1Params *v1.Params, newParams *chaincfg.Params) {
	oldLedger := v1Params.BlockOneLedger
	newLedger := newParams.BlockOneLedger

	if len(oldLedger) != len(newLedger) {
		panic(fmt.Sprintf("%s ledger length mismatch", name))
	}

	for i := range oldLedger {
		// Convert old address to full script.
		addr, err := dcrutil.DecodeAddress(oldLedger[i].Address)
		if err != nil {
			panic(err)
		}
		oldScript, err := txscript.PayToAddrScript(addr)
		if err != nil {
			panic(err)
		}

		// Ensure all fields are expected values.
		if newLedger[i].ScriptVersion != 0 {
			panic(fmt.Sprintf("%s ledger entry %d script version not 0", name, i))
		}
		if !bytes.Equal(oldScript, newLedger[i].Script) {
			panic(fmt.Sprintf("%s ledger entry %d script mismatch", name, i))
		}
		if oldLedger[i].Amount != newLedger[i].Amount {
			panic(fmt.Sprintf("%s ledger entry %d amount mismatch", name, i))
		}
	}
}

func main() {
	compareLedgers("mainnet", &v1.MainNetParams, chaincfg.MainNetParams())
	compareLedgers("testnet3", &v1.TestNet3Params, chaincfg.TestNet3Params())
	compareLedgers("simnet", &v1.SimNetParams, chaincfg.SimNetParams())
	compareLedgers("regnet", &v1.RegNetParams, chaincfg.RegNetParams())
}

Result:

$ go build && ./comparedefs
$

I also confirmed the reduced weight:

$ goweight | grep chaincfg
  379 kB github.com/decred/dcrd/chaincfg/v2