Merge upstream (#2)

* Improve error message about non-active segwit on simnet

I started playing with simnet and was confronted with error message:

```
[ERR] FNDG: Unable to broadcast funding tx for ChannelPoint(<point>:0):
-22: TX rejected: transaction <tx> has witness data, but segwit isn't active yet
```

I wasn't aware of the activation period so I got quite puzzled.
Google helped. But I think the message could mention likely cause.

Newly it optionally prints something like:

```
(The threshold for segwit activation is 300 blocks on simnet, current best height is 113)
```

* btcctl: add regtest mode to btcctl

* build: replace travis-ci with github actions.

test go 1.14
use golangci-lint

* build: update deps

* build: clean linter warnings

* btcjson: change getblock default verbosity to 1

This change makes btcd's getblock command match bitcoind's. Previously
the default verbosity was 0, which caused errors when using the
rpcclient library to connect to a bitcoind node - getblock would
unmarshall incorrectly since it didn't expect a verbosity=1 result when
it did not specify verbosity.

* rpcclient: send legacy GetBlock request for backwards compatibility

Without this, users of this library wouldn't be able to issue GetBlock
requests to nodes which haven't updated to support the latest request
format, namely the use of a single `int` parameter to denote verbosity
instead of two `bool`s.

* rpcclient: Add cookie auth

Based on Hugo Landau's cookie auth implementation for Namecoin's ncdns.

Fixes btcsuite#1054

* rpcclient: Refactor cookie caching

* rpcclient: Try user+pass auth before cookie auth

* rpcclient: Read first line of cookie instead of trimming space

* rpcclient: serialize nil inputs to empty list

* Improve chain state init efficiency

Remove unnecessary slice of all block indexes and
remove DB iteration over all block indexes that
used to determined the size of the slice.

* Add blockchain.NewUtxoEntry() to directly create entries for UtxoViewpoint

The current methods to add to a UtxoViewpoint don't allow for a situation where
we have only UTXO data but not a whole transaction.  This commit allows
contstruction of a UtxoEntry without requiring a full MsgTx.

AddTxOut() and AddTxOuts() both require a whole transaction, including the inputs,
which are only used in order to calculate the txid.  In some situations, such as
with use of the utreexo accumulator, we only have the utxo data but not the
transaction which created it.

For reference, utreexo's initial usage of the blockchain.NewUtxoEntry() function is at
https://github.com/mit-dci/utreexo/pull/135/files#diff-3f7b8f9991ea957f1f4ad9f5a95415f0R96

* Add getchaintxstats JSON-RPC client command

* Add fundrawtransaction RPC call

* Add getbalances RPC client command

* rpcclient: Add GetTransactionWatchOnly method

* peer: knownInventory, sentNonces - use generic lru

While here, also rename and generalize limitMap and apply to
other maps which need to be bounded.

* btcec: Avoid panic in fieldVal.SetByteSlice for large inputs

The implementation has been adapted from the dcrec module in dcrd. The
bug was initially fixed in decred/dcrd@3d9cda1 while transitioning to a
constant time algorithm. A large set of test vectors were subsequently
added in decred/dcrd@8c6b52d.

The function signature has been preserved for backwards compatibility.
This means that returning whether the value has overflowed, and the
corresponding test vectors have not been backported.

This fixes btcsuite#1170 and closes a previous attempt to fix the bug in btcsuite#1178.

* config+service_windows: add flag to disable win service

To run integration tests with btcd on Windows in non-interactive
environments (such as the Travis build with Windows machines), we
need to make sure we can still spawn a child process instead of only a
windows background service.

* updated docs for getblock-verbosity fixes

* Update json_rpc_api.md

Corrections suggested by @onyb btcsuite#1608 (comment)

* netsync: handle notfound messages from peers

backport from decred/dcrd#2253

When a peer sends a notfound message, remove the hash from requested
map.  Also increase notfound ban score and return early if it
disconnects the peer.

* release: update release script path

* release: remove old scripts and update process doc

- remove prep_release.sh and notes.sample
- update license in release.sh
- add notes for maintainers on the release process
- mention CHANGES file modifications

* Update CHANGES file for 0.21.0 release

Also updated changes for 0.20.1, and added a small note about changes
since 0.12.0.

* btcd: bump version to v0.21.0-beta

* Handle Dash transaction version 3

* Handle new payload

* Change version field size to int16

* Include payload size and payload in serialization process

Co-authored-by: Antonin Hildebrand <antonin@hildebrand.cz>
Co-authored-by: Dan Cline <kidscline01@gmail.com>
Co-authored-by: David Hill <dhill@mindcry.org>
Co-authored-by: Henry <henry.wfisher@gmail.com>
Co-authored-by: Wilmer Paulino <wilmer.paulino@gmail.com>
Co-authored-by: Olaoluwa Osuntokun <laolu32@gmail.com>
Co-authored-by: JeremyRand <jeremyrand@airmail.cc>
Co-authored-by: Torkel Rogstad <torkel@rogstad.io>
Co-authored-by: Mikael Lindlof <mikuz.dev@gmail.com>
Co-authored-by: adiabat <rx@awsomnet.org>
Co-authored-by: Federico Bond <federicobond@gmail.com>
Co-authored-by: Anirudha Bose <anirudha.bose@ledger.fr>
Co-authored-by: Javed Khan <tuxcanfly@gmail.com>
Co-authored-by: Anirudha Bose <anirudha.bose@alumni.cern>
Co-authored-by: Oliver Gugger <gugger@gmail.com>
Co-authored-by: qqjettkgjzhxmwj <37233887+JettScythe@users.noreply.github.com>
Co-authored-by: Dan Cline <dan@dancline.net>
Co-authored-by: John C. Vernaleo <jcv@netpurgatory.com>
Co-authored-by: Mariusz Nowostawski <nowostawski@gmail.com>

.github/workflows/go.yml

@@ -0,0 +1,27 @@
+name: Build and Test
+on: [push, pull_request]
+jobs:
+  build:
+    name: Go CI
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        go: [1.13, 1.14]
+    steps:
+      - name: Set up Go
+        uses: actions/setup-go@v2
+        with:
+          go-version: ${{ matrix.go }}
+      - name: Check out source
+        uses: actions/checkout@v2
+      - name: Install Linters
+        run: "curl -sfL https://install.goreleaser.com/github.com/golangci/golangci-lint.sh | sh -s -- -b $(go env GOPATH)/bin v1.26.0"
+      - name: Build
+        env:
+          GO111MODULE: "on"
+        run: go build ./...
+      - name: Test
+        env:
+          GO111MODULE: "on"
+        run: |
+          sh ./goclean.sh

CHANGES

@@ -3,6 +3,161 @@ User visible changes for btcd
   A full-node bitcoin implementation written in Go
 ============================================================================
 
+Changes in 0.21.0 (Thu Aug 27 2020)
+  - Network-related changes:
+    - Handle notfound messages from peers in netsync package (#1603)
+  - RPC changes:
+    - Add compatibility for getblock RPC changes in bitcoind 0.15.0 (#1529)
+    - Add new optional Params field to rpcclient.ConnConfig (#1467)
+    - Add new error code ErrRPCInWarmup in btcjson (#1541)
+    - Add compatibility for changes to getmempoolentry response in bitcoind
+      0.19.0 (#1524)
+    - Add rpcclient methods for estimatesmartfee and generatetoaddress
+      commands (#1500)
+    - Add rpcclient method for getblockstats command (#1500)
+    - Parse serialized transaction from createrawtransaction command using
+      both segwit, and legacy format (#1502)
+    - Support cookie-based authentication in rpcclient (#1460)
+    - Add rpcclient method for getchaintxstats command (#1571)
+    - Add rpcclient method for fundrawtransaction command (#1553)
+    - Add rpcclient method for getbalances command (#1595)
+    - Add new method rpcclient.GetTransactionWatchOnly (#1592)
+  - Crypto changes:
+    - Fix panic in fieldVal.SetByteSlice when called with large values, and
+      improve the method to be 35% faster (#1602)
+  - btcctl changes:
+    - Added -regtest mode to btcctl (#1556)
+  - Misc changes:
+    - Fix a bug due to a deadlock in connmgr's dynamic ban scoring (#1509)
+    - Add blockchain.NewUtxoEntry() to directly create entries for
+      UtxoViewpoint (#1588)
+    - Replace LRU cache implementation in peer package with a generic one
+      from decred/dcrd (#1599)
+  - Contributors (alphabetical order):
+    - Anirudha Bose
+    - Antonin Hildebrand
+    - Dan Cline
+    - Daniel McNally
+    - David Hill
+    - Federico Bond
+    - George Tankersley
+    - Henry
+    - Henry Harder
+    - Iskander Sharipov
+    - Ivan Kuznetsov
+    - Jake Sylvestre
+    - Javed Khan
+    - JeremyRand
+    - Jin
+    - John C. Vernaleo
+    - Kulpreet Singh
+    - Mikael Lindlof
+    - Murray Nesbitt
+    - Nisen
+    - Olaoluwa Osuntokun
+    - Oliver Gugger
+    - Steven Roose
+    - Torkel Rogstad
+    - Tyler Chambers
+    - Wilmer Paulino
+    - Yash Bhutwala
+    - adiabat
+    - jalavosus
+    - mohanson
+    - qqjettkgjzhxmwj
+    - qshuai
+    - shuai.qi
+    - tpkeeper
+
+Changes in v0.20.1 (Wed Nov 13 2019)
+  - RPC changes:
+    - Add compatibility for bitcoind v0.19.0 in rpcclient and btcjson
+      packages (#1484)
+  - Contributors (alphabetical order):
+    - Eugene Zeigel
+    - Olaoluwa Osuntokun
+    - Wilmer Paulino
+
+Changes in v0.20.0 (Tue Oct 15 2019)
+  - Significant changes made since 0.12.0. See git log or refer to release
+    notes on GitHub for full details.
+  - Contributors (alphabetical order):
+    - Albert Puigsech Galicia
+    - Alex Akselrod
+    - Alex Bosworth
+    - Alex Manuskin
+    - Alok Menghrajani
+    - Anatoli Babenia
+    - Andy Weidenbaum
+    - Calvin McAnarney
+    - Chris Martin
+    - Chris Pacia
+    - Chris Shepherd
+    - Conner Fromknecht
+    - Craig Sturdy
+    - Cédric Félizard
+    - Daniel Krawisz
+    - Daniel Martí
+    - Daniel McNally
+    - Dario Nieuwenhuis
+    - Dave Collins
+    - David Hill
+    - David de Kloet
+    - GeertJohan
+    - Grace Noah
+    - Gregory Trubetskoy
+    - Hector Jusforgues
+    - Iskander (Alex) Sharipov
+    - Janus Troelsen
+    - Jasper
+    - Javed Khan
+    - Jeremiah Goyette
+    - Jim Posen
+    - Jimmy Song
+    - Johan T. Halseth
+    - John C. Vernaleo
+    - Jonathan Gillham
+    - Josh Rickmar
+    - Jon Underwood
+    - Jonathan Zeppettini
+    - Jouke Hofman
+    - Julian Meyer
+    - Kai
+    - Kamil Slowikowski
+    - Kefkius
+    - Leonardo Lazzaro
+    - Marco Peereboom
+    - Marko Bencun
+    - Mawueli Kofi Adzoe
+    - Michail Kargakis
+    - Mitchell Paull
+    - Nathan Bass
+    - Nicola 'tekNico' Larosa
+    - Olaoluwa Osuntokun
+    - Pedro Martelletto
+    - Ricardo Velhote
+    - Roei Erez
+    - Ruben de Vries
+    - Rune T. Aune
+    - Sad Pencil
+    - Shuai Qi
+    - Steven Roose
+    - Tadge Dryja
+    - Tibor Bősze
+    - Tomás Senart
+    - Tzu-Jung Lee
+    - Vadym Popov
+    - Waldir Pimenta
+    - Wilmer Paulino
+    - benma
+    - danda
+    - dskloet
+    - esemplastic
+    - jadeblaquiere
+    - nakagawa
+    - preminem
+    - qshuai
+
 Changes in 0.12.0 (Fri Nov 20 2015)
   - Protocol and network related changes:
     - Add a new checkpoint at block height 382320 (#555)

README.md

@@ -1,9 +1,9 @@
 btcd
 ====
 
-[![Build Status](https://travis-ci.org/btcsuite/btcd.png?branch=master)](https://travis-ci.org/btcsuite/btcd)
-[![ISC License](http://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org)
-[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](http://godoc.org/github.com/btcsuite/btcd)
+[![Build Status](https://github.com/btcsuite/btcd/workflows/Build%20and%20Test/badge.svg)](https://github.com/btcsuite/btcd/actions)
+[![ISC License](https://img.shields.io/badge/license-ISC-blue.svg)](http://copyfree.org)
+[![GoDoc](https://img.shields.io/badge/godoc-reference-blue.svg)](https://godoc.org/github.com/btcsuite/btcd)
 
 btcd is an alternative full node bitcoin implementation written in Go (golang).
 

blockchain/chainio.go

@@ -1149,18 +1149,9 @@ func (b *BlockChain) initChainState() error {
 
 		blockIndexBucket := dbTx.Metadata().Bucket(blockIndexBucketName)
 
-		// Determine how many blocks will be loaded into the index so we can
-		// allocate the right amount.
-		var blockCount int32
-		cursor := blockIndexBucket.Cursor()
-		for ok := cursor.First(); ok; ok = cursor.Next() {
-			blockCount++
-		}
-		blockNodes := make([]blockNode, blockCount)
-
 		var i int32
 		var lastNode *blockNode
-		cursor = blockIndexBucket.Cursor()
+		cursor := blockIndexBucket.Cursor()
 		for ok := cursor.First(); ok; ok = cursor.Next() {
 			header, status, err := deserializeBlockRow(cursor.Value())
 			if err != nil {
@@ -1193,7 +1184,7 @@ func (b *BlockChain) initChainState() error {
 
 			// Initialize the block node for the block, connect it,
 			// and add it to the block index.
-			node := &blockNodes[i]
+			node := new(blockNode)
 			initBlockNode(node, header, parent)
 			node.status = status
 			b.index.addNode(node)

blockchain/difficulty_test.go

@@ -63,7 +63,7 @@ func TestCalcWork(t *testing.T) {
 	}
 
 	for x, test := range tests {
-		bits := uint32(test.in)
+		bits := test.in
 
 		r := CalcWork(bits)
 		if r.Int64() != test.out {

blockchain/indexers/addrindex_test.go

@@ -68,7 +68,7 @@ func (b *addrIndexBucket) printLevels(addrKey [addrKeySize]byte) string {
 		if !bytes.Equal(k[:levelOffset], addrKey[:]) {
 			continue
 		}
-		level := uint8(k[levelOffset])
+		level := k[levelOffset]
 		if level > highestLevel {
 			highestLevel = level
 		}
@@ -105,7 +105,7 @@ func (b *addrIndexBucket) sanityCheck(addrKey [addrKeySize]byte, expectedTotal i
 		if !bytes.Equal(k[:levelOffset], addrKey[:]) {
 			continue
 		}
-		level := uint8(k[levelOffset])
+		level := k[levelOffset]
 		if level > highestLevel {
 			highestLevel = level
 		}

blockchain/utxoviewpoint.go

@@ -111,6 +111,22 @@ func (entry *UtxoEntry) Clone() *UtxoEntry {
 	}
 }
 
+// NewUtxoEntry returns a new UtxoEntry built from the arguments.
+func NewUtxoEntry(
+	txOut *wire.TxOut, blockHeight int32, isCoinbase bool) *UtxoEntry {
+	var cbFlag txoFlags
+	if isCoinbase {
+		cbFlag |= tfCoinBase
+	}
+
+	return &UtxoEntry{
+		amount:      txOut.Value,
+		pkScript:    txOut.PkScript,
+		blockHeight: blockHeight,
+		packedFlags: cbFlag,
+	}
+}
+
 // UtxoViewpoint represents a view into the set of unspent transaction outputs
 // from a specific point of view in the chain.  For example, it could be for
 // the end of the main chain, some point in the history of the main chain, or

btcec/field.go

@@ -226,20 +226,24 @@ func (f *fieldVal) SetBytes(b *[32]byte) *fieldVal {
 	return f
 }
 
-// SetByteSlice packs the passed big-endian value into the internal field value
-// representation.  Only the first 32-bytes are used.  As a result, it is up to
-// the caller to ensure numbers of the appropriate size are used or the value
-// will be truncated.
+// SetByteSlice interprets the provided slice as a 256-bit big-endian unsigned
+// integer (meaning it is truncated to the first 32 bytes), packs it into the
+// internal field value representation, and returns the updated field value.
+//
+// Note that since passing a slice with more than 32 bytes is truncated, it is
+// possible that the truncated value is less than the field prime.  It is up to
+// the caller to decide whether it needs to provide numbers of the appropriate
+// size or if it is acceptable to use this function with the described
+// truncation behavior.
 //
 // The field value is returned to support chaining.  This enables syntax like:
 // f := new(fieldVal).SetByteSlice(byteSlice)
 func (f *fieldVal) SetByteSlice(b []byte) *fieldVal {
 	var b32 [32]byte
-	for i := 0; i < len(b); i++ {
-		if i < 32 {
-			b32[i+(32-len(b))] = b[i]
-		}
+	if len(b) > 32 {
+		b = b[:32]
 	}
+	copy(b32[32-len(b):], b)
 	return f.SetBytes(&b32)
 }