[ASHF-35] Update base (#23)

* [FAB-17819] Discovery returns user friendly errors

Currently, the discovery service filters out peers that don't have the chaincode installed
early on in the computation, and as a result - the service cannot distinguish from a case
where there are not enough alive peers to satisfy the endorsement policy, or that there are
enough peers but the chaincode is not installed on enough of them.

This change set defers the chaincode filtering to the end of the computation, so
the layouts and peer group mapping is creating without taking into account if the peers
have the chaincode installed on them, and if there is no layout that can be satisfied
without taking into account the chaincodes - the error that is returned now
is "no peer combination can satisfy the endorsement policy",
instead of "cannot satisfy any principal combination".

Afterwards, the layouts are being inspected once again, and then the layouts
that cannot be satisfied are filtered out, when the error returned
when no layout can be satisfied is now: "required chaincodes are not installed on sufficient peers".

Change-Id: I74eb29b30aec1a87842d220414c73872cdbc8304
Signed-off-by: yacovm <yacovm@il.ibm.com>

* Fix docker network leak from RAFT integration test (hyperledger#1203)

Signed-off-by: Matthew Sykes <matthew.sykes@gmail.com>

* [FAB-17786] Update upgrade_dbs peer command to drop state couchdb (hyperledger#1187)

The existing upgrade_dbs command does not automatically drop state couchdbs
and therefore a separate step is required to drop couchdbs, This PR updates
the command to automatically drop state couchdbs.

In addition, it checks upgrade eligibility before upgrade so that it will not
drop databases if it is already the expected format.

Signed-off-by: Wenjian Qiao <wenjianq@gmail.com>

* [FAB-17774] support orderer restart without genesis block (hyperledger#1197)

* adding support for orderer restart without genesis block.

Signed-off-by: Chongxin Luo <Chongxin.Luo@ibm.com>

* Remove interface for block storage

- Removed the interface for better code navigation, as There is only a
 single implementation for block storage
- Merged the remaining code in the package blkstorgae with the
implementation in the package fsblkstorage and used the name
blkstorgae for the final package
- Moved the internal single proto message with in same package

Signed-off-by: manish <manish.sethi@gmail.com>

* [LEDGER] Move UpdatesBytesBuilder to txmgr pkg (hyperledger#1209)

- mv updateBatch bytes constructor to txmgr pkg
Currently, we create the bytes representation of updateBatch
to compute the hash of updateBatch which would be included in
the block along with the validation results. The utility for
converting a updateBatch to a deterministic bytes is present
in the privacyenabledstate pkg. However, this utility is
used only by the txmgr and hence, we move the utility
function to txmgr pkg from privacyenabledstate pkg.

- rename proto messages
In the updateBatch proto, we have defined proto messages such
as KVWriteProto and KVWriteBatchProto. As we shouldn't append
the keyword proto to messages, we rename KVWriteProto to KVWrite
and KVWriteBatchProto to Updates.

- rename function names
The function name such as buildForKeys(), buildForColls() are not
very explicit in what they do. When the buildForColls() calls
buildForKeys(), it adds even more confusion as the first-level
function, i.e., deterministicBytesForPubAndHashUpdates() is also
calling buildForKeys(). Hence, we have used the following function
names instead:
(1) genKVsFromNsUpdates()
(2) genKVsFromCollsUpdates()
(3) genKVs()

FAB-17830

Signed-off-by: senthil <cendhu@gmail.com>

* Update grpc-go to v1.29.1 (hyperledger#1213)

Signed-off-by: Gari Singh <gari.r.singh@gmail.com>

* [FAB-17831] Use generic constant/var names in dataformat.go (hyperledger#1212)

Currently Version1x and Version20 are defined in dataformat.go.
They should be renamed to more generic names such as
PreviousFormat and CurrentFormat.

The format values will change only when the data format is changed
in ledger. If a Fabric version does not introduce a new data format,
CurrentFormat will remain the same as the latest format prior to
the Fabric version.

Signed-off-by: Wenjian Qiao <wenjianq@gmail.com>

* [LEDGER] rm interface from pvtdatastorage pkg (hyperledger#1217)

rm pvtdatastorage interfaces

As we have only one implementation of the pvtdatastore and not
planning to any a new one, we can safely remove the interface.
This also helps in code navigation and to avoid type casting
such as s.(*store) in the test.

FAB-17843

Signed-off-by: senthil <cendhu@gmail.com>

* FAB-15710 Ch.Part.API: orderer config & hook into http server (hyperledger#1218)

- Expose a configuration object for channel participation at
  the top level (like Metrics). Even though the channel
  participation API shares the same endpoint and TLS config
  with operations, placing it at the top level will allow us
  to separate these APIs to different endpoints in the future
  without changing the config structure.

- Extend operations.System to be able to register a handler
  for additional APIs.

- Implement a skeleton handler for the channel participation API.

- Register the skeleton handler to the http server in
  operations.System during the server boot sequence.

Signed-off-by: Yoav Tock <tock@il.ibm.com>
Change-Id: I5cf15dffa29985cba60e5aaf31d189e755a3a1ef

* Update the vagrant dev environment

- Remove GOPATH in favor of modules
- Move to Ubuntu 20.04
- Remove docker-compose as it's unnecessary for build and test

Signed-off-by: Matthew Sykes <sykesmat@us.ibm.com>

* Add function in blockstore to export TxIDs

FAB-17837

Signed-off-by: manish <manish.sethi@gmail.com>

* reduce #arguments in a few kvledger methods (hyperledger#1210)

As the number of arguments passed to newKVLedger(),
initTxMgr() is quite high, we reduce it by introducing
initializer struct.

FAB-17683

Signed-off-by: senthil <cendhu@gmail.com>

Co-authored-by: yacovm <yacovm@il.ibm.com>
Co-authored-by: Matthew Sykes <sykesmat@us.ibm.com>
Co-authored-by: wen <wenjianq@gmail.com>
Co-authored-by: Dereck <Chongxin.Luo@ibm.com>
Co-authored-by: manish <manish.sethi@gmail.com>
Co-authored-by: Senthil Nathan N <cendhu@users.noreply.github.com>
Co-authored-by: Gari Singh <gari.r.singh@gmail.com>
Co-authored-by: Yoav Tock <tock@il.ibm.com>

common/ledger/blkstorage/fsblkstorage/block_serialization.go → common/ledger/blkstorage/block_serialization.go

@@ -4,7 +4,7 @@ Copyright IBM Corp. All Rights Reserved.
 SPDX-License-Identifier: Apache-2.0
 */
 
-package fsblkstorage
+package blkstorage
 
 import (
 	"github.com/golang/protobuf/proto"

common/ledger/blkstorage/fsblkstorage/block_serialization_test.go → common/ledger/blkstorage/block_serialization_test.go

@@ -4,7 +4,7 @@ Copyright IBM Corp. All Rights Reserved.
 SPDX-License-Identifier: Apache-2.0
 */
 
-package fsblkstorage
+package blkstorage
 
 import (
 	"testing"

common/ledger/blkstorage/fsblkstorage/block_stream.go → common/ledger/blkstorage/block_stream.go

@@ -4,7 +4,7 @@ Copyright IBM Corp. All Rights Reserved.
 SPDX-License-Identifier: Apache-2.0
 */
 
-package fsblkstorage
+package blkstorage
 
 import (
 	"bufio"

common/ledger/blkstorage/fsblkstorage/block_stream_test.go → common/ledger/blkstorage/block_stream_test.go

@@ -1,20 +1,10 @@
 /*
-Copyright IBM Corp. 2016 All Rights Reserved.
+Copyright IBM Corp. All Rights Reserved.
 
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-		 http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
+SPDX-License-Identifier: Apache-2.0
 */
 
-package fsblkstorage
+package blkstorage
 
 import (
 	"testing"

common/ledger/blkstorage/fsblkstorage/blockfile_helper.go → common/ledger/blkstorage/blockfile_helper.go

@@ -4,7 +4,7 @@ Copyright IBM Corp. All Rights Reserved.
 SPDX-License-Identifier: Apache-2.0
 */
 
-package fsblkstorage
+package blkstorage
 
 import (
 	"io/ioutil"

common/ledger/blkstorage/fsblkstorage/blockfile_helper_test.go → common/ledger/blkstorage/blockfile_helper_test.go

@@ -4,7 +4,7 @@ Copyright IBM Corp. All Rights Reserved.
 SPDX-License-Identifier: Apache-2.0
 */
 
-package fsblkstorage
+package blkstorage
 
 import (
 	"io/ioutil"
@@ -18,9 +18,8 @@ import (
 )
 
 func TestConstructCheckpointInfoFromBlockFiles(t *testing.T) {
-	testPath := "/tmp/tests/fabric/common/ledger/blkstorage/fsblkstorage"
 	ledgerid := "testLedger"
-	conf := NewConf(testPath, 0)
+	conf := NewConf(testPath(), 0)
 	blkStoreDir := conf.getLedgerBlockDir(ledgerid)
 	env := newTestEnv(t, conf)
 	util.CreateDirIfMissing(blkStoreDir)

common/ledger/blkstorage/fsblkstorage/blockfile_mgr.go → common/ledger/blkstorage/blockfile_mgr.go

@@ -4,7 +4,7 @@ Copyright IBM Corp. All Rights Reserved.
 SPDX-License-Identifier: Apache-2.0
 */
 
-package fsblkstorage
+package blkstorage
 
 import (
 	"bytes"
@@ -17,16 +17,12 @@ import (
 	"github.com/golang/protobuf/proto"
 	"github.com/hyperledger/fabric-protos-go/common"
 	"github.com/hyperledger/fabric-protos-go/peer"
-	"github.com/hyperledger/fabric/common/flogging"
-	"github.com/hyperledger/fabric/common/ledger/blkstorage"
 	"github.com/hyperledger/fabric/common/ledger/util"
 	"github.com/hyperledger/fabric/common/ledger/util/leveldbhelper"
 	"github.com/hyperledger/fabric/protoutil"
 	"github.com/pkg/errors"
 )
 
-var logger = flogging.MustGetLogger("fsblkstorage")
-
 const (
 	blockfilePrefix = "blockfile_"
 )
@@ -87,7 +83,7 @@ At start up a new manager:
 		-- If index and file system are not in sync, syncs index from the FS
   *)  Updates blockchain info used by the APIs
 */
-func newBlockfileMgr(id string, conf *Conf, indexConfig *blkstorage.IndexConfig, indexStore *leveldbhelper.DBHandle) *blockfileMgr {
+func newBlockfileMgr(id string, conf *Conf, indexConfig *IndexConfig, indexStore *leveldbhelper.DBHandle) *blockfileMgr {
 	logger.Debugf("newBlockfileMgr() initializing file-based block storage for ledger: %s ", id)
 	//Determine the root directory for the blockfile storage, if it does not exist create it
 	rootDir := conf.getLedgerBlockDir(id)
@@ -183,7 +179,7 @@ func syncCPInfoFromFS(rootDir string, cpInfo *checkpointInfo) {
 	logger.Debugf("status of file [%s]: exists=[%t], size=[%d]", filePath, exists, size)
 	//Test is !exists because when file number is first used the file does not exist yet
 	//checks that the file exists and that the size of the file is what is stored in cpinfo
-	//status of file [/tmp/tests/ledger/blkstorage/fsblkstorage/blocks/blockfile_000000]: exists=[false], size=[0]
+	//status of file [<blkstorage_location>/blocks/blockfile_000000]: exists=[false], size=[0]
 	if !exists || int(size) == cpInfo.latestFileChunksize {
 		// check point info is in sync with the file on disk
 		return

common/ledger/blkstorage/fsblkstorage/blockfile_mgr_test.go → common/ledger/blkstorage/blockfile_mgr_test.go

@@ -4,7 +4,7 @@ Copyright IBM Corp. 2016 All Rights Reserved.
 SPDX-License-Identifier: Apache-2.0
 */
 
-package fsblkstorage
+package blkstorage
 
 import (
 	"fmt"
@@ -195,9 +195,9 @@ func TestBlockfileMgrGetTxById(t *testing.T) {
 func TestBlockfileMgrGetTxByIdDuplicateTxid(t *testing.T) {
 	env := newTestEnv(t, NewConf(testPath(), 0))
 	defer env.Cleanup()
-	blkStore, err := env.provider.OpenBlockStore("testLedger")
+	blkStore, err := env.provider.Open("testLedger")
 	assert.NoError(env.t, err)
-	blkFileMgr := blkStore.(*fsBlockStore).fileMgr
+	blkFileMgr := blkStore.fileMgr
 	bg, gb := testutil.NewBlockGenerator(t, "testLedger", false)
 	assert.NoError(t, blkFileMgr.addBlock(gb))
 

common/ledger/blkstorage/fsblkstorage/blockfile_rw.go → common/ledger/blkstorage/blockfile_rw.go

@@ -4,7 +4,7 @@ Copyright IBM Corp. All Rights Reserved.
 SPDX-License-Identifier: Apache-2.0
 */
 
-package fsblkstorage
+package blkstorage
 
 import (
 	"os"

common/ledger/blkstorage/fsblkstorage/blockfile_scan_test.go → common/ledger/blkstorage/blockfile_scan_test.go

@@ -1,20 +1,10 @@
 /*
-Copyright IBM Corp. 2016 All Rights Reserved.
+Copyright IBM Corp. All Rights Reserved.
 
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-		 http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
+SPDX-License-Identifier: Apache-2.0
 */
 
-package fsblkstorage
+package blkstorage
 
 import (
 	"os"