create regression test dirs

Fixes FAB-1369.
Regression tests have been developed in multiple languages
for blackbox testing of various components of hyperledger
fabric. The intent is to soon place many tests scripts and
supporting infrastructure under folder fabric/bddtests/regression/
to store them close to the fabric code with which they correlate.
Next we will be constructing jenkins automation test suites and
run them and post results for the community.

Change-Id: Ie78ddd833e3c9792ff622847c9195703bbe10222
Signed-off-by: Scott Zwierzynski <scottz@us.ibm.com>

bddtests/regression/README.md

@@ -0,0 +1,87 @@
+## regression
+Regression Test Suites scripts, and folders for: execution results for
+significant releases, and
+configuration scripts and supporting files in a variety of languages:
+
+- [GO](https://github.com/hyperledger/fabric/bddtests/regression/go)
+  language test scripts
+- [NODE](https://github.com/hyperledger/fabric/bddtests/regression/node)
+  language test scripts
+  Note: a flexible-use performance engine, which can be configured and used
+  for a variety of tests, is located in the
+  [node/performance](https://github.com/hyperledger/fabric/bddtests/regression/node/performance)
+  folder.  For examples how to use it, check there, or look for examples in the
+  daily test suite and long-run test suite scripts.
+- [results](https://github.com/hyperledger/fabric/bddtests/regression/results)
+  folder containing logfiles and results of running the Test Suites on
+  significant releases, for reference
+
+
+### Continous Integration Setup
+Continuous Integration team will execute the *daily test suite* each day when a
+merge commit has been pushed to fabric repository.
+We have configured Jenkins in vLaunch to execute the Daily Test Suite:
+the jobs listed below are downstream jobs which are run after successfully
+executing the upstream job.
+After the build is successfully executed, Jenkins will post build results
+back to **rel-criteria-build** slack channel,
+and generate a test summary report, to be available for viewing for 30 days.
+For significant releases, the results will be stored
+in **bddtests/regression/results** folder.
+
+
+### Daily Test Suite - daily_test_suite.sh
+Expected total time duration is between 13 - 16 hours.
+
+* Consensus Acceptance Tests (CAT) - using chaincode example02, GO, gRPC
+  - Objective of CAT tests is to ensure the stability and resiliency of the
+    BFT Batch design.
+* Large Networks Basic API And Consensus Tests - using chaincode example02,
+  GO, gRPC
+  - Objective of Basic gRPC API test is to ensure basic gRPC API functions
+    are working as expected
+* Ledger Stress Tests (LST) for API and 20K runs with concurrency and
+  1K payload - using chaincode addrecs, GO, gRPC
+* Speed Tests for measuring turnaround performance of the communication path
+  and blockchain network processing - using chaincode addrecs, GO, gRPC
+* Concurrency Tests - using chaincode addrecs, GO, gRPC
+  - Objective of Concurrency test is to ensure system is able to accept
+    multiple threads concurrently for specified number of mins.
+* Complex Transactions Tests - using chaincode auction, node, gRPC
+  - Objective of this test is to send complex transactions using
+    auction chaincode
+* Performance Tests, random sized payloads - using chaincodes example02
+  and auction, node, gRPC
+  - CI performs below tests in Performance testing
+     * Invoke on chaincode_example02 using 4 peers and
+       4 Threads for 180 secs
+     * Query on chaincode_example02 using 4 peers and
+       4 Threads for 180 secs
+     * Invoke on auction chaincode using 4 peers and
+       1000 Tx for each of the 4 Threads
+     * Query on auction chaincode using 4 peers and
+       1000 Tx for each of the 4 Threads
+     * Invoke on auction chaincode using 4 peers and 4 Threads.
+       Each Invoke is followed by a Query on every Thread
+     * Invoke on example02 chaincode using 4 peers and 4 Threads.
+Each Invoke is followed by a Query on every Thread
+
+
+### LongRun Test Suite - longrun_test_suite.sh
+Expected total time duration is 72 hours when executed from
+Jenkins automation scripts, which run several parallel jobs.
+
+* Consensus peers restarts tests, 2 tests, each approximately 10 hours -
+  using chaincode example02, GO, gRPC
+* Ledger Stress Test 1 Million transactions, approximately 36 hours -
+  using chaincode addrecs, GO, gRPC
+* Ledger Stress Test, 72 hours - using chaincode addrecs, GO, gRPC
+  - Send parallel Invoke requests on all 4 peers
+* Performance Tests, 72 hours, random sized payloads - using chaincode auction,
+  node, gRPC (includes variable traffic, concurrency, complex transactions)
+  - Invoke 1 Tx per sec on 1 peer for 72 hrs (1 Thread)
+
+
+### Full Regression Test Suite
+Execute both the Daily Test Suite AND the Long Run Test Suite.
+

bddtests/regression/daily_test_suite.sh

@@ -0,0 +1,130 @@
+#!/bin/bash
+#
+# Copyright IBM Corp. 2016 All Rights Reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# DESCRIPTION:
+# These are the tests in the Daily Test Suite.
+# To run these tests, user may clone the fabric, make code changes if
+# desired, build images, and run this script.
+
+
+# Setup
+
+WORKDIR=`pwd`
+
+
+
+
+
+# Note to test developers: Many tests in the Daily Test Suite are
+# specific tests that already contain the TEST OBJECTIVE and STEPS.
+# However, in some cases, such as running a flexible test engine to
+# perform different tests by using configuration parameters or
+# environment variables, it must be provided here.
+# For EACH of those test cases, before invoking the test, please
+# provide a comment block like this:
+
+#-----------------------------------------------------------------------
+# TEST OBJECTIVE : basic network setup, deploy, invoke, query all peers
+#                  using gRPC interface, and exercise chaincode APIs
+# SETUP STEPS:
+#   1. Set environment variable: export TEST_NETWORK=LOCAL
+# TEST DETAILS (Optional):
+#   1. Setup local docker network with security with 4 peer nodes
+#   2. Deploy chaincode example02
+#   3. Send Invoke Requests on all peers using go routines
+#   4. Verify query results and chainheights match on all peers
+#-----------------------------------------------------------------------
+
+
+
+
+
+########################################################################
+# Language        API                 Functional Area
+# GO TDK          gRPC                Consensus Acceptance Tests
+########################################################################
+
+cd $WORKDIR/go/tdk/CAT
+#UNDER CONSTRUCTION...
+#go run CAT_100_Startup_DIQ_API.go
+#go run CAT_101_BasicConsensus_S1_R1_S2_S1_R1_R2.go
+#go run CAT_102_S1_IQDQIQ.go
+
+
+
+########################################################################
+# Language        API                 Functional Area
+# GO TDK          gRPC                Transaction Speed Measurement
+########################################################################
+
+cd $WORKDIR/go/tdk/ledgerstresstest
+#./run_speed_tests.sh
+
+
+
+########################################################################
+# Language        API                 Functional Area
+# GO TDK          gRPC                Ledger Stress Tests
+########################################################################
+
+cd $WORKDIR/go/tdk/ledgerstresstest
+#go run BasicFuncExistingNetworkLST.go
+#go run LST_1client1peer20K.go
+#go run LST_2client1peer20K.go
+#go run LST_2client2peer20K.go
+#go run LST_4client1peer20K.go
+#go run LST_4client4peer20K.go
+
+
+
+########################################################################
+# Language        API                 Functional Area
+# GO TDK          gRPC                Concurrency/Ledger
+########################################################################
+
+cd $WORKDIR/go/tdk/ledgerstresstest
+#go run conc4p1min1000Thrd1TxPerLoop_LOCAL.go
+#go run conc4p1min400Thrd_LOCAL.go
+
+
+
+########################################################################
+# Language        API                 Functional Area
+# Node.js         gRPC                Concurrency with Auction
+########################################################################
+
+cd $WORKDIR/node/
+
+
+
+########################################################################
+# Language        API                 Functional Area
+# Node.js         gRPC                Complex Transactions with Auction
+########################################################################
+
+cd $WORKDIR/node/
+
+
+
+########################################################################
+# Language        API                 Functional Area
+# Node.js         gRPC                Performance engine Baseline tests
+########################################################################
+
+cd $WORKDIR/node/performance/
+
+

bddtests/regression/go/README.md

@@ -0,0 +1,8 @@
+## GO tests, scripts, and supporting files
+(For all scripts added to this directory, add information below to
+describe pre-requisites, setup steps, and usage examples.)
+
+## SETUP
+
+## USAGE
+

bddtests/regression/go/tdk/CAT/README.md

@@ -0,0 +1,74 @@
+## Consensus Acceptance Testcases (CAT)
+Hyperledger Fabric Go language Test Development Kit (TDK)
+tests can be executed on local fabric in vagrant environment with docker
+containers.
+Additional details (the date the tests were run, commit image,
+test config parameters, the test steps, output summary and details, etc.)
+are available in the GO_TEST files in this folder.
+
+### CAT test naming convention
+
+The testnames themselves indicate the steps performed. For example:
+
+```
+    CAT_210_S2S1_IQ_R1_IQ.go    Consensus Acceptance Test (CAT) suite, testcase number 210
+           _S2S1                Stop Validating PEERs VP2 and VP1 at virtually the same time
+                _IQ             Send some Invoke requests to all running peers, and Query all to validate A/B/ChainHeight results
+                   _R1          Restart VP1
+                      _IQ       Send some Invoke requests to all running peers, and Query all to validate A/B/ChainHeight results
+```
+
+### Test Coverage
+
+The objective of Consensus Acceptance Tests (CAT) is to ensure the
+stability and resiliency of the
+PBFT Batch design when Byzantine faults occur in a 4 peer network.
+Test areas coverage:
+
+Stop 1 peer: the network continues to process deploys, invokes, and query
+transactions.
+Perform this operation on each peer in the fabric.
+While exactly 3 peers are running, their ledgers will remain in synch.
+(In a 4 peer network, 3 represents 2F+1, the minimum number required for
+consensus.)
+Restarting a 4th peer will cause it to join in the network operations again.
+Note: when queried after having been restarted, the ledger on extra
+peers ( additional nodes beyond  2(F+1) )
+may appear to lag for some time. It will catch up if another peer is
+stopped (leaving it as one of exactly 2F+1 participating peers), OR, with
+no further disruptions in the network,
+it could catch up after huge numbers of transaction batches are processed.
+
+Stop 2 peers: the network should halt advancement, due to a lack of consensus.
+Restarting 1 or 2 peers will cause the network to resume processing
+transactions because enough nodes are available to reach consensus. This may
+include processing transactions that were received and queued by any running
+peers while consensus was halted.
+
+Stop 3 peers: the network should halt advancement due to a lack of consensus.
+Restarting just one of the peers should not resume consensus.
+Restarting 2 or 3 peers should cause the network to resume consensus.
+
+Deploys should be processed, or queued if appropriate, with any number of
+running peers.
+
+
+### RESULTS SUMMARY
+
+```
+    Consensus - Failed Testcases            Opened Bugs        Description
+
+    CAT_111_SnIQRnIQ_cycleDownLoop.go       FAB-337            dup Tx
+    CAT_303_S0S1S2_IQ_R0R1R2_IQ.go          FAB-333            lost Tx after restart 3 peers together
+    CAT_305_S1S2S3_IQ_R1R2R3_IQ.go          FAB-333            lost Tx after restart 3 peers together
+    CAT_407_S0S1S2_D_I_R0R1_IQ.go           FAB-911            lost Tx from vp0 after stop vp0/vp1/vp2, deploy, restart vp0/vp1, invokes
+    CAT_408_S0S1S2_D_I_R0R1R2_IQ.go         FAB-335            deploy failed when restart 3 peers together
+    CAT_409_S1S2S3_D_I_R1R2_IQ.go           FAB-334/FAB-912    lost Tx after all peers after stop vp1/vp2/vp3, deploy, restart vp1/vp2, invokes
+    CAT_410_S1S2S3_D_I_R1R2R3_IQ.go         FAB-335            deploy failed when restart 3 peers together
+    CAT_412_S0S1S2_D_I_R1R2_IQ.go           FAB-334/FAB-912    lost Tx after all peers after stop vp1/vp2/vp3, deploy, restart vp1/vp2, invokes
+
+    CRT_502_StopAndRestart1or2_10Hrs.go     FAB-331            acked Tx lost after stop/restart peers many times
+
+    CAT_201_S2S1_IQDIQ.go (using Pauses)    FAB-336            no http response from vp0, while peers vp1,vp2 docker paused
+```
+