[FAB-10714] fix data race on grpc.EnableTracing

Setting the global variable inside of a connection factory does not
provide any guarantees that some other component in the process has not
already created and/or established gRPC sessions. This has resulted in
data races in the tests.

Moving the code to the peer initialization where it should be safer to
set.

Change-Id: Ifa6c87c88d1336ceea72755db85de19774ba8082
Signed-off-by: Matthew Sykes <sykesmat@us.ibm.com>

Author: sykesm

core/deliverservice/deliveryclient.go

@@ -262,7 +262,6 @@ func DefaultConnectionFactory(channelID string) func(endpoint string) (*grpc.Cli
 		} else {
 			dialOpts = append(dialOpts, grpc.WithInsecure())
 		}
-		grpc.EnableTracing = true
 		ctx := context.Background()
 		ctx, _ = context.WithTimeout(ctx, getConnectionTimeout())
 		return grpc.DialContext(ctx, endpoint, dialOpts...)

peer/node/start.go

@@ -134,6 +134,11 @@ func serve(args []string) error {
 		}
 	}
 
+	// Trace RPCs with the golang.org/x/net/trace package. This was moved out of
+	// the deliver service connection factory as it has process wide implications
+	// and was racy with respect to initialization of gRPC clients and servers.
+	grpc.EnableTracing = true
+
 	flogging.SetPeerStartupModulesMap()
 
 	logger.Infof("Starting %s", version.GetInfo())