MB-20493 cbdatasource: refresh cluster on memcached auth error

Found in a test case, during a swap rebalance, a cbdatasource worker
would continue to try to talk to a node that was swap-rebalanced out,
receiving an authentication error from cbauth.  This change propagates
that as an asynchronous refresh cluster kick event, which forces a new
cluster map to be retrieved, which will then shut down that worker.

Change-Id: I9ed21a8ae2ef2a628d5fe5edbbfe266ca0768513
Reviewed-on: http://review.couchbase.org/71197
Reviewed-by: Marty Schoch <marty.schoch@gmail.com>
Tested-by: Steve Yen <steve.yen@gmail.com>

cbdatasource/cbdatasource.go

@@ -833,6 +833,11 @@ func (d *bucketDataSource) worker(server string, workerCh chan []uint16) int {
 				d.receiver.OnError(fmt.Errorf("worker auth,"+
 					" AuthenticateMemcachedConn, server: %s, err: %v",
 					server, err))
+
+				// If we can't authenticate, then maybe a node was rebalanced out,
+				// so ask for a cluster refresh just in case.
+				d.Kick("worker-auth-AuthenticateMemcachedConn")
+
 				return 0
 			}
 			atomic.AddUint64(&d.stats.TotWorkerAuthenticateMemcachedConnOk, 1)