Fixes shutdown deadlock in websocket client

[fhats-stripe]

Reviewers

r? @stripe/developer-products

Summary

This change solves a deadlock that can happen in stripe listen when the underlying websocket connection closes or is reconnecting while the websocket client is attempting to send a message. The deadlock happens when the writePump has a message to send:

	for {
		select {
		case outMsg, ok := <-c.send:

runs into an error trying to send the message because the connection has been closed for one reason or another:

			err = c.conn.WriteJSON(outMsg)
			if err != nil {

and attempts to re-queue the message it was going to process:

				// Requeue the message to be processed when writePump restarts
				c.send <- outMsg

c.send is unbuffered, and only read from in this select. Sending messages to c.send in the same block that processes reads from c.send will always deadlock, as message sending to unbuffered channels blocks until the message is read.

This PR fixes that deadlock by making c.send a buffered channel, allowing the re-queueing behavior we're looking for.

This condition is not all that common, but given stripe listen tries to reconnect every once in a while this could happen more often than you'd think in a busy stripe listen session.

This change comes with a unit test meant to provoke the regression to verify it remains fixed. That test was a little challenging to write (I'm still learning how to do these well), and to support the new test case I added some synchronization primitives in the websocket client to make it easier to monkey with the client from the test case.

[charliecruzan-stripe]

thankyou for the solid description, it makes this much easier to review!