gRPC Client creates new streams even after receiving GOAWAY frames

[yashykt]

There is a race within gRPC where the following events can happen -

1. Client starts a new RPC on a connected channel.
2. Client channel routes the RPC to the connected channel during the pick.
3. Client transport receives a GOAWAY frame from the server, and marks the channel to be in TRANSIENT_FAILURE.
4. Since we have already picked the connected channel, the client transport tries to create a new stream.

The HTTP2 spec says that clients should not start new streams if a GOAWAY has already been received. Also, if the GOAWAY frame mentioned a last_stream_id, the server is free to ignore/discard frames from the new stream. This means that the client will keep waiting for a response from the server till the deadline exceeds, even though we know that a GOAWAY has already been received.