ensure responseChannel is drained

[koonpeng]

There is a potential for deadlocks

core/src/go/cmd/http-relay-client/client/client.go

Lines 708 to 712 in f173d97

	if err != nil {
	slog.Error("Closing backend connection",
	slog.String("ID", *resp.Id), ilog.Err(err))
	break
	}

if we exit the loop preemptively, we fail to fully consume responseChannel and

core/src/go/cmd/http-relay-client/client/client.go

Line 487 in f173d97

out <- resp

might deadlock.

#222 fixes the error handling which may have caused the deadlock. Before, it checks for incorrect backoff.PermanentError which never happens so the responseChannel is always fully consumed.

[ensonic]

Can you add more detail for which transport this Close() call would be needed?

[koonpeng]

The "default" http client doesn't panic. I didn't really tested if it would panic in our case.
In theory, we can also not close here. The server should eventually close the connection, then streamBackend will end up closing hresp.Body. Closing it here would just propagate the disconnect to the backend faster.

[ensonic]

Thanks maybe reword as:
// This is also closed in streamToBackend. Closing here too to ensure the disconnect propagates faster in case e.g. the default http client is used.

[ensonic]

Otherwise, thanks & +1