Add a toxic to simulate "connection reset"

[holyjak]

A problem I experience regularly (in Java) is the java.net.SocketException: Connection reset [by peer]. It would be awesome to have a toxic that can cause it. Thanks!

Some background: https://docs.oracle.com/javase/8/docs/technotes/guides/net/articles/connection_release.html

[jpittis]

You should get that when the proxy is in a down state. slow_close might also work or does that not trigger the right kind of error?

Yeah... I'm not sure it's possible to trigger one of those through the usual socket API which means it's hard to write a toxic to do this. It's an OS thing no? I'm a little flaky on my understanding of this.

[boly38]

I'm also interested by this usecase.
It seems feasible in java following this example:

• step A) accepting a socket, and map a stream reader
• step B) brute force exit

client class connects, then sleeps two seconds before attempting to read the stream from the now defunct server socket

Then, I dont know if it's easy to translate this sample/behaviour onto custom toxiproxy toxic.
(my side I didn't success to reproduce "connection reset" -kind of TCP level error- using down, slow_close or limit_data toxics )

[holyjak]

I do not understand it deeply enough to know how to simulate it but there must be a way to do something at the other end (toxiproxy) here to simulate "connection reset by peer" .

@boly38 What you describe is I think the other case, when the connection is not reset by the peer but within the JVM itself due to trying to read from a closed socket or something like that.

[jpittis]

Alrighty. I was wrong. We can simulate a connection reset by peer from an application.

Here's a Go script I wrote that triggers a connection reset by peer using SetLinger(0).

Link to executable Gist.

Running the experiment looks like this:

$ go run experiment.go
2019/01/02 08:29:36 connect err: EOF
exit status 1
$ go run experiment.go rst
2019/01/02 08:29:40 connect err: read tcp 127.0.0.1:50361->127.0.0.1:4444: read: connection reset by peer
exit status 1

This sounds like something we could for sure support in Toxiproxy.

[jpittis]

I don't have time to work on this. I may have time to suggest a way to implement it.

Until then, my only thoughts are that we'd need to figure out how to trigger a SetLinger(0) directly on the connecting client before close.

• I'm pretty sure the client is closed here.
• What's hard about this problem is toxics don't currently have direct access to the socket layer of abstraction so we'd need to find a way to properly abstract SoLinger before close.

[chaosbox]

@jpittis
Created a PR to add this toxic, when you get a chance let us know what you think.

#247

[miry]

Released #247 in https://github.com/Shopify/toxiproxy/releases/tag/v2.2.0