fix exit_backend() race condition

[inimino]

Adds an exception type that was missing from exit_backend() (found by API fuzz testing by tests/0040*).

[inimino]

I pushed another change to this same branch since I was here, this takes away the printing of error_msg to stdout from the Python SDK. Given a typical usage, I think this would be more confusing than helpful (I was confused by it myself already).

There might be some more Pythonic way to alert users to errors rather than requiring them to check error_msg, but that definitely wasn't it.

[ryttry]

When, or in which test cases under which scenarios, does IncompleteRead occur?

[inimino]

If the remote end goes away before reading any bytes, it will be RemoteDisconnected, after reading HTTP headers (incuding Content-Length) it will be IncompleteRead if fewer bytes are then sent.

[ryttry]

Which test cases, at which point in the test suite, or at what test conditions, are you meeting IncompleteRead that you regard as expected and therefore not raising its exception?

I think it's important we raise the exception, save areas that we know it will happen in because of known reasons that we accept, because we should almost always expect not having IncompleteRead while testing.

This ensures unexpected errors don't get shielded from the radar.

[inimino]

Did you notice that we are in the context of the exit_backend() method here?

This function exits the backend. Note that the API call will usually fail
because the backend exits before returning a HTTP response so we suppress
the error.

The user sends a request to the backend that says "end the process and free all resources" and the backend frees all resources instantly. The backend will not spend any more resources on any outstanding HTTP requests, including this one. The user would never possibly care about or want to see these errors in any possible scenario, so we suppress them.

[inimino]

Hey, I'm going to merge this as the other PR is also landing today. I'm not sure my explanation was sufficient here, but let's find a time to chat about this!

[ryttry]

Adds an exception type that was missing from exit_backend() (found by API fuzz testing by tests/0040*).

Where would I be able to find this test?

[inimino]

Adds an exception type that was missing from exit_backend() (found by API fuzz testing by tests/0040*).

Where would I be able to find this test?

https://github.com/Watchfulio/watchful/pull/1823

[inimino]

To observe this you could start the backend and exit it repeatedly a few thousand times. Most of the time you would get the "RemoteDisconnected" error but a few times per thousand you might get the "IncompleteRead" error, because the process teardown happened to occur in the middle of the bytes of the HTTP response being sent over the wire. (This is what I meant by calling it a race condition).

[inimino]

e.g. you could run something like this:

import etc etc

while True:
    backend_for_tests()
    client.exit_backend()

Remove the try/except from the exit_backend() function and run the above to see what I'm talking about.