Connection broken when calling watch()

[myakove]

Calling watch() (openshift.dynamic.client.DynamicClient#watch) fail from time to time with error urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))

[myakove]

self = <urllib3.response.HTTPResponse object at 0x7f19f6ac63c8>

@contextmanager
def _error_catcher(self):
    """
    Catch low-level python exceptions, instead re-raising urllib3
    variants, so that low-level exceptions are not leaked in the
    high-level api.

    On exit, release the connection back to the pool.
    """
    clean_exit = False

    try:
        try:
            yield

        except SocketTimeout:
            # FIXME: Ideally we'd like to include the url in the ReadTimeoutError but
            # there is yet no clean way to get at it from this context.
            raise ReadTimeoutError(self._pool, None, 'Read timed out.')

        except BaseSSLError as e:
            # FIXME: Is there a better way to differentiate between SSLErrors?
            if 'read operation timed out' not in str(e):  # Defensive:
                # This shouldn't happen but just in case we're missing an edge
                # case, let's avoid swallowing SSL errors.
                raise

            raise ReadTimeoutError(self._pool, None, 'Read timed out.')

        except (HTTPException, SocketError) as e:
            # This includes IncompleteRead.

          raise ProtocolError('Connection broken: %r' % e, e)

E urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))

[mmazur]

Is this the same bug I'm getting with OKD4? If it is, it looks like it's in urllib3's ability to parse okd4.

Traceback (most recent call last):
  File "/home/mmazur/.local/lib/python3.7/site-packages/urllib3/response.py", line 603, in _update_chunk_length
    self.chunk_left = int(line, 16)
ValueError: invalid literal for int() with base 16: b''
(…)
urllib3.exceptions.ProtocolError: ('Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read))

[fabianvf]

This is expected behavior, you need to handle the error and re-establish the watch. Related issue in the kubernetes-client (which is where this behavior is coming from): kubernetes-client/python#869

And an issue proposing a fix (by implementing higher level informers): kubernetes-client/python#868