Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fatal error when RPC node cannot be reached #7114

Closed
palango opened this issue Jun 1, 2021 · 1 comment
Closed

Fatal error when RPC node cannot be reached #7114

palango opened this issue Jun 1, 2021 · 1 comment
Projects
Raiden Berlin Sprint

Comments

@palango
Copy link
Contributor

palango commented Jun 1, 2021

Problem Definition

When the connection to the RPC node fails, Raiden crashes with an ugly stack trace instead of handling the error gracefully.

Traceback (most recent call last):
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/urllib3/connectionpool.py", line 416, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/opt/python@3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1369, in getresponse
    response.begin()
  File "/usr/local/opt/python@3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 310, in begin
    version, status, reason = self._read_status()
  File "/usr/local/opt/python@3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 271, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/opt/python@3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/gevent/_socketcommon.py", line 688, in recv_into
    return self._sock.recv_into(*args)
ConnectionResetError: [Errno 54] Connection reset by peer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/urllib3/connectionpool.py", line 720, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/urllib3/util/retry.py", line 400, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/urllib3/packages/six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/urllib3/connectionpool.py", line 672, in urlopen
    chunked=chunked,
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/urllib3/connectionpool.py", line 421, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/urllib3/connectionpool.py", line 416, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/local/opt/python@3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1369, in getresponse
    response.begin()
  File "/usr/local/opt/python@3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 310, in begin
    version, status, reason = self._read_status()
  File "/usr/local/opt/python@3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 271, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/local/opt/python@3.7/Frameworks/Python.framework/Versions/3.7/lib/python3.7/socket.py", line 589, in readinto
    return self._sock.recv_into(b)
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/gevent/_socketcommon.py", line 688, in recv_into
    return self._sock.recv_into(*args)
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/raiden/tasks.py", line 155, in check_chain_id
    current_id = web3.eth.chain_id
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/web3/eth.py", line 228, in chain_id
    return self._chain_id()
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/web3/module.py", line 57, in caller
    result = w3.manager.request_blocking(method_str, params, error_formatters)
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/web3/manager.py", line 154, in request_blocking
    response = self._make_request(method, params)
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/web3/manager.py", line 133, in _make_request
    return request_func(method, params)
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/web3/middleware/cache.py", line 134, in middleware
    return make_request(method, params)
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/web3/middleware/cache.py", line 134, in middleware
    return make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/web3/middleware/formatting.py", line 76, in apply_formatters
    response = make_request(method, params)
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/web3/middleware/gas_price_strategy.py", line 34, in middleware
    return make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/web3/middleware/formatting.py", line 76, in apply_formatters
    response = make_request(method, params)
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/web3/middleware/attrdict.py", line 33, in middleware
    response = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/web3/middleware/formatting.py", line 76, in apply_formatters
    response = make_request(method, params)
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/web3/middleware/normalize_errors.py", line 25, in middleware
    result = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/web3/middleware/formatting.py", line 76, in apply_formatters
    response = make_request(method, params)
  File "cytoolz/functoolz.pyx", line 250, in cytoolz.functoolz.curry.__call__
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/web3/middleware/formatting.py", line 76, in apply_formatters
    response = make_request(method, params)
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/raiden/network/rpc/client.py", line 708, in middleware
    response = make_request(method, params)
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/web3/middleware/exception_retry_request.py", line 113, in middleware
    return make_request(method, params)
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/web3/providers/rpc.py", line 95, in make_request
    **self.get_request_kwargs()
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/web3/_utils/request.py", line 39, in make_post_request
    response = session.post(endpoint_uri, data=data, *args, **kwargs)  # type: ignore
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/requests/sessions.py", line 590, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionResetError(54, 'Connection reset by peer'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/raiden/ui/cli.py", line 723, in _run
    run_services(kwargs)
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/raiden/ui/runners.py", line 105, in run_services
    raise_error=True,
  File "src/gevent/greenlet.py", line 1057, in gevent._gevent_cgreenlet.joinall
  File "src/gevent/greenlet.py", line 1073, in gevent._gevent_cgreenlet.joinall
  File "src/gevent/greenlet.py", line 371, in gevent._gevent_cgreenlet.Greenlet._raise_exception
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/gevent/_compat.py", line 65, in reraise
    raise value.with_traceback(tb)
  File "src/gevent/greenlet.py", line 906, in gevent._gevent_cgreenlet.Greenlet.run
  File "/Users/paul/Projects/brainbot/raiden/venv-test/lib/python3.7/site-packages/raiden/tasks.py", line 158, in check_chain_id
    "Could not reach ethereum RPC. "
RuntimeError: Could not reach ethereum RPC. Please check that your ethereum node is running and accessible.
FATAL: An unexpected exception occurred. A traceback has been written to /var/folders/g4/rq2cz9m969x4zy_9x04bdv9r0000gn/T/raiden-exception-2021-06-01T10-137gt_qsyi.txt
Could not reach ethereum RPC. Please check that your ethereum node is running and accessible.
ReturnCode.FATAL

System Description

Raiden version 2.0.0rc2
@fredo fredo added this to Mainnet Release Issues in Raiden Berlin Sprint Jun 9, 2021
@fredo fredo moved this from Mainnet Release Issues to Sprint Backlog in Raiden Berlin Sprint Jun 9, 2021
@fredo
Copy link
Contributor

fredo commented Jun 24, 2021

closed in favor of #7186

@fredo fredo closed this as completed Jun 24, 2021
Raiden Berlin Sprint automation moved this from Sprint Backlog to Done Jun 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
Raiden Berlin Sprint
  
Done
Milestone
No milestone
Development

No branches or pull requests
2 participants
@palango @fredo