You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
But I don't think we can solve the problem like this in general.
Expected behavior
The workflow should run whether behind a proxy or not.
To Reproduce
Steps to reproduce the behavior:
Be behind a proxy :)
Try to create a new Neural Network (Local) project
(I will have to try if I can replicate the setup to actually reproduce the problem)
Error message/traceback
Traceback (most recent call last):
File “C:\Program Files\ilastik-1.4.0-gpu\lib\site-packages\lazyflow\operator.py”, line 117, in call
instance = ABCMeta.call(cls, *args, **kwargs)
File “C:\Program Files\ilastik-1.4.0-gpu\lib\site-packages\ilastik\workflows\neuralNetwork_localWorkflow.py”, line 53, in init
super().init(shell, headless, workflow_cmdline_args, project_creation_args, *args, **kwargs)
File “C:\Program Files\ilastik-1.4.0-gpu\lib\site-packages\ilastik\workflows\neuralNetwork_nnWorkflowBase.py”, line 91, in init
self._createClassifierApplet(headless=self._headless, conn_str=connection_string)
File “C:\Program Files\ilastik-1.4.0-gpu\lib\site-packages\ilastik\workflows\neuralNetwork_localWorkflow.py”, line 67, in _createClassifierApplet
preferred_cuda_device_id, device_name = super()._configure_device(conn)
File “C:\Program Files\ilastik-1.4.0-gpu\lib\site-packages\ilastik\workflows\neuralNetwork_nnWorkflowBase.py”, line 245, in _configure_device
devices = conn.get_devices()
File “C:\Program Files\ilastik-1.4.0-gpu\lib\site-packages\lazyflow\operators\tiktorch\classifier.py”, line 355, in get_devices
resp = self._client.ListDevices(inference_pb2.Empty())
File “C:\Program Files\ilastik-1.4.0-gpu\lib\site-packages\grpc_channel.py”, line 946, in call
return _end_unary_response_blocking(state, call, False, None)
File “C:\Program Files\ilastik-1.4.0-gpu\lib\site-packages\grpc_channel.py”, line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = “failed to connect to all addresses”
debug_error_string = “{“created”:”@1683600100.468000000",“description”:“Failed to pick subchannel”,“file”:“src/core/ext/filters/client_channel/client_channel.cc”,“file_line”:3159,“referenced_errors”:[{“created”:“@1683600100.468000000”,“description”:“failed to connect to all addresses”,“file”:“src/core/lib/transport/error_utils.cc”,“file_line”:147,“grpc_status”:14}]}"
With GRPC_TRACE=all and GRPC_VERBOSITY=debug, we receive additional logs:
I0630 10:19:15.338000000 10824 src/core/ext/filters/client_channel/http_connect_handshaker.cc:333] Connecting to server 127.0.0.1:62817 via HTTP proxy ipv4:<proxy IP>:8080
(...)
I0630 10:19:15.355000000 10824 src/core/lib/channel/handshaker.cc:89] handshake_manager 0000024388134E10: error={"created":"@1688087955.355000000","description":"HTTP proxy returned response code 403","file":"src/core/ext/filters/client_channel/http_connect_handshaker.cc","file_line":253} shutdown=0 index=1, args={endpoint=(nil), args=(nil) {size=0: }, read_buffer=(nil) (length=0), exit_early=0}
Desktop:
ilastik version: 1.4.0-gpu
OS: Windows
The text was updated successfully, but these errors were encountered:
I am pretty sure this is due to biomageio.core. It tries to download some files in the beginning, and this process "gets stuck" behind proxy / within a VPN. I have noticed this before myself.
A user who works behind a company proxy could not start the Neural Network workflow: https://forum.image.sc/t/ilastik-neural-network-local-failed-to-start-from-the-beginning/80907/8
Logs (see below) indicate that the proxy server blocks GRPC requests from the ilastik frontend to the tiktorch server, responding with 403 Forbidden.
My guess is that the proxy is set up to only allow HTTPS, and as far as I can see we are not SSL-encrypting GRPC requests.
The user was able to get the workflow to work by setting
grpc.enable_http_proxy
to 0:But I don't think we can solve the problem like this in general.
Expected behavior
The workflow should run whether behind a proxy or not.
To Reproduce
Steps to reproduce the behavior:
(I will have to try if I can replicate the setup to actually reproduce the problem)
Error message/traceback
With
GRPC_TRACE=all
andGRPC_VERBOSITY=debug
, we receive additional logs:Desktop:
The text was updated successfully, but these errors were encountered: