connection refused during build

[waspinator]

Running nvidia-docker-compose build is having issue connecting with http://localhost:3476/docker/cli/json. I can't connect to that using a browser either.

Traceback (most recent call last):
  File "/usr/lib/python3.5/urllib/request.py", line 1254, in do_open
    h.request(req.get_method(), req.selector, req.data, headers)
  File "/usr/lib/python3.5/http/client.py", line 1106, in request
    self._send_request(method, url, body, headers)
  File "/usr/lib/python3.5/http/client.py", line 1151, in _send_request
    self.endheaders(body)
  File "/usr/lib/python3.5/http/client.py", line 1102, in endheaders
    self._send_output(message_body)
  File "/usr/lib/python3.5/http/client.py", line 934, in _send_output
    self.send(msg)
  File "/usr/lib/python3.5/http/client.py", line 877, in send
    self.connect()
  File "/usr/lib/python3.5/http/client.py", line 849, in connect
    (self.host,self.port), self.timeout, self.source_address)
  File "/usr/lib/python3.5/socket.py", line 711, in create_connection
    raise err
  File "/usr/lib/python3.5/socket.py", line 702, in create_connection
    sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/nvidia-docker-compose", line 55, in <module>
    resp = request.urlopen('http://{0}/docker/cli/json'.format(args.nvidia_docker_host)).read().decode()
  File "/usr/lib/python3.5/urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File "/usr/lib/python3.5/urllib/request.py", line 466, in open
    response = self._open(req, data)
  File "/usr/lib/python3.5/urllib/request.py", line 484, in _open
    '_open', req)
  File "/usr/lib/python3.5/urllib/request.py", line 444, in _call_chain
    result = func(*args)
  File "/usr/lib/python3.5/urllib/request.py", line 1282, in http_open
    return self.do_open(http.client.HTTPConnection, req)
  File "/usr/lib/python3.5/urllib/request.py", line 1256, in do_open
    raise URLError(err)
urllib.error.URLError: <urlopen error [Errno 111] Connection refused>

[waspinator]

this issue is with nvidia-docker

[jaredleekatzman]

Has this issue been fixed?

[eywalker]

@jaredleekatzman this is likely an issue with nvidia-docker. If you are encountering the same issue with nvidia-docker-compose, please verify that nvidia-docker is actually working first. If you encounter this issue specifically under nvidia-docker-compose, please let me know.

[jaredleekatzman]

@eywalker nvidia-docker works for me. But I'm still getting a similar error to above.

[eywalker]

What version of nvidia-docker are you using? If you are using version 2, then nvidia-docker-compose is not yet compatible with it. Version 1 of nvidia-docker starts a REST service by default on localhost:3476 (as documented here https://github.com/NVIDIA/nvidia-docker/wiki/nvidia-docker-plugin) and nvidia-docker-compose depends on this service. If you are using version 1 of nvidia-docker, then please verify that nvidia docker plugin is working - you can test it by commands like curl http://localhost:3476/docker/cli/json

[jaredleekatzman]

That makes sense then. I've upgraded to nvidia-docker2. If I down grade and still have problems, I'll let you know