Nested exceptions when the Docker socket is unreachable

[CarstenGrohmann]

Issue Type

• Bug report

Molecule and Ansible details

$ ansible --version
ansible [core 2.13.7]
  config file = /var/lib/jenkins/workspace/grade_ansible-sles-upgrade_MR-76/ansible.cfg
  configured module search path = ['/var/lib/jenkins/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/ansible
  ansible collection location = /var/lib/jenkins/.ansible/collections:/usr/share/ansible/collections
  executable location = /var/lib/jenkins/.python3.10/bin/ansible
  python version = 3.10.8 (main, Oct 28 2022, 17:28:32) [GCC]
  jinja version = 3.1.2
  libyaml = True
$ molecule --version
molecule 4.0.4 using python 3.10 
    ansible:2.13.7
    delegated:4.0.4 from molecule
    docker:2.1.0 from molecule_docker requiring collections: community.docker>=3.0.2 ansible.posix>=1.4.0

Molecule installation method (one of):

• pip

Ansible installation method (one of):

• pip

Detail any linters or test runners used:

Desired Behavior

Please catch these (the primary as well as the nested) exceptions and print a meaningful error message with the path of the missing socket.

Actual Behaviour

molecule converge is failing probably because the docker socket is missing. The failing due to the missing socket isn't unexpected, but that's triggers a exception and additionally 3 nested exceptions - that's an unexpected behaviour.

#  molecule converge
WARNING  The scenario config file ('/var/lib/jenkins/workspace/grade_ansible-myproj_MR-76/molecule/default/molecule.yml') has been modified since the scenario was created. If recent changes are important, reset the scenario with 'molecule destroy' to clean up created items or 'molecule reset' to clear current configuration.
INFO     default scenario test matrix: dependency, create, prepare, converge
INFO     Performing prerun with role_name_check=0...
INFO     Running ansible-galaxy collection install -v -r requirements.yml -p /var/lib/jenkins/.cache/ansible-compat/549545/collections
INFO     Set ANSIBLE_LIBRARY=/var/lib/jenkins/.cache/ansible-compat/549545/modules:/var/lib/jenkins/.ansible/plugins/modules:/usr/share/ansible/plugins/modules
INFO     Set ANSIBLE_COLLECTIONS_PATH=/var/lib/jenkins/.cache/ansible-compat/549545/collections:/var/lib/jenkins/.ansible/collections:/usr/share/ansible/collections
INFO     Set ANSIBLE_ROLES_PATH=/var/lib/jenkins/.cache/ansible-compat/549545/roles:roles:/var/lib/jenkins/workspace/grade_ansible-myproj_MR-76/roles
INFO     Running default > dependency
INFO     Running from /var/lib/jenkins/workspace/grade_ansible-myproj_MR-76 : ansible-galaxy collection install -vvv community.docker:>=3.0.2
INFO     Running from /var/lib/jenkins/workspace/grade_ansible-myproj_MR-76 : ansible-galaxy collection install -vvv ansible.posix:>=1.4.0
WARNING  Skipping, missing the requirements file.
WARNING  Skipping, missing the requirements file.
INFO     Running default > create
INFO     Sanity checks: 'docker'
Traceback (most recent call last):
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/urllib3/connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib64/python3.10/http/client.py", line 1282, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib64/python3.10/http/client.py", line 1328, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.10/http/client.py", line 1277, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.10/http/client.py", line 1037, in _send_output
    self.send(msg)
  File "/usr/lib64/python3.10/http/client.py", line 975, in send
    self.connect()
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/docker/transport/unixconn.py", line 30, in connect
    sock.connect(self.unix_socket)
FileNotFoundError: [Errno 2] No such file or directory

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/requests/adapters.py", line 489, in send
    resp = conn.urlopen(
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/urllib3/connectionpool.py", line 787, in urlopen
    retries = retries.increment(
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/urllib3/util/retry.py", line 550, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/urllib3/packages/six.py", line 769, in reraise
    raise value.with_traceback(tb)
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/urllib3/connectionpool.py", line 703, in urlopen
    httplib_response = self._make_request(
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/urllib3/connectionpool.py", line 398, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib64/python3.10/http/client.py", line 1282, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib64/python3.10/http/client.py", line 1328, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.10/http/client.py", line 1277, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.10/http/client.py", line 1037, in _send_output
    self.send(msg)
  File "/usr/lib64/python3.10/http/client.py", line 975, in send
    self.connect()
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/docker/transport/unixconn.py", line 30, in connect
    sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/docker/api/client.py", line 214, in _retrieve_server_version
    return self.version(api_version=False)["ApiVersion"]
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/docker/api/daemon.py", line 181, in version
    return self._result(self._get(url), json=True)
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/docker/api/client.py", line 237, in _get
    return self.get(url, **self._set_request_timeout(kwargs))
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/requests/sessions.py", line 600, in get
    return self.request("GET", url, **kwargs)
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/requests/sessions.py", line 587, in request
    resp = self.send(prep, **send_kwargs)
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/requests/sessions.py", line 701, in send
    r = adapter.send(request, **kwargs)
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/requests/adapters.py", line 547, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/jenkins/.python3.10/bin/molecule", line 8, in <module>
    sys.exit(main())
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/molecule/command/converge.py", line 102, in converge
    base.execute_cmdline_scenarios(scenario_name, args, command_args, ansible_args)
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/molecule/command/base.py", line 121, in execute_cmdline_scenarios
    execute_scenario(scenario)
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/molecule/command/base.py", line 165, in execute_scenario
    execute_subcommand(scenario.config, action)
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/molecule/command/base.py", line 154, in execute_subcommand
    return command(config).execute(args)
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/molecule/logger.py", line 188, in wrapper
    rt = func(*args, **kwargs)
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/molecule/command/create.py", line 94, in execute
    self._config.provisioner.create()
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/molecule/provisioner/ansible.py", line 771, in create
    pb.execute()
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/molecule/provisioner/ansible_playbook.py", line 117, in execute
    self._config.driver.sanity_checks()
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/molecule_docker/driver.py", line 245, in sanity_checks
    docker_client = docker.from_env()
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/docker/client.py", line 96, in from_env
    return cls(
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/docker/client.py", line 45, in __init__
    self.api = APIClient(*args, **kwargs)
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/docker/api/client.py", line 197, in __init__
    self._version = self._retrieve_server_version()
  File "/var/lib/jenkins/.python3.10/lib64/python3.10/site-packages/docker/api/client.py", line 221, in _retrieve_server_version
    raise DockerException(
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', FileNotFoundError(2, 'No such file or directory'))

[kenorb]

Same error on molecule prepare, my Docker Desktop is running fine on macOS. I can run hello-world container without any issue.

docker==6.1.3
molecule==5.1.0
molecule-plugins==23.4.1

Workaround

In Docker's Desktop, Advanced settings, Allow the default Docker socket to be used.

[alicekaerast]

Docker Desktop and Colima both place the docker socket in a non-standard location, meaning that molecule's docker plugin is unable to find it by default

For Colima you can export DOCKER_HOST=unix:///Users/$USER/.colima/default/docker.sock

For Docker Desktop you can export DOCKER_HOST=unix:///Users/$USER/.docker/run/docker.sock or you can follow the previously suggested workaround of "In Docker's Desktop, Advanced settings, Allow the default Docker socket to be used." (the latter requires admin rights)

But the original request to provide a clearer error message in these cases is still valid - perhaps along with a link to suggested fixes like the above

[alicekaerast]

Upon further investigation, the error is actually in the molecule-plugins repo and is caused by expecting an error from requests rather than docker. I've raised a PR that I believe will resolve this issue:

ansible-community/molecule-plugins#196

[zhan9san]

Thanks for @alicekaerast 's ansible-community/molecule-plugins#196