Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CEKit can't connect to a remote Docker daemon #536

Closed
cmoulliard opened this issue Jun 7, 2019 · 11 comments

Comments

Projects
2 participants
@cmoulliard
Copy link

commented Jun 7, 2019

Issue

Cekit 3.1 cant connect to a docker demon (running in minikube) as reported hereafter
even if a local docker client can

docker images         
REPOSITORY                                                       TAG                 IMAGE ID            CREATED             SIZE
quay.io/snowdrop/component-operator                              latest              02563e572365        5 hours ago         38.7MB
quay.io/snowdrop/component-operator                              <none>              dfe7ca0bcce3        19 hours ago        38.7MB
quay.io/snowdrop/component-operator                              <none>              3d986d7749f3        26 hours ago        38.7MB
quay.io/snowdrop/component-operator                              <none>              18b37cc9da09        29 hours ago        38.7MB
quay.io/kubernetes-service-catalog/service-catalog               v0.2.1              8493270ffdac        3 days ago          48.4MB
circleci/picard                                                  <none>              fb3ae985ea3c        7 days ago   

Error

cekit -v build docker      
2019-06-07 17:38:27,316 cekit        DEBUG    Running version 3.1.0
2019-06-07 17:38:27,316 cekit        DEBUG    Configuring CEKit...
2019-06-07 17:38:27,317 cekit        DEBUG    Removing dirty directory: 'target/repo'
2019-06-07 17:38:27,318 cekit        WARNING  You are running CEKit on an unknown platform. External dependencies suggestions may not work!
2019-06-07 17:38:27,318 cekit        DEBUG    Checking CEKit core dependencies...
2019-06-07 17:38:27,318 cekit        DEBUG    Checking if 'git' dependency is provided...
2019-06-07 17:38:27,318 cekit        DEBUG    CEKit dependency 'git' provided via the '/usr/local/bin/git' executable.
2019-06-07 17:38:27,318 cekit        DEBUG    All dependencies provided!
2019-06-07 17:38:27,436 cekit        INFO     Generating files for docker engine
2019-06-07 17:38:27,437 cekit        INFO     Initializing image descriptor...
2019-06-07 17:38:27,437 cekit        DEBUG    Checking CEKit generate dependencies...
2019-06-07 17:38:27,437 cekit        DEBUG    Checking if 'odcs-client' dependency is provided...
2019-06-07 17:38:27,437 cekit        DEBUG    Required CEKit library 'odcs-client' was found as a 'odcs' module!
2019-06-07 17:38:27,437 cekit        DEBUG    All dependencies provided!
2019-06-07 17:38:27,437 cekit        DEBUG    Removing old target directory
2019-06-07 17:38:27,438 cekit        DEBUG    Reading descriptor from 'image.yaml' file...
2019-06-07 17:38:27,451 cekit        DEBUG    Downloading module repository: 'modules'
2019-06-07 17:38:27,451 cekit        INFO     Preparing resource 'modules'
2019-06-07 17:38:27,451 cekit        DEBUG    Copying repository from '/Users/dabou/Temp/cekit-test/modules' to 'target/repo/modules'.
2019-06-07 17:38:27,455 cekit        DEBUG    Reading descriptor from '/Users/dabou/Temp/cekit-test/target/repo/modules/jdk8/module.yaml' file...
2019-06-07 17:38:27,459 cekit        DEBUG    Adding module 'jdk8', path: 'target/repo/modules/jdk8'
2019-06-07 17:38:27,459 cekit        DEBUG    Reading descriptor from '/Users/dabou/Temp/cekit-test/target/repo/modules/tomcat/module.yaml' file...
2019-06-07 17:38:27,469 cekit        DEBUG    Adding module 'tomcat', path: 'target/repo/modules/tomcat'
2019-06-07 17:38:27,470 cekit        DEBUG    Reading descriptor from '/Users/dabou/Temp/cekit-test/target/repo/modules/user/module.yaml' file...
2019-06-07 17:38:27,475 cekit        DEBUG    Adding module 'user', path: 'target/repo/modules/user'
2019-06-07 17:38:27,478 cekit        DEBUG    No artifacts to fetch
2019-06-07 17:38:27,482 cekit        INFO     Rendering Dockerfile...
2019-06-07 17:38:27,528 cekit        DEBUG    Dockerfile rendered
2019-06-07 17:38:27,528 cekit        DEBUG    Checking CEKit build dependencies...
2019-06-07 17:38:27,528 cekit        DEBUG    Checking if 'python-docker' dependency is provided...
2019-06-07 17:38:27,528 cekit        DEBUG    Required CEKit library 'python-docker' was found as a 'docker' module!
2019-06-07 17:38:27,528 cekit        DEBUG    Checking if 'docker-squash' dependency is provided...
2019-06-07 17:38:27,528 cekit        DEBUG    Required CEKit library 'docker-squash' was found as a 'docker_squash' module!
2019-06-07 17:38:27,528 cekit        DEBUG    All dependencies provided!
2019-06-07 17:38:27,529 cekit        INFO     Building container image using Docker...
2019-06-07 17:38:27,529 cekit        DEBUG    Building image with tags: 'my-example:1.0', 'my-example:latest'
2019-06-07 17:38:27,663 cekit        DEBUG    Caught ConnectionError attempting to communicate with Docker 
Traceback (most recent call last):
  File "/Users/dabou/cekit/lib/python3.7/site-packages/urllib3/connectionpool.py", line 603, in urlopen
    chunked=chunked)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/urllib3/connectionpool.py", line 355, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1016, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 956, in send
    self.connect()
  File "/Users/dabou/cekit/lib/python3.7/site-packages/docker/transport/unixconn.py", line 43, in connect
    sock.connect(self.unix_socket)
ConnectionRefusedError: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/dabou/cekit/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/Users/dabou/cekit/lib/python3.7/site-packages/urllib3/connectionpool.py", line 641, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/Users/dabou/cekit/lib/python3.7/site-packages/urllib3/util/retry.py", line 368, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/urllib3/connectionpool.py", line 603, in urlopen
    chunked=chunked)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/urllib3/connectionpool.py", line 355, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1016, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 956, in send
    self.connect()
  File "/Users/dabou/cekit/lib/python3.7/site-packages/docker/transport/unixconn.py", line 43, in connect
    sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionRefusedError(61, 'Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/dabou/cekit/lib/python3.7/site-packages/cekit/builders/docker_builder.py", line 95, in _build_with_docker
    stream = docker_client.build(**docker_args)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/docker/api/build.py", line 269, in build
    timeout=timeout,
  File "/Users/dabou/cekit/lib/python3.7/site-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/docker/api/client.py", line 226, in _post
    return self.post(url, **self._set_request_timeout(kwargs))
  File "/Users/dabou/cekit/lib/python3.7/site-packages/requests/sessions.py", line 581, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionRefusedError(61, 'Connection refused'))
2019-06-07 17:38:27,664 cekit        ERROR    ('Unknown ConnectionError from docker ; is the daemon started and correctly setup?', ConnectionError(ProtocolError('Connection aborted.', ConnectionRefusedError(61, 'Connection refused'))))
Traceback (most recent call last):
  File "/Users/dabou/cekit/lib/python3.7/site-packages/cekit/cli.py", line 341, in run
    command.execute()
  File "/Users/dabou/cekit/lib/python3.7/site-packages/cekit/builder.py", line 60, in execute
    self.run()
  File "/Users/dabou/cekit/lib/python3.7/site-packages/cekit/builders/docker_builder.py", line 191, in run
    image_id = self._build_with_docker(docker_client)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/cekit/builders/docker_builder.py", line 142, in _build_with_docker
    raise cekit_exception
cekit.errors.CekitError: ('Unknown ConnectionError from docker ; is the daemon started and correctly setup?', ConnectionError(ProtocolError('Connection aborted.', ConnectionRefusedError(61, 'Connection refused'))))

Docker info

docker info
Containers: 69
 Running: 42
 Paused: 0
 Stopped: 27
Images: 30
Server Version: 18.09.6
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: bb71b10fd8f58240ca47fbb579b9d1028eea7c84
runc version: N/A
init version: N/A (expected: )
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.15.0
Operating System: Buildroot 2018.05
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 4.717GiB
Name: minikube
ID: CYZX:EYSM:ITH4:IL3X:UVBX:E64M:V6Y4:QV6B:VD4D:UZFV:23J6:IITL
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
 provider=virtualbox
Experimental: false
Insecure Registries:
 10.96.0.0/12
 127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine
@goldmann

This comment has been minimized.

Copy link
Contributor

commented Jun 7, 2019

You are running Docker inside a VM (not locally) in this case you need to make the tool aware of where the daemon is running.

You need to set the DOCKER_HOST environment variable. That should do the trick. This is a standard Docker environment variable, but we should probably mention it in the docs.

https://stackoverflow.com/questions/25234792/what-does-the-docker-host-variable-do

@cmoulliard

This comment has been minimized.

Copy link
Author

commented Jun 7, 2019

Docker env vars are well defined

DOCKER_API_VERSION=1.39
DOCKER_CERT_PATH=/Users/dabou/.minikube/certs
DOCKER_HOST=tcp://192.168.99.101:2376
DOCKER_TLS_VERIFY=1
cekit build docker                                                                                             
2019-06-07 18:20:47,642 cekit        WARNING  You are running CEKit on an unknown platform. External dependencies suggestions may not work!
2019-06-07 18:20:47,775 cekit        INFO     Generating files for docker engine
2019-06-07 18:20:47,775 cekit        INFO     Initializing image descriptor...
2019-06-07 18:20:47,788 cekit        INFO     Preparing resource 'modules'
2019-06-07 18:20:47,813 cekit        INFO     Rendering Dockerfile...
2019-06-07 18:20:47,853 cekit        INFO     Building container image using Docker...
2019-06-07 18:20:48,001 cekit        ERROR    Unknown ConnectionError from docker ; is the daemon started and correctly setup?

export  DOCKER_HOST=tcp://192.168.99.101:2376
cekit build docker
2019-06-07 18:21:01,232 cekit        WARNING  You are running CEKit on an unknown platform. External dependencies suggestions may not work!
2019-06-07 18:21:01,355 cekit        INFO     Generating files for docker engine
2019-06-07 18:21:01,355 cekit        INFO     Initializing image descriptor...
2019-06-07 18:21:01,369 cekit        INFO     Preparing resource 'modules'
2019-06-07 18:21:01,394 cekit        INFO     Rendering Dockerfile...
2019-06-07 18:21:01,437 cekit        INFO     Building container image using Docker...
2019-06-07 18:21:01,576 cekit        ERROR    Unknown ConnectionError from docker ; is the daemon started and correctly setup?

@goldmann

This comment has been minimized.

Copy link
Contributor

commented Jun 7, 2019

Is there some more info when running with --verbose? I'll need to investigate it.

@cmoulliard

This comment has been minimized.

Copy link
Author

commented Jun 7, 2019

2019-06-07 18:25:30,450 cekit        INFO     Building container image using Docker...
2019-06-07 18:25:30,450 cekit        DEBUG    Building image with tags: 'my-example:1.0', 'my-example:latest'
2019-06-07 18:25:30,605 cekit        DEBUG    Caught ConnectionError attempting to communicate with Docker 
Traceback (most recent call last):
  File "/Users/dabou/cekit/lib/python3.7/site-packages/urllib3/connectionpool.py", line 603, in urlopen
    chunked=chunked)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/urllib3/connectionpool.py", line 355, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1016, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 956, in send
    self.connect()
  File "/Users/dabou/cekit/lib/python3.7/site-packages/docker/transport/unixconn.py", line 43, in connect
    sock.connect(self.unix_socket)
ConnectionRefusedError: [Errno 61] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/dabou/cekit/lib/python3.7/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/Users/dabou/cekit/lib/python3.7/site-packages/urllib3/connectionpool.py", line 641, in urlopen
    _stacktrace=sys.exc_info()[2])
  File "/Users/dabou/cekit/lib/python3.7/site-packages/urllib3/util/retry.py", line 368, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/urllib3/packages/six.py", line 685, in reraise
    raise value.with_traceback(tb)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/urllib3/connectionpool.py", line 603, in urlopen
    chunked=chunked)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/urllib3/connectionpool.py", line 355, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1229, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1275, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1224, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 1016, in _send_output
    self.send(msg)
  File "/usr/local/Cellar/python/3.7.3/Frameworks/Python.framework/Versions/3.7/lib/python3.7/http/client.py", line 956, in send
    self.connect()
  File "/Users/dabou/cekit/lib/python3.7/site-packages/docker/transport/unixconn.py", line 43, in connect
    sock.connect(self.unix_socket)
urllib3.exceptions.ProtocolError: ('Connection aborted.', ConnectionRefusedError(61, 'Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/dabou/cekit/lib/python3.7/site-packages/cekit/builders/docker_builder.py", line 95, in _build_with_docker
    stream = docker_client.build(**docker_args)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/docker/api/build.py", line 269, in build
    timeout=timeout,
  File "/Users/dabou/cekit/lib/python3.7/site-packages/docker/utils/decorators.py", line 46, in inner
    return f(self, *args, **kwargs)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/docker/api/client.py", line 226, in _post
    return self.post(url, **self._set_request_timeout(kwargs))
  File "/Users/dabou/cekit/lib/python3.7/site-packages/requests/sessions.py", line 581, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/requests/sessions.py", line 533, in request
    resp = self.send(prep, **send_kwargs)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/requests/sessions.py", line 646, in send
    r = adapter.send(request, **kwargs)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/requests/adapters.py", line 498, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', ConnectionRefusedError(61, 'Connection refused'))
2019-06-07 18:25:30,610 cekit        ERROR    ('Unknown ConnectionError from docker ; is the daemon started and correctly setup?', ConnectionError(ProtocolError('Connection aborted.', ConnectionRefusedError(61, 'Connection refused'))))
Traceback (most recent call last):
  File "/Users/dabou/cekit/lib/python3.7/site-packages/cekit/cli.py", line 341, in run
    command.execute()
  File "/Users/dabou/cekit/lib/python3.7/site-packages/cekit/builder.py", line 60, in execute
    self.run()
  File "/Users/dabou/cekit/lib/python3.7/site-packages/cekit/builders/docker_builder.py", line 191, in run
    image_id = self._build_with_docker(docker_client)
  File "/Users/dabou/cekit/lib/python3.7/site-packages/cekit/builders/docker_builder.py", line 142, in _build_with_docker
    raise cekit_exception
cekit.errors.CekitError: ('Unknown ConnectionError from docker ; is the daemon started and correctly setup?', ConnectionError(ProtocolError('Connection aborted.', ConnectionRefusedError(61, 'Connection refused'))))
@goldmann

This comment has been minimized.

Copy link
Contributor

commented Jun 8, 2019

I had a brief look at the code and it looks like the environment variable is not honored in CEKit. I fooled myself, because the docker-squash tool we use has support for it (https://github.com/goldmann/docker-squash/blob/master/docs/environment_variables.adoc#docker_host). I just need to implement similar approach in CEKit directly.

Thanks for the report!

@goldmann goldmann changed the title Cekit can't connect to a local docker daemon CEKit can't connect to a remote Docker daemon Jun 8, 2019

@goldmann

This comment has been minimized.

Copy link
Contributor

commented Jun 8, 2019

@cmoulliard Are you considering this issue a blocker for you? I could provide a patch early next week for you to test it.

@cmoulliard

This comment has been minimized.

Copy link
Author

commented Jun 8, 2019

This is blocking but I will continue my tests Tuesday/Wednesday next week ;-)

@goldmann

This comment has been minimized.

Copy link
Contributor

commented Jun 10, 2019

@goldmann

This comment has been minimized.

Copy link
Contributor

commented Jun 10, 2019

@goldmann

This comment has been minimized.

Copy link
Contributor

commented Jun 11, 2019

I've opened #537 with the change.

goldmann added a commit to goldmann/cekit that referenced this issue Jun 11, 2019

@goldmann goldmann added this to To do in Release 3.2.0 via automation Jun 11, 2019

@goldmann goldmann modified the milestones: 3.1.1, 3.2.0 Jun 11, 2019

@goldmann goldmann moved this from To do to In progress in Release 3.2.0 Jun 11, 2019

goldmann added a commit to goldmann/cekit that referenced this issue Jun 11, 2019

Use DOCKER_* environment variables to properly create Docker client
Thi smakes it possible to use DOCKER_HOST and related
environent variables that would instruct the library where to
connect to the Docker daemon. This makes is possible to use
non-local Docker daemons.

Fixes cekit#536
@cmoulliard

This comment has been minimized.

Copy link
Author

commented Jun 12, 2019

Build is working with your patch

2019-06-12 15:05:38,997 cekit        DEBUG    Checking if 'git' dependency is provided...
2019-06-12 15:05:38,997 cekit        DEBUG    CEKit dependency 'git' provided via the '/usr/local/bin/git' executable.
2019-06-12 15:05:38,997 cekit        DEBUG    All dependencies provided!
2019-06-12 15:05:39,118 cekit        INFO     Generating files for docker engine
2019-06-12 15:05:39,118 cekit        INFO     Initializing image descriptor...
2019-06-12 15:05:39,118 cekit        DEBUG    Checking CEKit generate dependencies...
2019-06-12 15:05:39,118 cekit        DEBUG    Checking if 'odcs-client' dependency is provided...
2019-06-12 15:05:39,118 cekit        DEBUG    Required CEKit library 'odcs-client' was found as a 'odcs' module!
2019-06-12 15:05:39,118 cekit        DEBUG    All dependencies provided!
2019-06-12 15:05:39,118 cekit        DEBUG    Removing old target directory
2019-06-12 15:05:39,119 cekit        DEBUG    Reading descriptor from 'image.yaml' file...
2019-06-12 15:05:39,131 cekit        DEBUG    Downloading module repository: 'modules'
2019-06-12 15:05:39,132 cekit        INFO     Preparing resource 'modules'
2019-06-12 15:05:39,132 cekit        DEBUG    Copying repository from '/Users/dabou/Temp/cekit-test/modules' to 'target/repo/modules'.
2019-06-12 15:05:39,136 cekit        DEBUG    Reading descriptor from '/Users/dabou/Temp/cekit-test/target/repo/modules/jdk8/module.yaml' file...
2019-06-12 15:05:39,139 cekit        DEBUG    Adding module 'jdk8', path: 'target/repo/modules/jdk8'
2019-06-12 15:05:39,140 cekit        DEBUG    Reading descriptor from '/Users/dabou/Temp/cekit-test/target/repo/modules/tomcat/module.yaml' file...
2019-06-12 15:05:39,148 cekit        DEBUG    Adding module 'tomcat', path: 'target/repo/modules/tomcat'
2019-06-12 15:05:39,149 cekit        DEBUG    Reading descriptor from '/Users/dabou/Temp/cekit-test/target/repo/modules/user/module.yaml' file...
2019-06-12 15:05:39,154 cekit        DEBUG    Adding module 'user', path: 'target/repo/modules/user'
2019-06-12 15:05:39,156 cekit        DEBUG    No artifacts to fetch
2019-06-12 15:05:39,159 cekit        INFO     Rendering Dockerfile...
2019-06-12 15:05:39,199 cekit        DEBUG    Dockerfile rendered
2019-06-12 15:05:39,199 cekit        DEBUG    Checking CEKit build dependencies...
2019-06-12 15:05:39,199 cekit        DEBUG    Checking if 'python-docker' dependency is provided...
2019-06-12 15:05:39,199 cekit        DEBUG    Required CEKit library 'python-docker' was found as a 'docker' module!
2019-06-12 15:05:39,199 cekit        DEBUG    Checking if 'docker-squash' dependency is provided...
2019-06-12 15:05:39,199 cekit        DEBUG    Required CEKit library 'docker-squash' was found as a 'docker_squash' module!
2019-06-12 15:05:39,200 cekit        DEBUG    All dependencies provided!
2019-06-12 15:05:39,200 cekit        INFO     Building container image using Docker...
2019-06-12 15:05:39,200 cekit        DEBUG    Building image with tags: 'my-example:1.0', 'my-example:latest'
2019-06-12 15:05:39,200 cekit        DEBUG    Preparing Docker client...
2019-06-12 15:05:39,226 cekit        DEBUG    Docker client ready and working
2019-06-12 15:05:39,231 cekit        DEBUG    {'Version': '1.13.1', 'ApiVersion': '1.26', 'MinAPIVersion': '1.12', 'GitCommit': '07f3374/1.13.1', 'GoVersion': 'go1.10.3', 'Os': 'linux', 'Arch': 'amd64', 'KernelVersion': '3.10.0-957.5.1.el7.x86_64', 'BuildTime': '2019-02-13T17:10:12.956717356+00:00', 'PkgVersion': 'docker-1.13.1-91.git07f3374.el7.centos.x86_64'}
2019-06-12 15:05:39,557 cekit        INFO     Docker: Step 1/7 : FROM centos:7
2019-06-12 15:05:42,629 cekit        INFO     Docker: ---> 9f38484d220f
2019-06-12 15:05:42,629 cekit        INFO     Docker: Step 2/7 : USER root
2019-06-12 15:05:42,666 cekit        INFO     Docker: ---> Running in 4a18e3aaa24e
2019-06-12 15:05:42,732 cekit        INFO     Docker: ---> 6b75cecff062
2019-06-12 15:05:42,743 cekit        INFO     Docker: Removing intermediate container 4a18e3aaa24e
2019-06-12 15:05:42,743 cekit        INFO     Docker: Step 3/7 : LABEL description "My Example Tomcat Image" io.cekit.version "3.1.dev0" summary "My Example Tomcat Image"
2019-06-12 15:05:42,767 cekit        INFO     Docker: ---> Running in d90d98f56fd9
2019-06-12 15:05:42,825 cekit        INFO     Docker: ---> d6fa66c5be3d
2019-06-12 15:05:42,835 cekit        INFO     Docker: Removing intermediate container d90d98f56fd9
2019-06-12 15:05:42,835 cekit        INFO     Docker: Step 4/7 : USER root
2019-06-12 15:05:42,860 cekit        INFO     Docker: ---> Running in a571a318f241
2019-06-12 15:05:42,922 cekit        INFO     Docker: ---> 09a45d17f6b5
2019-06-12 15:05:42,934 cekit        INFO     Docker: Removing intermediate container a571a318f241
2019-06-12 15:05:42,935 cekit        INFO     Docker: Step 5/7 : RUN [ ! -d /tmp/scripts ] || rm -rf /tmp/scripts
2019-06-12 15:05:42,957 cekit        INFO     Docker: ---> Running in 4acfbf44a19c
2019-06-12 15:05:42,958 cekit        INFO     Docker: 
2019-06-12 15:05:43,481 cekit        INFO     Docker: ---> cfad8f1bff6f
2019-06-12 15:05:43,492 cekit        INFO     Docker: Removing intermediate container 4acfbf44a19c
2019-06-12 15:05:43,492 cekit        INFO     Docker: Step 6/7 : RUN [ ! -d /tmp/artifacts ] || rm -rf /tmp/artifacts
2019-06-12 15:05:43,520 cekit        INFO     Docker: ---> Running in 104ed5b69eb6
2019-06-12 15:05:43,521 cekit        INFO     Docker: 
2019-06-12 15:05:43,973 cekit        INFO     Docker: ---> edecfac0ed23
2019-06-12 15:05:43,986 cekit        INFO     Docker: Removing intermediate container 104ed5b69eb6
2019-06-12 15:05:43,986 cekit        INFO     Docker: Step 7/7 : USER root
2019-06-12 15:05:44,011 cekit        INFO     Docker: ---> Running in c13de4e0860c
2019-06-12 15:05:44,066 cekit        INFO     Docker: ---> 3cac05c09312
2019-06-12 15:05:44,076 cekit        INFO     Docker: Removing intermediate container c13de4e0860c
2019-06-12 15:05:44,077 cekit        INFO     Docker: Successfully built 3cac05c09312
2019-06-12 15:05:44,077 cekit        INFO     Squashing image 3cac05c09312...
2019-06-12 15:05:44,081 cekit        INFO     docker-squash version 1.0.7, Docker 07f3374/1.13.1, API 1.26...
2019-06-12 15:05:44,082 cekit        INFO     Using v2 image format
2019-06-12 15:05:44,082 cekit        DEBUG    Using /var/folders/56/dtp67r4n1hv79q2hrh_dbwcc0000gn/T/docker-squash-6gd7z3ot as the temporary directory
2019-06-12 15:05:44,093 cekit        INFO     Old image has 9 layers
2019-06-12 15:05:44,094 cekit        DEBUG    Old layers: ['<missing>', '<missing>', 'sha256:9f38484d220fa527b1fb19747638497179500a1bed8bf0498eb788229229e6e1', 'sha256:6b75cecff062caa18fb5edabfbd03c691f6bdba27ece844a82b92b600a147ee2', 'sha256:d6fa66c5be3d78f70d02c0505d5ac41a2d4a4c514bcb510822788cf8b59b2b04', 'sha256:09a45d17f6b582342c3ee23d91c5f3557b2ba69283944d8ec930feea4f879adb', 'sha256:cfad8f1bff6f0f21eb84606f2bd8c2ed381f75e5a58bfbf0206f390cc51eaf77', 'sha256:edecfac0ed23bc3f15c0404c8343fbe5d36a7d8d0792959bbe2e2fd1b7a0b13b', 'sha256:3cac05c0931255245ac67b6864e782e4f2a4ac380b30f18d9f8a752d9160fd03']
2019-06-12 15:05:44,094 cekit        DEBUG    We detected layer as the argument to squash
2019-06-12 15:05:44,098 cekit        DEBUG    Layer ID to squash from: sha256:9f38484d220fa527b1fb19747638497179500a1bed8bf0498eb788229229e6e1
2019-06-12 15:05:44,098 cekit        INFO     Checking if squashing is necessary...
2019-06-12 15:05:44,098 cekit        INFO     Attempting to squash last 6 layers...
2019-06-12 15:05:44,098 cekit        DEBUG    Layers to squash: ['sha256:6b75cecff062caa18fb5edabfbd03c691f6bdba27ece844a82b92b600a147ee2', 'sha256:d6fa66c5be3d78f70d02c0505d5ac41a2d4a4c514bcb510822788cf8b59b2b04', 'sha256:09a45d17f6b582342c3ee23d91c5f3557b2ba69283944d8ec930feea4f879adb', 'sha256:cfad8f1bff6f0f21eb84606f2bd8c2ed381f75e5a58bfbf0206f390cc51eaf77', 'sha256:edecfac0ed23bc3f15c0404c8343fbe5d36a7d8d0792959bbe2e2fd1b7a0b13b', 'sha256:3cac05c0931255245ac67b6864e782e4f2a4ac380b30f18d9f8a752d9160fd03']
2019-06-12 15:05:44,098 cekit        DEBUG    Layers to move: ['<missing>', '<missing>', 'sha256:9f38484d220fa527b1fb19747638497179500a1bed8bf0498eb788229229e6e1']
2019-06-12 15:05:44,099 cekit        INFO     Saving image sha256:3cac05c0931255245ac67b6864e782e4f2a4ac380b30f18d9f8a752d9160fd03 to /var/folders/56/dtp67r4n1hv79q2hrh_dbwcc0000gn/T/docker-squash-6gd7z3ot/old directory...
2019-06-12 15:05:44,099 cekit        DEBUG    Try #1...
2019-06-12 15:05:47,511 cekit        DEBUG    Extracting image using iterator over raw data
2019-06-12 15:05:51,956 cekit        INFO     Image saved!
2019-06-12 15:05:51,957 cekit        INFO     Squashing image '3cac05c09312'...
2019-06-12 15:05:51,958 cekit        DEBUG    Reading '/var/folders/56/dtp67r4n1hv79q2hrh_dbwcc0000gn/T/docker-squash-6gd7z3ot/old/manifest.json' JSON file...
2019-06-12 15:05:51,958 cekit        DEBUG    Reading '/var/folders/56/dtp67r4n1hv79q2hrh_dbwcc0000gn/T/docker-squash-6gd7z3ot/old/3cac05c0931255245ac67b6864e782e4f2a4ac380b30f18d9f8a752d9160fd03.json' JSON file...
2019-06-12 15:05:51,959 cekit        INFO     Starting squashing...
2019-06-12 15:05:51,959 cekit        DEBUG    Generating list of files in layer 'a80555e39a139dbcfb00d603a16b2660ac38e196f68b8473509c7ba518e5fdd4'...
2019-06-12 15:05:52,633 cekit        DEBUG    Done, found 10714 files
2019-06-12 15:05:52,637 cekit        INFO     Squashing file '/var/folders/56/dtp67r4n1hv79q2hrh_dbwcc0000gn/T/docker-squash-6gd7z3ot/old/c09cf288d1b75e71a8ab77c99295f4b63f5dd76f6982c68e7c97afc58730f86a/layer.tar'...
2019-06-12 15:05:52,638 cekit        DEBUG    Searching for marker files in '/var/folders/56/dtp67r4n1hv79q2hrh_dbwcc0000gn/T/docker-squash-6gd7z3ot/old/c09cf288d1b75e71a8ab77c99295f4b63f5dd76f6982c68e7c97afc58730f86a/layer.tar' archive...
2019-06-12 15:05:52,638 cekit        DEBUG    Found 'run/secrets/.wh..wh..opq' marker file
2019-06-12 15:05:52,638 cekit        DEBUG    Done, found 1 files
2019-06-12 15:05:52,638 cekit        DEBUG    Searching for symbolic links in '/var/folders/56/dtp67r4n1hv79q2hrh_dbwcc0000gn/T/docker-squash-6gd7z3ot/old/c09cf288d1b75e71a8ab77c99295f4b63f5dd76f6982c68e7c97afc58730f86a/layer.tar' archive...
2019-06-12 15:05:52,639 cekit        DEBUG    Done, found 0 files
2019-06-12 15:05:52,639 cekit        DEBUG    Skipping '/run/secrets/.wh..wh..opq' marker file, at the end of squashing we'll see if it's necessary to add it back
2019-06-12 15:05:52,639 cekit        DEBUG    Marker files to add: ['run/secrets/.wh..wh..opq']
2019-06-12 15:05:52,642 cekit        DEBUG    Skipping 'run/secrets/.wh..wh..opq' marker file...
2019-06-12 15:05:52,643 cekit        INFO     Squashing finished!
2019-06-12 15:05:53,385 cekit        DEBUG    Moving unmodified layer 'a80555e39a139dbcfb00d603a16b2660ac38e196f68b8473509c7ba518e5fdd4'...
2019-06-12 15:05:53,385 cekit        DEBUG    No name and tag provided for the image, skipping generating repositories file
2019-06-12 15:05:53,385 cekit        DEBUG    Removing from disk already squashed layers...
2019-06-12 15:05:53,386 cekit        INFO     Original image size: 199.79 MB
2019-06-12 15:05:53,386 cekit        INFO     Squashed image size: 199.80 MB
2019-06-12 15:05:53,387 cekit        INFO     If the squashed image is larger than original it means that there were no meaningful files to squash and it just added metadata. Are you sure you specified correct parameters?
2019-06-12 15:05:53,387 cekit        INFO     New squashed image ID is 6da5f1ff412ecf06691df429cb32d98a65ffb59f9a546efbe23145e9dee8a886
2019-06-12 15:05:53,387 cekit        DEBUG    Generating tar archive for the squashed image...
2019-06-12 15:05:54,680 cekit        DEBUG    Archive generated
2019-06-12 15:05:54,743 cekit        DEBUG    Loading squashed image...
2019-06-12 15:05:58,576 cekit        DEBUG    Image loaded!
2019-06-12 15:05:58,628 cekit        DEBUG    Cleaning up /var/folders/56/dtp67r4n1hv79q2hrh_dbwcc0000gn/T/docker-squash-6gd7z3ot temporary directory
2019-06-12 15:05:58,699 cekit        INFO     Removing old 3cac05c09312 image...
2019-06-12 15:05:58,795 cekit        INFO     Image removed!
2019-06-12 15:05:58,795 cekit        INFO     Done
2019-06-12 15:05:58,809 cekit        INFO     Image built and available under following tags: my-example:1.0, my-example:latest
2019-06-12 15:05:58,810 cekit        INFO     Finished!

Congratulations !!

Release 3.2.0 automation moved this from In progress to Done Jun 12, 2019

goldmann added a commit that referenced this issue Jun 12, 2019

Use DOCKER_* environment variables to properly create Docker client
Thi smakes it possible to use DOCKER_HOST and related
environent variables that would instruct the library where to
connect to the Docker daemon. This makes is possible to use
non-local Docker daemons.

Fixes #536

@goldmann goldmann self-assigned this Jun 14, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.