Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Image import does not work #69

Closed
bartv opened this issue May 17, 2020 · 3 comments
Closed

Image import does not work #69

bartv opened this issue May 17, 2020 · 3 comments
Assignees
@kazaamjt

Comments

@bartv
Copy link
Contributor

bartv commented May 17, 2020

The image import fails for various reasons. Tested on openstack rocky, stein, train. The following 3 errors are observed in random order in deploy and repairs. On 5 different clusters:

Traceback (most recent call last):
  File "/opt/inmanta/lib64/python3.6/site-packages/inmanta/agent/handler.py", line 858, in execute
    self.create_resource(ctx, desired)
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/code/modules/inmanta_plugins.openstack.py", line 1195, in create_resource
    self._glance.images.image_import(
AttributeError: 'Controller' object has no attribute 'image_import'

Traceback (most recent call last):
  File "/opt/inmanta/lib64/python3.6/site-packages/urllib3/connection.py", line 160, in _new_conn
    (self._dns_host, self.port), self.timeout, **extra_kw
  File "/opt/inmanta/lib64/python3.6/site-packages/urllib3/util/connection.py", line 84, in create_connection
    raise err
  File "/opt/inmanta/lib64/python3.6/site-packages/urllib3/util/connection.py", line 74, in create_connection
    sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/inmanta/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/opt/inmanta/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 392, in _make_request
    conn.request(method, url, **httplib_request_kw)
  File "/usr/lib64/python3.6/http/client.py", line 1254, in request
    self._send_request(method, url, body, headers, encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1300, in _send_request
    self.endheaders(body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1249, in endheaders
    self._send_output(message_body, encode_chunked=encode_chunked)
  File "/usr/lib64/python3.6/http/client.py", line 1036, in _send_output
    self.send(msg)
  File "/usr/lib64/python3.6/http/client.py", line 974, in send
    self.connect()
  File "/opt/inmanta/lib64/python3.6/site-packages/urllib3/connection.py", line 187, in connect
    conn = self._new_conn()
  File "/opt/inmanta/lib64/python3.6/site-packages/urllib3/connection.py", line 172, in _new_conn
    self, "Failed to establish a new connection: %s" % e
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7f077fdadeb8>: Failed to establish a new connection: [Errno 110] Connection timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/inmanta/lib64/python3.6/site-packages/requests/adapters.py", line 449, in send
    timeout=timeout
  File "/opt/inmanta/lib64/python3.6/site-packages/urllib3/connectionpool.py", line 725, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/opt/inmanta/lib64/python3.6/site-packages/urllib3/util/retry.py", line 439, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='node5.ii.inmanta.com', port=9292): Max retries exceeded with url: /v2/images?limit=20 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f077fdadeb8>: Failed to establish a new connection: [Errno 110] Connection timed out',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/env/lib/python3.6/site-packages/keystoneauth1/session.py", line 926, in _send_request
    resp = self.session.request(method, url, **kwargs)
  File "/opt/inmanta/lib64/python3.6/site-packages/requests/sessions.py", line 530, in request
    resp = self.send(prep, **send_kwargs)
  File "/opt/inmanta/lib64/python3.6/site-packages/requests/sessions.py", line 643, in send
    r = adapter.send(request, **kwargs)
  File "/opt/inmanta/lib64/python3.6/site-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='node5.ii.inmanta.com', port=9292): Max retries exceeded with url: /v2/images?limit=20 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f077fdadeb8>: Failed to establish a new connection: [Errno 110] Connection timed out',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/env/lib/python3.6/site-packages/glanceclient/common/http.py", line 328, in request
    **kwargs)
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/env/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 237, in request
    return self.session.request(url, method, **kwargs)
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/env/lib/python3.6/site-packages/keystoneauth1/session.py", line 835, in request
    resp = send(**kwargs)
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/env/lib/python3.6/site-packages/keystoneauth1/session.py", line 942, in _send_request
    raise exceptions.ConnectFailure(msg)
keystoneauth1.exceptions.connection.ConnectFailure: Unable to establish connection to http://node5.ii.inmanta.com:9292/v2/images?limit=20: HTTPConnectionPool(host='node5.ii.inmanta.com', port=9292): Max retries exceeded with url: /v2/images?limit=20 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f077fdadeb8>: Failed to establish a new connection: [Errno 110] Connection timed out',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/inmanta/lib64/python3.6/site-packages/inmanta/agent/handler.py", line 844, in execute
    self.read_resource(ctx, current)
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/code/modules/inmanta_plugins.openstack.py", line 1141, in read_resource
    image = self._get_image(resource)
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/code/modules/inmanta_plugins.openstack.py", line 1114, in _get_image
    for image in self._glance.images.list()
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/code/modules/inmanta_plugins.openstack.py", line 1113, in <listcomp>
    image
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/env/lib/python3.6/site-packages/glanceclient/common/utils.py", line 518, in __next__
    return self._next()
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/env/lib/python3.6/site-packages/glanceclient/common/utils.py", line 507, in _next
    obj, resp = next(self._self_wrapped)
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/env/lib/python3.6/site-packages/glanceclient/v2/images.py", line 183, in list
    for image, resp in paginate(url, page_size, limit):
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/env/lib/python3.6/site-packages/glanceclient/v2/images.py", line 110, in paginate
    resp, body = self.http_client.get(next_url, headers=req_id_hdr)
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/env/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 375, in get
    return self.request(url, 'GET', **kwargs)
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/env/lib/python3.6/site-packages/glanceclient/common/http.py", line 340, in request
    raise exc.CommunicationError(message=message)
glanceclient.exc.CommunicationError: Error finding address for http://node5.ii.inmanta.com:9292/v2/images?limit=20: Unable to establish connection to http://node5.ii.inmanta.com:9292/v2/images?limit=20: HTTPConnectionPool(host='node5.ii.inmanta.com', port=9292): Max retries exceeded with url: /v2/images?limit=20 (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f077fdadeb8>: Failed to establish a new connection: [Errno 110] Connection timed out',))

Traceback (most recent call last):
  File "/opt/inmanta/lib64/python3.6/site-packages/inmanta/agent/handler.py", line 865, in execute
    self.update_resource(ctx, dict(changes), desired)
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/code/modules/inmanta_plugins.openstack.py", line 1269, in update_resource
    self._glance.images.update(ctx.get("image_id"), **kwargs)
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/env/lib/python3.6/site-packages/glanceclient/v2/images.py", line 301, in update
    resp, _ = self.http_client.patch(url, headers=hdrs, data=image.patch)
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/env/lib/python3.6/site-packages/keystoneauth1/adapter.py", line 387, in patch
    return self.request(url, 'PATCH', **kwargs)
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/env/lib/python3.6/site-packages/glanceclient/common/http.py", line 342, in request
    return self._handle_response(resp)
  File "/var/lib/inmanta/f81be80b-8582-499f-b002-7cdd2f2df98a/agent/env/lib/python3.6/site-packages/glanceclient/common/http.py", line 107, in _handle_response
    raise exc.from_response(resp, resp.content)
glanceclient.exc.HTTPException: 415 Unsupported Media Type: The request media type application/octet-stream is not supported by this server. (HTTP N/A)
@bartv
Copy link
Contributor Author

bartv commented May 17, 2020 •
edited

This is how terraform does it with the web-download import mechanism:
https://github.com/terraform-providers/terraform-provider-openstack/blob/master/openstack/resource_openstack_images_image_v2.go

And a presentation about this mechanism:
https://object-storage-ca-ymq-1.vexxhost.net/swift/v1/6e4619c416ff4bd19e1c087f27a43eea/www-assets-prod/presentation-media/Glance-Image-Import-SYD.pdf

@bartv
Copy link
Contributor Author

bartv commented May 17, 2020 •
edited

A workflow on the commandline:

 # glance task-create --type import --input '{"import_from_format": "qcow2", "import_from": "http://file.ii.inmanta.com/images/os/CentOS-7-x86_64-GenericCloud-2003.qcow2", "image_properties": {"disk_format": "qcow2", "container_format": "bare"}}'
+------------+----------------------------------------------------------------------------------+
| Property   | Value                                                                            |
+------------+----------------------------------------------------------------------------------+
| created_at | 2020-05-17T18:30:08Z                                                             |
| id         | 91931ead-10ff-493c-a968-069d1531380f                                             |
| input      | {"import_from_format": "qcow2", "import_from":                                   |
|            | "http://file.ii.inmanta.com/images/os/CentOS-7-x86_64-GenericCloud-2003.qcow2",  |
|            | "image_properties": {"disk_format": "qcow2", "container_format": "bare"}}        |
| message    |                                                                                  |
| owner      | 94073c3e53a140afb1fbc06098c73259                                                 |
| result     | None                                                                             |
| status     | pending                                                                          |
| type       | import                                                                           |
| updated_at | 2020-05-17T18:30:08Z                                                             |
+------------+----------------------------------------------------------------------------------+

 # glance task-list                       
+--------------------------------------+--------+---------+----------------------------------+
| ID                                   | Type   | Status  | Owner                            |
+--------------------------------------+--------+---------+----------------------------------+
| 91931ead-10ff-493c-a968-069d1531380f | import | success | 94073c3e53a140afb1fbc06098c73259 |
+--------------------------------------+--------+---------+----------------------------------+

# glance task-show 91931ead-10ff-493c-a968-069d1531380f
+------------+----------------------------------------------------------------------------------+
| Property   | Value                                                                            |
+------------+----------------------------------------------------------------------------------+
| created_at | 2020-05-17T18:30:08Z                                                             |
| expires_at | 2020-05-19T18:30:16Z                                                             |
| id         | 91931ead-10ff-493c-a968-069d1531380f                                             |
| input      | {"import_from_format": "qcow2", "import_from":                                   |
|            | "http://file.ii.inmanta.com/images/os/CentOS-7-x86_64-GenericCloud-2003.qcow2",  |
|            | "image_properties": {"disk_format": "qcow2", "container_format": "bare"}}        |
| message    |                                                                                  |
| owner      | 94073c3e53a140afb1fbc06098c73259                                                 |
| result     | {"image_id": "0f232a18-eacf-4d59-bb85-279a61443802"}                             |
| status     | success                                                                          |
| type       | import                                                                           |
| updated_at | 2020-05-17T18:30:16Z                                                             |
+------------+----------------------------------------------------------------------------------+

# glance image-update --name 20200422-centos-7 --visibility public 0f232a18-eacf-4d59-bb85-279a61443802
+------------------+----------------------------------------------------------------------------------+
| Property         | Value                                                                            |
+------------------+----------------------------------------------------------------------------------+
| checksum         | ef7f109590094e37c54aead73f3cdedc                                                 |
| container_format | bare                                                                             |
| created_at       | 2020-05-17T18:30:08Z                                                             |
| disk_format      | qcow2                                                                            |
| id               | 0f232a18-eacf-4d59-bb85-279a61443802                                             |
| min_disk         | 0                                                                                |
| min_ram          | 0                                                                                |
| name             | 20200422-centos-7                                                                |
| os_hash_algo     | sha512                                                                           |
| os_hash_value    | 72a90b056215b7a4f216bd962082cceecfc7ea8e9a93b80f141d2a07f8d7890c735a19a30be0a758 |
|                  | 0f24897bc451c95750f39569c83205b642c1e60d43f92c93                                 |
| os_hidden        | False                                                                            |
| owner            | 94073c3e53a140afb1fbc06098c73259                                                 |
| protected        | False                                                                            |
| size             | 858783744                                                                        |
| status           | active                                                                           |
| tags             | []                                                                               |
| updated_at       | 2020-05-17T18:36:22Z                                                             |
| virtual_size     | Not available                                                                    |
| visibility       | public                                                                           |
+------------------+----------------------------------------------------------------------------------+

@kazaamjt kazaamjt self-assigned this May 19, 2020
@kazaamjt
Copy link
Contributor

Unable to reproduce at the moment.
Going to need a code sample.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kazaamjt kazaamjt

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@bartv @wouterdb @kazaamjt