Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Airgapped: envoy gets stuck installing with Failed to resolve 'raw.githubusercontent.com' attempting to download relation schema #98

Closed
NohaIhab opened this issue May 24, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@NohaIhab
Copy link
Contributor

NohaIhab commented May 24, 2024

Bug Description

When deploying CKF 1.8/stable on airgapped, the envoy charm gets stuck in maintenance with the unit message being (leader-elected) installing charm software.
In the relevant logs, it is observed that envoy charm is trying to download the shcema for the grpc relation.
Looking at the metadata.yaml, the relation is defined as follows:

  grpc:
    interface: grpc
    schema: https://raw.githubusercontent.com/canonical/operator-schemas/master/grpc.yaml
    versions: [v1]

Indeed we are using a remote reference for the schema, this is a blocker for deploying envoy in airgapped.
The same applies for the grpc-web relation.

To fix this, we can define the schema in the metadata.yaml and have the url set in the __schema_source attribute for reference.

This was reported in canonical/bundle-kubeflow#818, see point 1..

To Reproduce

Deploy envoy 2.0/stable in airgapped

Environment

microk8s 1.25-strict/stable
juju 3.1/stable

Relevant Log Output

2024-05-24 07:19:00 ERROR juju-log Uncaught exception while in charm code:
Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/urllib3/connection.py", line 203, in _new_conn
    sock = connection.create_connection(
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/urllib3/util/connection.py", line 60, in create_connection
    for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
  File "/usr/lib/python3.8/socket.py", line 918, in getaddrinfo
    for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolution

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/urllib3/connectionpool.py", line 790, in urlopen
    response = self._make_request(
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/urllib3/connectionpool.py", line 491, in _make_request
    raise new_e
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/urllib3/connectionpool.py", line 467, in _make_request
    self._validate_conn(conn)
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/urllib3/connectionpool.py", line 1092, in _validate_conn
    conn.connect()
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/urllib3/connection.py", line 611, in connect
    self.sock = sock = self._new_conn()
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/urllib3/connection.py", line 210, in _new_conn
    raise NameResolutionError(self.host, self, e) from e
urllib3.exceptions.NameResolutionError: <urllib3.connection.HTTPSConnection object at 0x7bab20b7d940>: Failed to resolve 'raw.githubusercontent.com' ([Errno -3] Temporary failure in name resolution)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/requests/adapters.py", line 486, in send
    resp = conn.urlopen(
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/urllib3/connectionpool.py", line 844, in urlopen
    retries = retries.increment(
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/urllib3/util/retry.py", line 515, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /canonical/operator-schemas/master/grpc.yaml (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7bab20b7d940>: Failed to resolve 'raw.githubusercontent.com' ([Errno -3] Temporary failure in name resolution)"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./src/charm.py", line 308, in <module>
    main(Operator)
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/ops/main.py", line 441, in main
    _emit_charm_event(charm, dispatcher.event_name)
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/ops/main.py", line 149, in _emit_charm_event
    event_to_emit.emit(*args, **kwargs)
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/ops/framework.py", line 342, in emit
    framework._emit(event)
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/ops/framework.py", line 839, in _emit
    self._reemit(event_path)
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/ops/framework.py", line 928, in _reemit
    custom_handler(event)
  File "./src/charm.py", line 165, in set_pod_spec
    interfaces = self._get_interfaces()
  File "./src/charm.py", line 261, in _get_interfaces
    interfaces = get_interfaces(self)
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/serialized_data_interface/sdi.py", line 351, in get_interfaces
    return {
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/serialized_data_interface/sdi.py", line 352, in <dictcomp>
    endpoint: get_interface(charm, endpoint)
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/serialized_data_interface/sdi.py", line 375, in get_interface
    schema = utils.get_schema(interface["schema"])
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/serialized_data_interface/utils.py", line 68, in get_schema
    response = _get_schema_response_from_remote(schema)
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/serialized_data_interface/utils.py", line 93, in _get_schema_response_from_remote
    response = requests.get(url=url, proxies=proxies)
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/requests/api.py", line 73, in get
    return request("get", url, params=params, **kwargs)
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/requests/api.py", line 59, in request
    return session.request(method=method, url=url, **kwargs)
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/requests/sessions.py", line 589, in request
    resp = self.send(prep, **send_kwargs)
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/requests/sessions.py", line 703, in send
    r = adapter.send(request, **kwargs)
  File "/var/lib/juju/agents/unit-envoy-0/charm/venv/requests/adapters.py", line 519, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='raw.githubusercontent.com', port=443): Max retries exceeded with url: /canonical/operator-schemas/master/grpc.yaml (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7bab20b7d940>: Failed to resolve 'raw.githubusercontent.com' ([Errno -3] Temporary failure in name resolution)"))

Additional Context

No response

@NohaIhab NohaIhab added the bug Something isn't working label May 24, 2024
Copy link

Thank you for reporting us your feedback!

The internal ticket has been created: https://warthogs.atlassian.net/browse/KF-5741.

This message was autogenerated

@NohaIhab
Copy link
Contributor Author

closed by #99

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant