Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Parallels: Consul clients encounter connection error #3

Open
loicsaintroch opened this issue Jun 25, 2022 · 5 comments · May be fixed by #5
Open

Parallels: Consul clients encounter connection error #3

loicsaintroch opened this issue Jun 25, 2022 · 5 comments · May be fixed by #5
Assignees
@loicsaintroch
Labels
bug Something isn't working help wanted Extra attention is needed

Comments

@loicsaintroch
Copy link
Member

loicsaintroch commented Jun 25, 2022 •
edited

What OS are you using?

macOS

What architecture are you using?

arm64

What Vagrant provider are you using?

parallels

Unexpected behavior

When using Parallels Desktop, Consul clients can connect to Consul servers but a connection error occurs. This prevents both Consul and Nomad client services to be registered in Consul even though they run as expected.

What kind of specifics do we need to set in Parallels Desktop to make this work as expected?
This doesn't happen when using VirtualBox.

Related issues:

Steps to reproduce

Run HashiBox with Parallels Desktop.

Expected behavior

No connection errors. Both Consul and Nomad client services should be registered in Consul.

Logs

-- Logs begin at Sat 2022-06-25 10:41:00 PDT, end at Sat 2022-06-25 10:47:55 PDT. --
Jun 25 10:42:53 node-client-1 systemd[1]: Starting "HashiCorp Consul"...
Jun 25 10:42:53 node-client-1 consul[22223]: ==> Starting Consul agent...
Jun 25 10:42:53 node-client-1 consul[22223]:            Version: '1.12.0'
Jun 25 10:42:53 node-client-1 consul[22223]:            Node ID: '94d27eb5-17a4-a047-33e0-8adfde2750d2'
Jun 25 10:42:53 node-client-1 consul[22223]:          Node name: 'node-client-1'
Jun 25 10:42:53 node-client-1 consul[22223]:         Datacenter: 'us' (Segment: '')
Jun 25 10:42:53 node-client-1 consul[22223]:             Server: false (Bootstrap: false)
Jun 25 10:42:53 node-client-1 consul[22223]:        Client Addr: [192.168.61.10] (HTTP: 8500, HTTPS: -1, gRPC: -1, DNS: 8600)
Jun 25 10:42:53 node-client-1 consul[22223]:       Cluster Addr: 192.168.61.10 (LAN: 8301, WAN: 8302)
Jun 25 10:42:53 node-client-1 consul[22223]:            Encrypt: Gossip: false, TLS-Outgoing: false, TLS-Incoming: false, Auto-Encrypt-TLS: false
Jun 25 10:42:53 node-client-1 consul[22223]: ==> Log data will now stream in as it occurs:
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.373-0700 [INFO]  agent.client.serf.lan: serf: EventMemberJoin: node-client-1 192.168.61.10
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.373-0700 [INFO]  agent.router: Initializing LAN area manager
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.373-0700 [INFO]  agent: Started DNS server: address=192.168.61.10:8600 network=udp
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.373-0700 [INFO]  agent: Started DNS server: address=192.168.61.10:8600 network=tcp
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.374-0700 [INFO]  agent: Starting server: address=192.168.61.10:8500 network=tcp protocol=http
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.374-0700 [INFO]  agent: started state syncer
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.374-0700 [INFO]  agent: Consul agent running!
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.374-0700 [INFO]  agent: Retry join is supported for the following discovery methods: cluster=LAN discovery_methods="aliyun aws azure digitalocean gce k8s linode mdns os packet scaleway softlayer tencentcloud triton vsphere"
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.374-0700 [INFO]  agent: Joining cluster...: cluster=LAN
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.374-0700 [INFO]  agent: (LAN) joining: lan_addresses=[192.168.60.10, 192.168.60.20, 192.168.60.30]
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.374-0700 [WARN]  agent.router.manager: No servers available
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.374-0700 [ERROR] agent.anti_entropy: failed to sync remote state: error="No known Consul servers"
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.375-0700 [INFO]  agent.client.serf.lan: serf: EventMemberJoin: node-server-1 192.168.60.10
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.375-0700 [INFO]  agent.client.serf.lan: serf: EventMemberJoin: node-client-3 192.168.61.30
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.375-0700 [INFO]  agent.client.serf.lan: serf: EventMemberJoin: node-client-2 192.168.61.20
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.375-0700 [INFO]  agent.client.serf.lan: serf: EventMemberJoin: node-server-3 192.168.60.30
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.376-0700 [INFO]  agent.client.serf.lan: serf: EventMemberJoin: node-server-2 192.168.60.20
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.376-0700 [INFO]  agent.client: adding server: server="node-server-1 (Addr: tcp/192.168.60.10:8300) (DC: us)"
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.376-0700 [INFO]  agent.client: adding server: server="node-server-3 (Addr: tcp/192.168.60.30:8300) (DC: us)"
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.376-0700 [INFO]  agent.client: adding server: server="node-server-2 (Addr: tcp/192.168.60.20:8300) (DC: us)"
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.378-0700 [INFO]  agent: (LAN) joined: number_of_nodes=3
Jun 25 10:42:53 node-client-1 systemd[1]: Started "HashiCorp Consul".
Jun 25 10:42:53 node-client-1 consul[22223]: 2022-06-25T10:42:53.398-0700 [INFO]  agent: Join cluster completed. Synced with initial agents: cluster=LAN num_agents=3
Jun 25 10:42:56 node-client-1 consul[22223]: 2022-06-25T10:42:56.378-0700 [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to node-server-2 but other probes failed, network may be misconfigured
Jun 25 10:42:57 node-client-1 consul[22223]: 2022-06-25T10:42:57.380-0700 [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to node-server-1 but other probes failed, network may be misconfigured
Jun 25 10:42:59 node-client-1 consul[22223]: 2022-06-25T10:42:59.375-0700 [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to node-server-3 but other probes failed, network may be misconfigured
Jun 25 10:43:01 node-client-1 consul[22223]: 2022-06-25T10:43:01.374-0700 [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to node-server-1 but other probes failed, network may be misconfigured
Jun 25 10:43:02 node-client-1 consul[22223]: 2022-06-25T10:43:02.375-0700 [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to node-server-3 but other probes failed, network may be misconfigured
Jun 25 10:43:03 node-client-1 consul[22223]: 2022-06-25T10:43:03.378-0700 [WARN]  agent: [core]grpc: addrConn.createTransport failed to connect to {us-192.168.60.30:8300 node-server-3 <nil> 0 <nil>}. Err: connection error: desc = "transport: Error while dialing dial tcp 192.168.61.10:0->192.168.60.30:8300: i/o timeout". Reconnec>
Jun 25 10:43:04 node-client-1 consul[22223]: 2022-06-25T10:43:04.376-0700 [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to node-server-2 but other probes failed, network may be misconfigured
Jun 25 10:43:04 node-client-1 consul[22223]: 2022-06-25T10:43:04.779-0700 [ERROR] agent.client: RPC failed to server: method=Catalog.NodeServiceList server=192.168.60.10:8300 error="rpc error getting client: failed to get conn: dial tcp 192.168.61.10:0->192.168.60.10:8300: i/o timeout"
Jun 25 10:43:04 node-client-1 consul[22223]: 2022-06-25T10:43:04.781-0700 [ERROR] agent.anti_entropy: failed to sync remote state: error="rpc error getting client: failed to get conn: dial tcp 192.168.61.10:0->192.168.60.10:8300: i/o timeout"
Jun 25 10:43:04 node-client-1 consul[22223]: 2022-06-25T10:43:04.781-0700 [ERROR] agent.client: RPC failed to server: method=Catalog.ListDatacenters server=192.168.60.10:8300 error="rpc error getting client: failed to get conn: rpc error: lead thread didn't get connection"
Jun 25 10:43:04 node-client-1 consul[22223]: 2022-06-25T10:43:04.781-0700 [INFO]  agent.http: Request cancelled: method=GET url=/v1/catalog/datacenters from=192.168.61.10:33066 error="rpc error getting client: failed to get conn: rpc error: lead thread didn't get connection"
Jun 25 10:43:04 node-client-1 consul[22223]: 2022-06-25T10:43:04.783-0700 [ERROR] agent.client: RPC failed to server: method=Catalog.Register server=192.168.60.10:8300 error="rpc error getting client: failed to get conn: rpc error: lead thread didn't get connection"
Jun 25 10:43:04 node-client-1 consul[22223]: 2022-06-25T10:43:04.784-0700 [WARN]  agent: Syncing node info failed.: error="rpc error getting client: failed to get conn: rpc error: lead thread didn't get connection"
Jun 25 10:43:04 node-client-1 consul[22223]: 2022-06-25T10:43:04.784-0700 [ERROR] agent: failed to sync changes: error="rpc error getting client: failed to get conn: rpc error: lead thread didn't get connection"
Jun 25 10:43:05 node-client-1 consul[22223]: 2022-06-25T10:43:05.377-0700 [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to node-server-2 but other probes failed, network may be misconfigured
Jun 25 10:43:07 node-client-1 consul[22223]: 2022-06-25T10:43:07.375-0700 [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to node-server-3 but other probes failed, network may be misconfigured
Jun 25 10:43:09 node-client-1 consul[22223]: 2022-06-25T10:43:09.375-0700 [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to node-server-1 but other probes failed, network may be misconfigured
Jun 25 10:43:12 node-client-1 consul[22223]: 2022-06-25T10:43:12.373-0700 [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to node-server-3 but other probes failed, network may be misconfigured
Jun 25 10:43:13 node-client-1 consul[22223]: 2022-06-25T10:43:13.376-0700 [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to node-server-1 but other probes failed, network may be misconfigured
Jun 25 10:43:13 node-client-1 consul[22223]: 2022-06-25T10:43:13.377-0700 [WARN]  agent: [core]grpc: addrConn.createTransport failed to connect to {us-192.168.60.20:8300 node-server-2 <nil> 0 <nil>}. Err: connection error: desc = "transport: Error while dialing dial tcp 192.168.61.10:0->192.168.60.20:8300: i/o timeout". Reconnec>
Jun 25 10:43:13 node-client-1 consul[22223]: 2022-06-25T10:43:13.377-0700 [WARN]  agent: [core]grpc: addrConn.createTransport failed to connect to {us-192.168.60.10:8300 node-server-1 <nil> 0 <nil>}. Err: connection error: desc = "transport: Error while dialing dial tcp 192.168.61.10:0->192.168.60.10:8300: i/o timeout". Reconnec>
Jun 25 10:43:13 node-client-1 consul[22223]: 2022-06-25T10:43:13.545-0700 [WARN]  agent: Check is now critical: check=_nomad-check-2c3e0263793586b31542ff179fddaba87a4a9e0b
Jun 25 10:43:14 node-client-1 consul[22223]: 2022-06-25T10:43:14.379-0700 [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to node-server-2 but other probes failed, network may be misconfigured
Jun 25 10:43:15 node-client-1 consul[22223]: 2022-06-25T10:43:15.164-0700 [ERROR] agent.client: RPC failed to server: method=Catalog.NodeServiceList server=192.168.60.30:8300 error="rpc error getting client: failed to get conn: dial tcp 192.168.61.10:0->192.168.60.30:8300: i/o timeout"
Jun 25 10:43:15 node-client-1 consul[22223]: 2022-06-25T10:43:15.165-0700 [ERROR] agent.anti_entropy: failed to sync remote state: error="rpc error getting client: failed to get conn: dial tcp 192.168.61.10:0->192.168.60.30:8300: i/o timeout"
Jun 25 10:43:15 node-client-1 consul[22223]: 2022-06-25T10:43:15.165-0700 [ERROR] agent.client: RPC failed to server: method=Catalog.Register server=192.168.60.30:8300 error="rpc error getting client: failed to get conn: rpc error: lead thread didn't get connection"
Jun 25 10:43:15 node-client-1 consul[22223]: 2022-06-25T10:43:15.166-0700 [WARN]  agent: Syncing node info failed.: error="rpc error getting client: failed to get conn: rpc error: lead thread didn't get connection"
Jun 25 10:43:15 node-client-1 consul[22223]: 2022-06-25T10:43:15.166-0700 [ERROR] agent: failed to sync changes: error="rpc error getting client: failed to get conn: rpc error: lead thread didn't get connection"
Jun 25 10:43:15 node-client-1 consul[22223]: 2022-06-25T10:43:15.166-0700 [ERROR] agent.client: RPC failed to server: method=Catalog.ListDatacenters server=192.168.60.30:8300 error="rpc error getting client: failed to get conn: rpc error: lead thread didn't get connection"
Jun 25 10:43:15 node-client-1 consul[22223]: 2022-06-25T10:43:15.166-0700 [ERROR] agent.http: Request error: method=GET url=/v1/catalog/datacenters from=192.168.61.10:33110 error="rpc error getting client: failed to get conn: rpc error: lead thread didn't get connection"
Jun 25 10:43:15 node-client-1 consul[22223]: 2022-06-25T10:43:15.167-0700 [ERROR] agent.client: RPC failed to server: method=Catalog.ListDatacenters server=192.168.60.30:8300 error="rpc error getting client: failed to get conn: rpc error: lead thread didn't get connection"
Jun 25 10:43:15 node-client-1 consul[22223]: 2022-06-25T10:43:15.167-0700 [INFO]  agent.http: Request cancelled: method=GET url=/v1/catalog/datacenters from=192.168.61.10:33104 error="rpc error getting client: failed to get conn: rpc error: lead thread didn't get connection"
Jun 25 10:43:16 node-client-1 consul[22223]: 2022-06-25T10:43:16.374-0700 [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to node-server-3 but other probes failed, network may be misconfigured
Jun 25 10:43:17 node-client-1 consul[22223]: 2022-06-25T10:43:17.376-0700 [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to node-server-1 but other probes failed, network may be misconfigured
Jun 25 10:43:19 node-client-1 consul[22223]: 2022-06-25T10:43:19.374-0700 [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to node-server-2 but other probes failed, network may be misconfigured
Jun 25 10:43:19 node-client-1 consul[22223]: 2022-06-25T10:43:19.424-0700 [INFO]  agent: Newer Consul version available: new_version=1.12.2 current_version=1.12.0
Jun 25 10:43:22 node-client-1 consul[22223]: 2022-06-25T10:43:22.374-0700 [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to node-server-2 but other probes failed, network may be misconfigured
Jun 25 10:43:23 node-client-1 consul[22223]: 2022-06-25T10:43:23.375-0700 [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to node-server-1 but other probes failed, network may be misconfigured
Jun 25 10:43:24 node-client-1 consul[22223]: 2022-06-25T10:43:24.377-0700 [WARN]  agent.client.memberlist.lan: memberlist: Was able to connect to node-server-3 but other probes failed, network may be misconfigured
Jun 25 10:43:24 node-client-1 consul[22223]: 2022-06-25T10:43:24.382-0700 [WARN]  agent: [core]grpc: addrConn.createTransport failed to connect to {us-192.168.60.30:8300 node-server-3 <nil> 0 <nil>}. Err: connection error: desc = "transport: Error while dialing dial tcp 192.168.61.10:0->192.168.60.30:8300: i/o timeout". Reconnec>
Jun 25 10:43:25 node-client-1 consul[22223]: 2022-06-25T10:43:25.177-0700 [WARN]  agent: Check is now critical: check=_nomad-check-2c3e0263793586b31542ff179fddaba87a4a9e0b
@loicsaintroch loicsaintroch added bug Something isn't working help wanted Extra attention is needed labels Jun 25, 2022
@loicsaintroch loicsaintroch self-assigned this Jun 25, 2022
@makp0 makp0 mentioned this issue Apr 2, 2023
Closed
@makp0
Copy link

makp0 commented Apr 3, 2023

Hi @loicsaintroch,
could you please provide guidance on how to run it on VirtualBox on ARM64 as a temporary alternative. I've tried both vmvare and virtualbox, but none worked for me.

@loicsaintroch
Copy link
Member Author

loicsaintroch commented Apr 3, 2023 •
edited

Sure.

First, you need to reset the config files since we write the Consul HTTP token statically in two places:

$ git reset --hard

Also, I would make sure to destroy existing resources:

$ make destroy

Cleanup and set proper environment variables. Here's the .env I'd use for a fresh install given your setup:

export VAGRANT_CLIENT_RAM=2048
export VAGRANT_SERVER_RAM=1024
export VAGRANT_PROVIDER=virtualbox
export UBUNTU_VERSION=20.04-arm64

Run the init process:

$ make init

As far as I can see, the only thing that could potentially not work on Linux are these lines because sed doesn't have the exact same behavior between macOS and Linux iirc.

@makp0
Copy link

makp0 commented Apr 3, 2023 •
edited

Based on what I've tried, it should error because the box https://app.vagrantup.com/bento/boxes/ubuntu-20.04-arm64 is built only for parallels.

@loicsaintroch
Copy link
Member Author

loicsaintroch commented Apr 3, 2023 •
edited

Indeed, you're right.

Edit: Upgrading Ubuntu to 22.04 won't fix the compatibility issues between amd64 and arm64. I'll try to find some time to investigate this Parallels issue and let you know.

@loicsaintroch loicsaintroch linked a pull request Apr 5, 2023 that will close this issue
Open
@loicsaintroch loicsaintroch linked a pull request Apr 5, 2023 that will close this issue
refactor: Use dynamic addresses via templates #5
Open
@loicsaintroch
Copy link
Member Author

PR opened. Everything seems to work as expected on Parallels Desktop now.
Can I let you try @makp0 and confirm it works for you as well?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@loicsaintroch loicsaintroch

Labels
bug Something isn't working help wanted Extra attention is needed
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

refactor: Use dynamic addresses via templates nunchistudio/hashibox
2 participants
@loicsaintroch @makp0