hubble: Hubble node_name field should contain cluster name

[Maddy007-maha]

Description:
This field node_name currently only contains the node name, without its cluster-mesh prefix.
If the cluster name is not the default cluster name then it should show the node_name along with the prefixed cluster name

Its should be shown as below:
cluster/<node_name>
Eg: cluster1/node-ip.compute.internal

$ hubble observe --last 2 -o compact --print-node-name
Nov 3 11:40:45.407 [cluster5/ip-172-0-51-175.us-west-2.compute.internal]: [...]
Nov 3 11:40:46.554 [cluster7/ip-172-0-121-242.us-west-2.compute.internal ]: [...]

Fixes: #13872

Signed-off-by: Maddy007-maha mahadev.panchal@accuknox.com

[Maddy007-maha]

Hi, I have edited the configMap of cilium with following cluster information as below:
changed from:

cluster-id: ""
cluster-name: default

to:

cluster-id: "1"
cluster-name: cluster1

Then restarted the cilium pods and hubble pods, its been observed that local node is unavailable. Am I missing something and do we have any doc to follow to setup single cluster.
$ hubble status

Handling connection for 4245
Healthcheck (via localhost:4245): Ok
Current/Max Flows: 0/0
Flows/s: N/A
Connected Nodes: 0/1
Unavailable Nodes: 1
  - cluster1/minikube

$ hubble observe -o compact --last 2 --print-node-name

Handling connection for 4245
Apr 28 15:16:12.878 [hubble-relay-765f884d5d-vfb4q]: 1 nodes are unavailable: cluster1/minikube

I can see the cilium node list and its showing properly

$ kubectl -n kube-system exec -ti cilium-2pq7h -- cilium node list
Defaulted container "cilium-agent" out of: cilium-agent, clean-cilium-state (init)
Name                IPv4 Address   Endpoint CIDR   IPv6 Address   Endpoint CIDR
cluster1/minikube   192.168.49.2   10.0.0.0/24
I see some failures in the hubble-relay saying that peer is not reachable from the hubble-relay logs:
level=warning msg="Failed to create gRPC client" address="192.168.49.2:4244" error="context deadline exceeded" hubble-tls=true next-try-in=1h25m20s peer=cluster1/minikube subsys=hubble-relay
level=info msg="No connection to peer cluster1/minikube, skipping" address="192.168.49.2:4244" subsys=hubble-relay
level=info msg=Connecting address="192.168.49.2:4244" hubble-tls=true peer=cluster1/minikube subsys=hubble-relay
level=warning msg="Failed to create gRPC client" address="192.168.49.2:4244" error="context deadline exceeded" hubble-tls=true next-try-in=1h30m0s peer=cluster1/minikube subsys=hubble-relay 

Kindly throw some light :)

[Maddy007-maha]

Any update on reviewing this PR?

[gandro]

Apologies for the delayed review. Thanks a lot for the PR, this is overall the right approach.

There are some failing unit tests that need to be fixed in github.com/cilium/cilium/pkg/hubble/monitor:

https://travis-ci.com/github/cilium/cilium/jobs/502667881#L14680

[gandro]

Awesome, thanks again!

[gandro]

test-me-please

[gandro]

@Maddy007-maha

Kindly throw some light :)

As for the issue regarding clustermesh, I don't think minikube is the right way to test it, as it doesn't really have node connectivity. I recommend using kind instead and following the guide: https://docs.cilium.io/en/latest/gettingstarted/clustermesh/clustermesh/#gs-clustermesh

[gandro]

test-1.21-4.9

[gandro]

@Maddy007-maha No need to re-request a review, this just needs to pass CI at this point (which it has) and requires a review from a janitor as well. Then it will be merged.

Edit: I'm also re-running the conformance tests. This slightly changes the behavior of Hubble's output in clustermesh, thus we want to make sure it doesn't break the new cilium-cli based CI