Make cilium status output more succinct by default

[joestringer]

Simplify output of `cilium status` by default, add new `--verbose`, `--brief` options

Add --verbose mode and some extra options to choose how much output you want from the cilium status API. Gather the verbose output in bugtool. Fold in a minor change to phrase controller states in terms of success rather than failure.

Example output with this series:

$ cilium status                                                                                                                                                                                                                                                
KVStore:                    Ok         Consul: 172.17.0.2:8300                  
ContainerRuntime:           Ok                                                  
Kubernetes:                 Disabled                                            
Cilium:                     Ok         OK                                       
NodeMonitor:                Disabled                                            
Cilium health daemon:       Warning   Get http:///var/run/cilium/health.sock/v1beta/hello: dial unix /var/run/cilium/health.sock:
 connect: no such file or directory                                             
IPv4 address pool:   3/65535 allocated                                                   
IPv6 address pool:   2/65535 allocated                                                   
Controller Status:     3/3 healthy

There's also this command for a brief, one-line summary:

$ cilium status --brief
OK

[joestringer]

test-me-please

[tgraf]

Can you also add a --brief option which prints the overall status on a single line? We need this for the Kubernetse healthcheck.

[joestringer]

@tgraf sure, do you have an example of how that output should look?

From a quick search, it looks something like we need this on success:

ok

Or this on failure:

error: ...

[tgraf]

@tgraf sure, do you have an example of how that output should look?

From a quick search, it looks something like we need this on success:

ACK

[aanm]

Since we are touching this can we also change it to something like

Allocated IPv4 addresses:   (3/65535 allocated)
Allocated IPv6 addresses:   (2/65535 allocated)
Known cluster nodes:        (1/1 connected (or "status ok"?))
 cilium-master (localhost):
  Primary Address:   10.0.2.15
   Type:             InternalIP
  AllocRange:        10.11.0.0/16
Controller Status (3/3 success)

Also, I'm assuming you are changing this because of the number of allocated IP addresses. Can we use @ianvernon 's IP "coalesce" functionality to present the IPs in CIDRs when possible?

[joestringer]

@aanm looks good, one comment:

Known cluster nodes: (1/1 connected (or "status ok"?))

^ Writing 1 connected is trivial, which would reflect the number of nodes that we've learned through the orchestrator. Writing N/M connected suggests some kind of health check, but we don't have that information in the main daemon or the client at this time, are you proposing that cilium status makes another call to cilium-health to determine this on the client side?

[aanm]

I think so, we were already showing up all nodes in the cluster with cilium status so why not give the full result of cilium-health?

[joestringer]

@aanm Seems like a reasonable idea. I'll play around with it a bit. Do we really need node information like allocrange for all other nodes in cilium status?

[joestringer]

Here's a quick mockup:

When a node is unhealthy, regular output (only print unhealthy nodes):

$ cilium status                                                                 
...                                                                             
IPAM:                                                                           
 IPv4 addresses:   3/65535 allocated                                            
 IPv6 addresses:   2/65535 allocated                                            
Cluster health:    N/M nodes healthy
  cilium-test:                                                                  
    Host connectivity to 10.0.2.15:                                             
      ICMP:          OK, RTT=395.968µs                                          
      HTTP via L3:   Connection timed out     

With all nodes healthy, option to print all health details:

$ cilium status --all-health                                                    
...                                                                             
Cluster health:    1/1 nodes healthy                                            
  cilium-master (localhost):                                                                                                                                                                                                                                   
    Host connectivity to 10.0.2.15:                                             
      ICMP:          OK, RTT=395.968µs                                          
      HTTP via L3:   OK, RTT=3.211737ms 

When cilium-health isn't up:

$ cilium status
...                                                                                                                                                                                                                                                
Cluster health:    ?/1 nodes healthy (cilium-health: xxx)

[aanm]

@aanm Seems like a reasonable idea. I'll play around with it a bit. Do we really need node information like allocrange for all other nodes in cilium status?

@joestringer I was only thinking for the current node but we have the alloc range for all nodes in the pkg/node

[joestringer]

@aanm right, that's what the current output in cilium status is, which we're currently printing. However I suspect that this information isn't really that interesting, so we could disable it (it's still exposed in the JSON if you fetch via cilium status -o json). Instead, we could place the cilium-health status there as a one-liner, or with a short list of failing nodes. Then we can achieve your suggestion of the output something like cluster health: N/M nodes connected

Added patch for cilium status --brief

[joestringer]

@aanm I implemented your suggestion in #2858. It needed a bit of refactoring, so I kept it out of this PR.

[joestringer]

test-me-please