cmd: Introduce runtime and k8s columns

[mqasimsarfraz]

#1702 added support for k8s metadata in JSON/YAML but the support for columns was still missing. This PR allows the same information to be available as columns as well.

Related: #737
Fixes: #2066

Idea

• Before this change both ig and kubectl-gadget would filter k8s and runtime columns respectively rather then hiding. So we will switch to marking them hidden as needed by each client.
• The hidden columns can be displayed using -o columns as needed by the user.
• list-containers needs an extra work but k8s metadata can also be displayed using -o custom-columns

Tesing done

$ make minikube-start-containerd
$ make ig
$ docker cp ig minikube-containerd:/bin/ig
$ docker exec -it minikube-containerd bash
$ ig list-containers # same as before
RUNTIME.RUNTIMENAME RUNTIME.CONTAINERID                                              RUNTIME.CONTAINERNAME   RUNTIME.CONTAINERIMAGENAME                                 
containerd          0a9cbc23a8409cf015e389fa71f2422f28dbea1129e66a40e38685fb67424ef6 coredns                 k8s.gcr.io/coredns/coredns:v1.8.6                          
containerd          5e48d5e717d89f2237eb0b364d9c8b6265f949e5600f642d55420e1a4922e2e2 etcd                    k8s.gcr.io/etcd:3.5.3-0 
$ ig list-containers -o custom-columns=runtime.containerid,runtime.containername,k8s.pod,k8s.namespace
RUNTIME.CONTAINERID                                              RUNTIME.CONTAINERNAME   K8S.POD                                     K8S.NAMESPACE
0a9cbc23a8409cf015e389fa71f2422f28dbea1129e66a40e38685fb67424ef6 coredns                 coredns-6d4b75cb6d-94hkg                    kube-system  
5e48d5e717d89f2237eb0b364d9c8b6265f949e5600f642d55420e1a4922e2e2 etcd                    etcd-minikube-containerd                    kube-system

$ ig trace open -o columns=runtime.containername,k8s.pod,k8s.namespace,pid,comm
RUNTIME.CONTAINERNAME                                                K8S.POD                                                             K8S.NAMESPACE                                                       PID        COMM            
gadget                                                               gadget-p4kvb                                                        gadget                                                              448891     runc:[2:INIT]   
gadget                                                               gadget-p4kvb                                                        gadget                                                              448891     runc:[2:INIT]   

[mauriciovasquezbernal]

It looks great. I don't know if having the K8s. prefix in kubectl-gadget has been discussed, IMO that's good:

$ ./kubectl-gadget trace open
INFO[0000] Experimental features enabled                
K8S.NODE                  K8S.NAMESPACE             K8S.POD                  K8S.CONTAINER            PID        COMM         FD ERR P

I didn't find anything to be fixed in the code, only some things that are missing:

• It's missing to update all gadgets guides (sorry, I now it's to annoying to do it, perhaps a good regexp to replace can help)
• Prometheus documentation should be updated as well
  

  inspektor-gadget/docs/gadgets/prometheus.md

  Lines 95 to 98 in 146bc5a

  	labels:
  	- namespace
  	- pod
  	- container

  
  (other places in same file)

btw, this will also fix #2066.

[eiffel-fl]

I tested it and it works well, so thank you for this contribution:

root@minikube-containerd:/# ig list-containers -o custom-columns=runtime.containerid,runtime.containername,k8s.pod,k8s.namespace
RUNTIME.CONTAINE… RUNTIME.CONTAINERNAME                   K8S.POD                                 K8S.NAMESPACE                          
0dbc2b236492024a… coredns                                 coredns-6d4b75cb6d-6fz8w                kube-system                            
d316f9ad44642ad2… etcd                                    etcd-minikube-containerd                kube-system                            
ceec8a2150ffb369… kindnet-cni                             kindnet-t7z5q                           kube-system                            
c90e7729f491ff46… kube-apiserver                          kube-apiserver-minikube-containerd      kube-system                            
b9270e09b88d70ad… kube-controller-manager                 kube-controller-man…minikube-containerd kube-system                            
22f4ffe5fda3dfb7… kube-proxy                              kube-proxy-l8lvs                        kube-system                            
77ad2368722885b8… kube-scheduler                          kube-scheduler-minikube-containerd      kube-system                            
fa55550caeb0b391… storage-provisioner                     storage-provisioner                     kube-system                            
root@minikube-containerd:/# ig trace open -o columns=runtime.containername,k8s.pod,k8s.namespace,pid,comm | head
RUNTIME.CONTAINERNAME          K8S.POD                        K8S.NAMESPACE                  PID              COMM            
coredns                        coredns-6d4b75cb6d-6fz8w       kube-system                    22878            coredns         
storage-provisioner            storage-provisioner            kube-system                    23048            storage-provisi 
coredns                        coredns-6d4b75cb6d-6fz8w       kube-system                    22878            coredns         
coredns                        coredns-6d4b75cb6d-6fz8w       kube-system                    22878            coredns         
coredns                        coredns-6d4b75cb6d-6fz8w       kube-system                    22878            coredns         
kube-proxy                     kube-proxy-l8lvs               kube-system                    22562            kube-proxy      
kube-proxy                     kube-proxy-l8lvs               kube-system                    22562            kube-proxy      
kube-proxy                     kube-proxy-l8lvs               kube-system                    23370            iptables        
kube-proxy                     kube-proxy-l8lvs               kube-system                    23370            iptables        

I do not have a lot of comments, just address Mauricio comment and you can merge it.

[blanquicet]

I like the approach of hiding the columns instead of filtering them. It should also make the implementation of the --k8s and --runtime flags easier. BTW, given that you will add that in a second PR, this one is not fully fixing #737.

LGTM after managing the comments.

[mauriciovasquezbernal]

It should also make the implementation of the --k8s and --runtime flags

I'm wondering about what's the idea there. Provide independent flags to enable the runtime and k8s columns? IMO we should avoid adding so many flags, what about having something like -o columns=runtime? (we can try to find a different syntax for making it obvious it's for the full "kind" of columns.)

[mqasimsarfraz]

It's missing to update all gadgets guides (sorry, I now it's to annoying to do it, perhaps a good regexp to replace can help)

Done. Agreed, it wasn't fun to do but I realized we were missing RUNTIME columns for ig in docs as well so thanks for catching it!

[blanquicet]

It should also make the implementation of the --k8s and --runtime flags

I'm wondering about what's the idea there. Provide independent flags to enable the runtime and k8s columns? IMO we should avoid adding so many flags, what about having something like -o columns=runtime? (we can try to find a different syntax for making it obvious it's for the full "kind" of columns.)

My initial idea was to add --k8s only to ig and --runtime only to kubectl-gadget, but it was just my proposal, we can ask for opinions in the slack channel 🙂.

[mqasimsarfraz]

BTW, given that you will add that in a second PR, this one is not fully fixing #737.

I changed the linked issue from "Fixes" to "Related" to make sure we do follow up PR.

but it was just my proposal, we can ask for opinions in the slack channel 🙂.

Good idea. Let me start a discussion in the slack around this!

[mqasimsarfraz]

Thanks for the reviews!