File collector not reporting all resources or reporting them inconsistently

[myronmeier]

Hi. Thanks for the great tool. We were trying to run it on manifests using the -f option (the File collector) and found it to not always pick up deprecations in all the resources in the files specified or to pick them up differently when the tool is run again on the same files.

To reproduce, start with these 3 manifest files (I tried to trim them so they may not actually work as resources in a real cluster but they demonstrate this issue):

ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: app
  namespace: prod
spec:
  rules:
  - host: app.example.com
    http:
      paths:
      - backend:
          serviceName: backend
          servicePort: 8080
        path: /

stateful-set.yaml

---
apiVersion: apps/v1beta2
kind: StatefulSet
metadata:
  name: memecached
  namespace: prod
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: memcached
      app.kubernetes.io/instance: memcached
  serviceName: memcached
  template:
    metadata:
      labels:
        app.kubernetes.io/name: memcached
        app.kubernetes.io/instance: memcached
    spec:
      securityContext:
        fsGroup: 1001
      containers:
      - name: memcached
        image: memcached:1.5.20
        imagePullPolicy: ""
        securityContext:
          runAsUser: 1001
        command:
        - memcached
        - -m 64
        - -o
        - modern
        - -v
        ports:
        - name: memcache
          containerPort: 11211
        livenessProbe:
          tcpSocket:
            port: memcache
          initialDelaySeconds: 30
          timeoutSeconds: 5
        readinessProbe:
          tcpSocket:
            port: memcache
          initialDelaySeconds: 5
          timeoutSeconds: 1
        resources:
          requests:
            cpu: 50m
            memory: 64Mi
  updateStrategy:
    type: RollingUpdate

combined.yaml - this is just stateful-set.yaml appended to the end of ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: app
  namespace: prod
spec:
  rules:
  - host: app.example.com
    http:
      paths:
      - backend:
          serviceName: backend
          servicePort: 8080
        path: /
---
apiVersion: apps/v1beta2
kind: StatefulSet
metadata:
  name: memecached
  namespace: prod
spec:
  selector:
    matchLabels:
      app.kubernetes.io/name: memcached
      app.kubernetes.io/instance: memcached
  serviceName: memcached
  template:
    metadata:
      labels:
        app.kubernetes.io/name: memcached
        app.kubernetes.io/instance: memcached
    spec:
      securityContext:
        fsGroup: 1001
      containers:
        - name: memcached
          image: memcached:1.5.20
          imagePullPolicy: ""
          securityContext:
            runAsUser: 1001
          command:
            - memcached
            - -m 64
            - -o
            - modern
            - -v
          ports:
            - name: memcache
              containerPort: 11211
          livenessProbe:
            tcpSocket:
              port: memcache
            initialDelaySeconds: 30
            timeoutSeconds: 5
          readinessProbe:
            tcpSocket:
              port: memcache
            initialDelaySeconds: 5
            timeoutSeconds: 1
          resources:
            requests:
              cpu: 50m
              memory: 64Mi
  updateStrategy:
    type: RollingUpdate

Running on just the ingress.yaml file seems to always work:

$ kubent -c=false --helm2=false --helm3=false -f ingress.yaml
3:47PM INF >>> Kube No Trouble `kubent` <<<
3:47PM INF version 0.3.1 (git sha dev)
3:47PM INF Initializing collectors and retrieving data
3:47PM INF Retrieved 1 resources from collector name=File
3:47PM INF Loaded ruleset name=deprecated-1-16.rego
3:47PM INF Loaded ruleset name=deprecated-1-22.rego
__________________________________________________________________________________________
>>> Deprecated APIs removed in 1.22 <<<
------------------------------------------------------------------------------------------
KIND      NAMESPACE   NAME      API_VERSION
Ingress   prod        app       extensions/v1beta1
$

As does running on just stateful-set.yaml:

$ kubent -c=false --helm2=false --helm3=false -f stateful-set.yaml
3:48PM INF >>> Kube No Trouble `kubent` <<<
3:48PM INF version 0.3.1 (git sha dev)
3:48PM INF Initializing collectors and retrieving data
3:48PM INF Retrieved 1 resources from collector name=File
3:48PM INF Loaded ruleset name=deprecated-1-16.rego
3:48PM INF Loaded ruleset name=deprecated-1-22.rego
__________________________________________________________________________________________
>>> Deprecated APIs removed in 1.16  <<<
------------------------------------------------------------------------------------------
KIND          NAMESPACE   NAME         API_VERSION
StatefulSet   prod        memecached   apps/v1beta2
$

But running on both files only ever seems to pick up the resource in the last file, never both:

$ kubent -c=false --helm2=false --helm3=false -f ingress.yaml,stateful-set.yaml
3:50PM INF >>> Kube No Trouble `kubent` <<<
3:50PM INF version 0.3.1 (git sha dev)
3:50PM INF Initializing collectors and retrieving data
3:50PM INF Retrieved 2 resources from collector name=File
3:50PM INF Loaded ruleset name=deprecated-1-16.rego
3:50PM INF Loaded ruleset name=deprecated-1-22.rego
__________________________________________________________________________________________
>>> Deprecated APIs removed in 1.16  <<<
------------------------------------------------------------------------------------------
KIND          NAMESPACE   NAME         API_VERSION
StatefulSet   prod        memecached   apps/v1beta2
$ kubent -c=false --helm2=false --helm3=false -f stateful-set.yaml,ingress.yaml
3:50PM INF >>> Kube No Trouble `kubent` <<<
3:50PM INF version 0.3.1 (git sha dev)
3:50PM INF Initializing collectors and retrieving data
3:50PM INF Retrieved 2 resources from collector name=File
3:50PM INF Loaded ruleset name=deprecated-1-16.rego
3:50PM INF Loaded ruleset name=deprecated-1-22.rego
__________________________________________________________________________________________
>>> Deprecated APIs removed in 1.22 <<<
------------------------------------------------------------------------------------------
KIND      NAMESPACE   NAME      API_VERSION
Ingress   prod        app       extensions/v1beta1
$

(Notice the flipping of the order of the files in the -f option in the above 2 calls)

And weirdest of all, when run on the combined.yaml file, sometimes it reports 1 resource and sometimes the other (but I've never seen it report both)

$ kubent -c=false --helm2=false --helm3=false -f combined.yaml
3:52PM INF >>> Kube No Trouble `kubent` <<<
3:52PM INF version 0.3.1 (git sha dev)
3:52PM INF Initializing collectors and retrieving data
3:52PM INF Retrieved 2 resources from collector name=File
3:52PM INF Loaded ruleset name=deprecated-1-16.rego
3:52PM INF Loaded ruleset name=deprecated-1-22.rego
__________________________________________________________________________________________
>>> Deprecated APIs removed in 1.16  <<<
------------------------------------------------------------------------------------------
KIND          NAMESPACE   NAME         API_VERSION
StatefulSet   prod        memecached   apps/v1beta2
$ kubent -c=false --helm2=false --helm3=false -f combined.yaml
3:52PM INF >>> Kube No Trouble `kubent` <<<
3:52PM INF version 0.3.1 (git sha dev)
3:52PM INF Initializing collectors and retrieving data
3:52PM INF Retrieved 2 resources from collector name=File
3:52PM INF Loaded ruleset name=deprecated-1-16.rego
3:52PM INF Loaded ruleset name=deprecated-1-22.rego
__________________________________________________________________________________________
>>> Deprecated APIs removed in 1.22 <<<
------------------------------------------------------------------------------------------
KIND      NAMESPACE   NAME      API_VERSION
Ingress   prod        app       extensions/v1beta1
$

Sometimes this last one is not easy to reproduce. I had to run it over 10 times (all identical and on the same file) to get it to show the Ingress resource instead of the StatefulSet resource. Other times it seems to flip back and forth more often.

Notice that the resource count its reporting always seems to be correct even if its not detecting or reporting all the deprecations.

I tried to see if I could spot where it was but I don't know go and couldn't see anything obvious.

Thanks!

[stepanstipl]

Thank @myronmeier for a detailed bug report 👍 , this is a good catch and a great help for debugging the issue! I'll try to look at this soon. Also glad to hear you find the tool useful :)

[stepanstipl]

@myronmeier I was able to replicate this and I believe I've fixed this issue in #40, I've also improved tests to cover this problem. Please try the latest 0.3.2 release, thanks!