k8s Resource EventSource triggers updates all the time

[jonnylangefeld]

Describe the bug
As the title says, when using the k8s resource event source with eventType UPDATE, the gateway recognizes updates all the time, even without any changes to the underlying resource. This results in a lot of sensor triggers over time.

  NAME                      STATUS      AGE   DURATION   PRIORITY
  resource-workflow-w2gr4   Succeeded   21s   0s         0
  resource-workflow-z46sx   Succeeded   5m    0s         0
  resource-workflow-56xvd   Succeeded   8m    0s         0
  resource-workflow-h9kpz   Succeeded   10m   0s         0
  resource-workflow-ncb89   Succeeded   11m   0s         0
  resource-workflow-zm8lj   Succeeded   19m   0s         0
  resource-workflow-w7z27   Succeeded   20m   0s         0
  resource-workflow-ntw2n   Succeeded   25m   0s         0
  resource-workflow-cm7k4   Succeeded   27m   0s         0
  resource-workflow-p54t9   Succeeded   28m   0s         0
  my-workflow               Succeeded   28m   0s         0

The gatway logs reflect that as well through repeating log lines like the following

$ kubectl logs -l gateway-name=resource --all-containers
INFO[2020-07-19 04:41:48] detected update event                         event-source=example
INFO[2020-07-19 04:41:48] new event received, dispatching to gateway client  event-source=example
INFO[2020-07-19 04:41:48] dispatching event to subscribers              event-source=example
INFO[2020-07-19 04:41:48] dispatched event                              event-source=example
INFO[2020-07-19 04:43:42] detected update event                         event-source=example
INFO[2020-07-19 04:43:42] new event received, dispatching to gateway client  event-source=example
INFO[2020-07-19 04:43:42] dispatching event to subscribers              event-source=example
INFO[2020-07-19 04:43:42] dispatched event                              event-source=example
INFO[2020-07-19 04:45:03] detected update event                         event-source=example
INFO[2020-07-19 04:45:03] dispatching event to subscribers              event-source=example
INFO[2020-07-19 04:45:03] new event received, dispatching to gateway client  event-source=example
INFO[2020-07-19 04:45:03] dispatched event                              event-source=example

To Reproduce
Steps to reproduce the behavior:

1. Follow the example in the resource setup documentation. EventSource, Gateway and Sensor need to be applied. For your convenience here all commands to copy and paste as one:

kubectl apply -n argo-events -f https://raw.githubusercontent.com/argoproj/argo-events/stable/examples/event-sources/resource.yaml
kubectl apply -n argo-events -f https://raw.githubusercontent.com/argoproj/argo-events/stable/examples/gateways/resource.yaml
kubectl apply -n argo-events -f https://raw.githubusercontent.com/argoproj/argo-events/stable/examples/sensors/resource.yaml

2. Update the eventType in the EventSource spec and add UPDATE to the eventTypes array.

kubectl patch eventsource/resource-event-source -p '{"spec":{"resource":{"example":{"eventTypes":["ADD", "UPDATE"]}}}}' --type=merge

3. submit the workload ONCE:

argo submit <(cat <<EOF
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
  name: my-workflow
  labels:
    app: my-workflow
spec:
  entrypoint: whalesay
  templates:
  - name: whalesay
    container:
      image: docker/whalesay:latest
      command: [cowsay]
      args: ["hello world"]
EOF
)

Expected behavior
The target workflow defined in the sensor runs once when added.

Environment (please complete the following information):

• OS: macOS
• kubernetes Version: v1.15.12-gke.9
• argo version: v2.9.3
• argo events version: v0.15.0

Additional context
I tried this with other custom resources and the same thing happened there, so it's not specific to workflows

[whynowy]

@jonnylangefeld - The whole argo-events architecture has been changed in v0.17.0, and this issue should have gone away. Can you please verify?

[ccronca]

It can be reproduced using the latest version of argo-events:

• kubernetes Version: v1.18.5
• argo version: v2.11.1
• argo events version: v1.0.0

The enventsource pod logs shows repeating log lines like the following:

{"level":"info","ts":1607277916.723507,"logger":"argo-events.eventsource","caller":"resource/start.go:177","msg":"detected create event","eventSourceName":"resource","eventSourceType":"resource","eventName":"example"}
{"level":"info","ts":1607277916.7707205,"logger":"argo-events.eventsource","caller":"resource/start.go:185","msg":"detected update event","eventSourceName":"resource","eventSourceType":"resource","eventName":"example"}
{"level":"info","ts":1607277917.8035023,"logger":"argo-events.eventsource","caller":"resource/start.go:185","msg":"detected update event","eventSourceName":"resource","eventSourceType":"resource","eventName":"example"}
{"level":"info","ts":1607277920.417026,"logger":"argo-events.eventsource","caller":"resource/start.go:185","msg":"detected update event","eventSourceName":"resource","eventSourceType":"resource","eventName":"example"}
{"level":"info","ts":1607277921.442843,"logger":"argo-events.eventsource","caller":"resource/start.go:185","msg":"detected update event","eventSourceName":"resource","eventSourceType":"resource","eventName":"example"}
{"level":"info","ts":1607278086.146705,"logger":"argo-events.eventsource","caller":"resource/start.go:185","msg":"detected update event","eventSourceName":"resource","eventSourceType":"resource","eventName":"example"}

[github-actions]

This issue has been automatically marked as stale because it has not had
any activity in the last 60 days. It will be closed if no further activity
occurs. Thank you for your contributions.

[whynowy]

It can be reproduced using the latest version of argo-events:

• kubernetes Version: v1.18.5
• argo version: v2.11.1
• argo events version: v1.0.0

The enventsource pod logs shows repeating log lines like the following:

{"level":"info","ts":1607277916.723507,"logger":"argo-events.eventsource","caller":"resource/start.go:177","msg":"detected create event","eventSourceName":"resource","eventSourceType":"resource","eventName":"example"}
{"level":"info","ts":1607277916.7707205,"logger":"argo-events.eventsource","caller":"resource/start.go:185","msg":"detected update event","eventSourceName":"resource","eventSourceType":"resource","eventName":"example"}
{"level":"info","ts":1607277917.8035023,"logger":"argo-events.eventsource","caller":"resource/start.go:185","msg":"detected update event","eventSourceName":"resource","eventSourceType":"resource","eventName":"example"}
{"level":"info","ts":1607277920.417026,"logger":"argo-events.eventsource","caller":"resource/start.go:185","msg":"detected update event","eventSourceName":"resource","eventSourceType":"resource","eventName":"example"}
{"level":"info","ts":1607277921.442843,"logger":"argo-events.eventsource","caller":"resource/start.go:185","msg":"detected update event","eventSourceName":"resource","eventSourceType":"resource","eventName":"example"}
{"level":"info","ts":1607278086.146705,"logger":"argo-events.eventsource","caller":"resource/start.go:185","msg":"detected update event","eventSourceName":"resource","eventSourceType":"resource","eventName":"example"}

This is normal, every time there's an event detected (even it doesn't meet the filter conditions), it prints it out, it doesn't mean there's an event been triggered.

[whynowy]

Other than that, did you see anything bad with latest Argo Events? @camilocot @jonnylangefeld

[walcruz]

Hi guys, about this I have now the same problem when the resoruce is for deployments.
I'm using event 1.7.1.

resource:
    example:
      eventTypes:
      - UPDATE
      filter:
        afterStart: true
        labels:
        - key: app
          operation: ==
          value: nginx-dep
      group: apps
      namespace: argo-events
      resource: deployments
      version: v1

If I use 1 replica this make 6 workflows and If I chose 3... 9. (In deployment yml)

TY

I see this: juliev0@846df61

And the case it's only for v1alpha1. I'm using apps/v1.