Verbose Logs — No configuration option?

[jonwest]

Describe the bug
I'm trying to set up a simple event source to watch for when argo workflows are added, changed, or deleted, but I have to feel like I'm doing something wrong here. If I set up an EventSource like:

apiVersion: argoproj.io/v1alpha1
kind: EventSource
metadata:
  name: watch-workflows
  namespace: argo-events
spec:
  resource:
    workflows-changed:
      namespace: argo
      group: argoproj.io
      version: v1alpha1
      resource: workflows
      eventTypes:
        - ADD
        - UPDATE
        - DELETE
      filter:
        prefix: "my-prefix-"

Intuitively I would think that this is going to watch the argo namespace for workflow resources with a prefix of my-prefix- that are added, updated, or deleted, right? And ... I mean, it does, but the logs don't seem to make any sense.

I would expect to see log hits for when argo events detects changes that match the filter, but what I'm seeing is that argo events is logging every single workflow resource that it detects, with the first log entry saying it's detected an event with a workflow, and then another entry saying "Oh, that event didn't match the filter".

As a result, on a busy argo workflows server, this eventsource is emitting over 10k log entries in a matter of minutes as each workflow within the namespace is modified.

...I've gotta be doing something wrong, right? I could see this on a debug log level, but for info it feels pretty verbose, no?

I can see in the CLI I should be able to run argo-events --help controller but to be honest none of the commands that I've tried in order to get more information about the controller command seem to give me anything. I can see here that there's a klogLevel param that can be set, but it already appears to be the least verbose option.

Is it expected behaviour that at an info logging level that full, unmatched events are logged, but that matching events just have an entry that says something along the lines of "Hey, found a match! Executing the trigger!" and no further details?

Expected behavior

• I would expect that configuring logging levels would make a difference to the amount of logs generated. Maybe a debug level would include a full copy of the event, but for something like warning or info, I would expect to see maybe a "event with ID <ID,URL,ETC> was received but not matched" with no further details.

Screenshots
If applicable, add screenshots to help explain your problem.

Environment (please complete the following information):

• Kubernetes: 1.27.3
• Argo Events: 1.8.0

---

Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

[joepke]

Related: #2964