Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug: metrics cmdline does not enable metrics #3303

Closed
rafaeldtinoco opened this issue Jul 7, 2023 · 0 comments · Fixed by #3304
Closed

Bug: metrics cmdline does not enable metrics #3303

rafaeldtinoco opened this issue Jul 7, 2023 · 0 comments · Fixed by #3304

Comments

@rafaeldtinoco
Copy link
Contributor

rafaeldtinoco commented Jul 7, 2023

Description

After commit:

commit 0213c08
Author: Jose Donizetti jdbjunior@gmail.com
Date: Thu May 18 17:45:28 2023

tracee: add ready callback

The ready callback is executed once tracee is fully initialized,
and ready to receive events. This feature will be used for the
`/healthz` endpoint, which will only start to respond "OK" once
tracee is ready.

The following commandline:

$ sudo ./dist/tracee --metrics

Stopped registering prometheus stats. That happens because the prometheus metrics are registered at the end of Tracee.New() but it will fail because it depends on t.config.metricsEnabled which is enabled by the new introduced callback IF Server.metricsendpoint is also enabled (and it is only enabled after Tracee.New() is called).

I already have a patch that fixes this, will propose a PR (checking if tracee-rules metrics are being enabled).

Observations

This also brakes the make -f builder/Makefile.performance pyroscope-start dashboard (observed by Geyslan). So I'm checking what needs fixing (will need perf studies for event definitions).

@rafaeldtinoco rafaeldtinoco self-assigned this Jul 7, 2023
@rafaeldtinoco rafaeldtinoco added this to the v0.17.0 milestone Jul 7, 2023
rafaeldtinoco added a commit that referenced this issue Jul 18, 2023
Commit: 0213c08 ("tracee: add ready callback") made prometheus stats to
stop being registered due to a timing issue. This patch re-enables it
by registering prometheus counters in the callback function itself
instead of in Tracee.New() (too early).

Fixes: #3303
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant