Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(metrics): sync libs metrics collector refactor #3129

Merged
merged 4 commits into from Mar 14, 2024

Conversation

incertum
Copy link
Contributor

@incertum incertum commented Mar 6, 2024

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

/kind release

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area build

/area engine

/area tests

/area proposals

/area CI

What this PR does / why we need it:

Sync libs

Includes a handful of naming changes. A notice to https://falco.org/docs/metrics/falco-metrics/ will be added well in advance of Falco 0.38.0.

falco.hostname -> evt.hostname to be consistent with the newer evt.hostname filtercheck
cpu_usage_perc_total_host -> host_cpu_usage_perc
memory_used_host -> host_memory_used_kb (or host_memory_used_mb)
procs_running_host -> host_procs_running
open_fds_host -> host_open_fds
memory_rss -> memory_rss_kb (or memory_rss_mb)
memory_pss -> memory_pss_kb (or memory_pss_mb)
memory_vsz -> memory_vsz_kb (or memory_vsz_mb)
container_memory_used -> container_memory_used_bytes (or container_memory_used_mb)

Which issue(s) this PR fixes:

Fixes #

falcosecurity/libs#1463

Requires merged falcosecurity/libs#1652 plus a libs bump.

Special notes for your reviewer:

Does this PR introduce a user-facing change?:

feat(metrics): coherent metrics stats model  including few metrics naming changes

@incertum
Copy link
Contributor Author

incertum commented Mar 6, 2024

/milestone 0.38.0

@poiana poiana requested review from Kaizhe and LucaGuerra March 6, 2024 06:34
@poiana poiana added the approved label Mar 6, 2024
@poiana poiana added this to the 0.38.0 milestone Mar 6, 2024
@poiana poiana added the size/L label Mar 6, 2024
@incertum
Copy link
Contributor Author

incertum commented Mar 6, 2024

New example output:

{
  "hostname": "test",
  "output": "Falco metrics snapshot",
  "output_fields": {
    "evt.hostname": "test",
    "evt.source": "syscall",
    "evt.time": 1709705243681512818,
    "falco.container_memory_used_mb": 0.0,
    "falco.cpu_usage_perc": 1.6,
    "falco.duration_sec": 144,
    "falco.evts_rate_sec": 126065.4,
    "falco.host_boot_ts": 1708753667000000000,
    "falco.host_cpu_usage_perc": 2.3,
    "falco.host_memory_used_mb": 10169.4,
    "falco.host_num_cpus": 20,
    "falco.host_open_fds": 21272,
    "falco.host_procs_running": 3,
    "falco.kernel_release": "6.6.7-200.fc39.x86_64",
    "falco.memory_pss_mb": 135.8,
    "falco.memory_rss_mb": 138.0,
    "falco.memory_vsz_mb": 1125.3,
    "falco.n_added_fds": 287251,
    "falco.n_added_threads": 2479,
    "falco.n_cached_fd_lookups": 2799364,
    "falco.n_cached_thread_lookups": 3202933,
    "falco.n_containers": 2,
    "falco.n_drops_full_threadtable": 0,
    "falco.n_failed_fd_lookups": 21283,
    "falco.n_failed_thread_lookups": 4297,
    "falco.n_fds": 147123,
    "falco.n_missing_container_images": 0,
    "falco.n_noncached_fd_lookups": 567716,
    "falco.n_noncached_thread_lookups": 444988,
    "falco.n_removed_fds": 189764,
    "falco.n_removed_threads": 706,
    "falco.n_retrieve_evts_drops": 693392,
    "falco.n_retrieved_evts": 249868,
    "falco.n_store_evts_drops": 0,
    "falco.n_stored_evts": 251335,
    "falco.n_threads": 1772,
    "falco.num_evts": 3397379,
    "falco.num_evts_prev": 3271130,
    "falco.outputs_queue_num_drops": 0,
    "falco.start_ts": 1709705099677741260,
    "falco.version": "0.38.0-121+5185f15",
    "scap.engine_name": "bpf",
    "scap.evts_drop_rate_sec": 0.0,
    "scap.evts_rate_sec": 132035.8,
    "scap.n_drops": 0,
    "scap.n_drops_buffer_clone_fork_enter": 0,
    "scap.n_drops_buffer_clone_fork_exit": 0,
    "scap.n_drops_buffer_close_exit": 0,
    "scap.n_drops_buffer_connect_enter": 0,
    "scap.n_drops_buffer_connect_exit": 0,
    "scap.n_drops_buffer_dir_file_enter": 0,
    "scap.n_drops_buffer_dir_file_exit": 0,
    "scap.n_drops_buffer_execve_enter": 0,
    "scap.n_drops_buffer_execve_exit": 0,
    "scap.n_drops_buffer_open_enter": 0,
    "scap.n_drops_buffer_open_exit": 0,
    "scap.n_drops_buffer_other_interest_enter": 0,
    "scap.n_drops_buffer_other_interest_exit": 0,
    "scap.n_drops_buffer_proc_exit": 0,
    "scap.n_drops_buffer_total": 0,
    "scap.n_drops_bug": 0,
    "scap.n_drops_page_faults": 0,
    "scap.n_drops_perc": 0.0,
    "scap.n_drops_prev": 0,
    "scap.n_drops_scratch_map": 0,
    "scap.n_evts": 3436380,
    "scap.n_evts_prev": 3304152,
    "scap.page_fault_kern.avg_time_ns": 0,
    "scap.page_fault_kern.run_cnt": 0,
    "scap.page_fault_kern.run_time_ns": 0,
    "scap.page_fault_user.avg_time_ns": 0,
    "scap.page_fault_user.run_cnt": 0,
    "scap.page_fault_user.run_time_ns": 0,
    "scap.sched_process_e.avg_time_ns": 2066,
    "scap.sched_process_e.run_cnt": 719,
    "scap.sched_process_e.run_time_ns": 1485896,
    "scap.sched_switch.avg_time_ns": 0,
    "scap.sched_switch.run_cnt": 0,
    "scap.sched_switch.run_time_ns": 0,
    "scap.signal_deliver.avg_time_ns": 0,
    "scap.signal_deliver.run_cnt": 0,
    "scap.signal_deliver.run_time_ns": 0,
    "scap.sys_enter.avg_time_ns": 117,
    "scap.sys_enter.run_cnt": 7641969,
    "scap.sys_enter.run_time_ns": 896903922,
    "scap.sys_exit.avg_time_ns": 123,
    "scap.sys_exit.run_cnt": 7642055,
    "scap.sys_exit.run_time_ns": 946918695
  },
  "priority": "Informational",
  "rule": "Falco internal: metrics snapshot",
  "source": "internal",
  "time": "2024-03-06T06:07:23.681512818Z"
}

@incertum
Copy link
Contributor Author

incertum commented Mar 6, 2024

Website update PR: falcosecurity/falco-website#1286

@incertum
Copy link
Contributor Author

incertum commented Mar 8, 2024

To the reviewers: libs PR should be ready soon ...

See #3131 -> There needs to be a significant refactor to expand the Falco metrics following our libs refactor. Therefore, I would regard this PR as merely intending to sync libs and assume that the falco metrics interface will be refactored very soon anyway.

CC @sgaist

Includes a handful of naming changes. A notice to
https://falco.org/docs/metrics/falco-metrics/ will be added well
in advance of Falco 0.38.0

falco.hostname -> evt.hostname to be consistent with the newer evt.hostname filtercheck
cpu_usage_perc_total_host -> host_cpu_usage_perc
memory_used_host -> host_memory_used_kb (or host_memory_used_mb)
procs_running_host -> host_procs_running
open_fds_host -> host_open_fds

memory_rss -> memory_rss_kb (or memory_rss_mb)
memory_pss -> memory_pss_kb (or memory_pss_mb)
memory_vsz -> memory_vsz_kb (or memory_vsz_mb)
container_memory_used -> container_memory_used_bytes (or container_memory_used_mb)

Signed-off-by: Melissa Kilby <melissa.kilby.oss@gmail.com>
Signed-off-by: Melissa Kilby <melissa.kilby.oss@gmail.com>
@incertum incertum changed the title wip: feat(metrics): sync libs metrics collector refactor feat(metrics): sync libs metrics collector refactor Mar 13, 2024
Signed-off-by: Melissa Kilby <melissa.kilby.oss@gmail.com>
Signed-off-by: Federico Di Pierro <nierro92@gmail.com>
@FedeDP
Copy link
Contributor

FedeDP commented Mar 14, 2024

Bumped libs to 0.15.1 and rebased on top of master.

Copy link
Member

@Andreagit97 Andreagit97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@poiana
Copy link

poiana commented Mar 14, 2024

LGTM label has been added.

Git tree hash: 097f6511db698823ccbe101c5048f1d596c94846

Copy link
Contributor

@FedeDP FedeDP left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/approve

@poiana
Copy link

poiana commented Mar 14, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Andreagit97, FedeDP, incertum

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:
  • OWNERS [Andreagit97,FedeDP,incertum]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@poiana poiana merged commit 5fe9fc9 into falcosecurity:master Mar 14, 2024
26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

4 participants