Integration between eBPF and Apps

[thiagoftsm]

Summary

Fixes #8912
Fixes #8914

This PR brings the integration between the collectors eBPF and Apps.

Component Name

Collector

Test Plan

1 - Install Netdata using ./netdata-installer.sh --dont-start-it.
2 - Download the file files4.zip and extract it inside /usr/libexec/netdata/plugins.d.
3 - Run netdata and verify the charts on apps.plugin.
4 - Stop netdata after few seconds.
5 - Start netdata again.

Do the steps 4 and 5 at least 3 times. We should have consistency on all charts (general and apps) in all tests.

It is expected to see 14 charts inside apps submenu.

Additional Information

[ilyam8]

Neither Summary or Test plan are not clear. Part? What part?

so we expect that the reviewer will compile and verify any possible erro

What kind of errors should we expect? It depends on changes? (go to summary is not clear).

[thiagoftsm]

Neither Summary and Test plan are not clear. Part? What part?

so we expect that the reviewer will compile and verify any possible erro

What kind of errors should we expect? It depends on changes? (go to summary is not clear).

This is the motive it is [WIP] we are finalizing it before to give details ;)

[lgtm-com]

This pull request introduces 4 alerts when merging fdca546 into f3d7624 - view on LGTM.com

new alerts:

• 4 for Local variable hides global variable

[lgtm-com]

This pull request introduces 4 alerts when merging 2d04d4a into f3d7624 - view on LGTM.com

new alerts:

• 4 for Local variable hides global variable

[lgtm-com]

This pull request introduces 4 alerts when merging dc04f7f into 1aa2cd7 - view on LGTM.com

new alerts:

• 4 for Local variable hides global variable

[vlvkobal]

Why are you doing it this way? Why don't you assign function addresses in process_functions depending on STATIC macro?

[thiagoftsm]

The kickstart_static64 will need a different structure to compile, the STATIC is already used on netdata/kernel-collector when we need to drop completely the shared libraries, I am keeping this here for while to keep the same pattern used on the other repository.

[vlvkobal]

I'm not against using STATIC. I just prefer to do it in one place rather than several ones.

[lgtm-com]

This pull request introduces 1 alert when merging eb9853c into b63d58f - view on LGTM.com

new alerts:

• 1 for Local variable hides global variable

[thiagoftsm]

@vlvkobal I observed that we are having a small problem with the conditional variables. I inserted debug messages inside the function ebpf_socket_send_apps_data() and good part of the time that I was testing, the variables root_pid was cleaned, but this variable is used and set on process threads and the charts are created.

[underhood]

apps.task_close is still stuck at 0 in CentOS 8 but it works on ubuntu.

Centos:

on bubuntu it looks more healthy:

[thiagoftsm]

Why all the charts still have no titles? I keep asking it for months smile

I finally brought them, thank you to call my attention for this.

[thiagoftsm]

* Why did you make it so?

Because, this was the first collector I wrote for Netdata. ;)

* Why `update_every` is hardcoded?

A big mistake, I fixed it yesterday.

* Why there is no `plugin` and `module`?

Apps plugin also does not have at the end of the line both values, I am stopping on update every due the same reason.

netdata/collectors/apps.plugin/apps_plugin.c

Line 3202 in ba350b7

"CHART netdata.apps_cpu '' 'Apps Plugin CPU' 'milliseconds/s' apps.plugin netdata.apps_cpu stacked 140000 %1$d\n"

[thiagoftsm]

Zero charts:
Arch - 5.6.15
* file_open

I could not reproduct this problem when I tested on kernel 5.6.17.

* vfs_write_bytes
* vfs_read_bytes
* thread_create
* task_close (it works in general charts)

We really had a problem for entry mode with these charts, the fix that we brought with the release 0.4.3 fixed them.

Ubuntu 18.04 - 4.15.0
Ubuntu 20.04 - 5.4.0
CentOS 7 - 3.10.0
CentOS 8 - 4.18.0
Debian 10 - 4.19.0
Fedora 30 - 5.0.9
OpenSUSE Leap 15.1 - 4.12.14

* vfs_write_bytes
* vfs_read_bytes
* task_close (it works in general charts, except CentOS 8 and Debian 10)

The same thing happens on these distribution, they were affected for the same bug.

@vlvkobal , please, can you test them again? I am also testing on other kernels here.

[thiagoftsm]

apps.task_close is still stuck at 0 in CentOS 8 but it works on ubuntu.

CentOS 8 and Ubuntu have special kernels. For CentOS 8 we have a specific script to compile for it, I will take a look on it now.

[underhood]

@vlvkobal i am not sure we should test for Fed 30 as it is EOL soon

[underhood]

Fedora 32 AMD64 apps.file_open seems not right

Same chart from Ubuntu where it seems to be working right for comparison:

[vlvkobal]

On Arch 5.6.15, Ubuntu 20.04 apps.file_deleted, apps.vfs_read_call, apps.vfs_read_bytes, apps.process_create, apps.task_close are randomly accumulating some data. There were two CMake builds of Netdata:

[thiagoftsm]

On Arch 5.6.15, Ubuntu 20.04 apps.file_deleted, apps.vfs_read_call, apps.vfs_read_bytes, apps.process_create, apps.task_close are randomly accumulating some data. There were two CMake builds of Netdata:

The problem was fixed with the last commit.

[squash-labs]

Manage this branch in Squash

Test this branch here: https://thiagoftsmebpf-apps-ox6u7.squash.io