Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

in_node_exporter_metrics: Implement NVMe metrics #8049

Merged

Conversation

cosmo0920
Copy link
Contributor

@cosmo0920 cosmo0920 commented Oct 17, 2023

In the official Prometheus exporter, NVMe metrics is provided as a default collecting collector.
Then, we should provide it in fluent-bit. This is because non numerical metrics of NVMe is not collected other types of metrics on node_exporter_metrics.

Closes #7888


Enter [N/A] in the box, if an item is not applicable to your change.

Testing
Before we can approve your change; please submit the following in a comment:

  • Example configuration file for the change
$ bin/fluent-bit -i node_exporter_metrics -p metrics=nvme -o stdout -v
  • Debug log output from testing the change
Fluent Bit v2.2.0
* Copyright (C) 2015-2023 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

[2023/10/17 11:15:47] [ info] Configuration:
[2023/10/17 11:15:47] [ info]  flush time     | 1.000000 seconds
[2023/10/17 11:15:47] [ info]  grace          | 5 seconds
[2023/10/17 11:15:47] [ info]  daemon         | 0
[2023/10/17 11:15:47] [ info] ___________
[2023/10/17 11:15:47] [ info]  inputs:
[2023/10/17 11:15:47] [ info]      node_exporter_metrics
[2023/10/17 11:15:47] [ info] ___________
[2023/10/17 11:15:47] [ info]  filters:
[2023/10/17 11:15:47] [ info] ___________
[2023/10/17 11:15:47] [ info]  outputs:
[2023/10/17 11:15:47] [ info]      stdout.0
[2023/10/17 11:15:47] [ info] ___________
[2023/10/17 11:15:47] [ info]  collectors:
[2023/10/17 11:15:47] [ info] [fluent bit] version=2.2.0, commit=645540f60d, pid=4145677
[2023/10/17 11:15:47] [debug] [engine] coroutine stack size: 24576 bytes (24.0K)
[2023/10/17 11:15:47] [ info] [storage] ver=1.1.6, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2023/10/17 11:15:47] [ info] [cmetrics] version=0.6.3
[2023/10/17 11:15:47] [ info] [ctraces ] version=0.3.1
[2023/10/17 11:15:47] [ info] [input:node_exporter_metrics:node_exporter_metrics.0] initializing
[2023/10/17 11:15:47] [ info] [input:node_exporter_metrics:node_exporter_metrics.0] storage_strategy='memory' (memory only)
[2023/10/17 11:15:47] [ info] [input:node_exporter_metrics:node_exporter_metrics.0] path.procfs = /proc
[2023/10/17 11:15:47] [ info] [input:node_exporter_metrics:node_exporter_metrics.0] path.sysfs  = /sys
[2023/10/17 11:15:47] [debug] [input:node_exporter_metrics:node_exporter_metrics.0] enabled metrics nvme
[2023/10/17 11:15:47] [debug] [input:node_exporter_metrics:node_exporter_metrics.0] [thread init] initialization OK
[2023/10/17 11:15:47] [ info] [input:node_exporter_metrics:node_exporter_metrics.0] thread instance initialized
[2023/10/17 11:15:47] [debug] [node_exporter_metrics:node_exporter_metrics.0] created event channels: read=30 write=31
[2023/10/17 11:15:47] [debug] [stdout:stdout.0] created event channels: read=34 write=35
[2023/10/17 11:15:47] [ info] [sp] stream processor started
[2023/10/17 11:15:47] [ info] [output:stdout:stdout.0] worker #0 started
[2023/10/17 11:15:51] [debug] [input chunk] update output instances with new chunk size diff=448, records=0, input=node_exporter_metrics.0
[2023/10/17 11:15:52] [debug] [task] created task=0x7ff044021970 id=0 OK
[2023/10/17 11:15:52] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
2023-10-17T02:15:51.675829985Z node_nvme_info{device="nvme0",firmware_revision="5B2QGXA7",model="Samsung SSD 980 PRO 1TB",serial="S5GXNF0T324173Y",state="live"} = 1
2023-10-17T02:15:51.675829985Z node_nvme_info{device="nvme1",firmware_revision="3B7QCXE7",model="Samsung SSD 960 EVO 500GB",serial="S3X4NB0JC07119X",state="live"} = 1
[2023/10/17 11:15:52] [debug] [out flush] cb_destroy coro_id=0
[2023/10/17 11:15:52] [debug] [task] destroy task=0x7ff044021970 (task_id=0)
^C[2023/10/17 11:15:53] [engine] caught signal (SIGINT)
[2023/10/17 11:15:53] [ warn] [engine] service will shutdown in max 5 seconds
[2023/10/17 11:15:53] [debug] [input:node_exporter_metrics:node_exporter_metrics.0] thread pause instance
[2023/10/17 11:15:53] [ info] [engine] service has stopped (0 pending tasks)
[2023/10/17 11:15:53] [debug] [input:node_exporter_metrics:node_exporter_metrics.0] thread pause instance
[2023/10/17 11:15:53] [ info] [output:stdout:stdout.0] thread worker #0 stopping...
[2023/10/17 11:15:53] [ info] [output:stdout:stdout.0] thread worker #0 stopped
[2023/10/17 11:15:53] [debug] [input:node_exporter_metrics:node_exporter_metrics.0] thread exit instance
  • Attached Valgrind output that shows no leaks or memory corruption was found
==4145841== 
==4145841== HEAP SUMMARY:
==4145841==     in use at exit: 0 bytes in 0 blocks
==4145841==   total heap usage: 3,215 allocs, 3,215 frees, 951,497 bytes allocated
==4145841== 
==4145841== All heap blocks were freed -- no leaks are possible
==4145841== 
==4145841== For lists of detected and suppressed errors, rerun with: -s
==4145841== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

  • Run local packaging test showing all targets (including any new ones) build.
  • Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

  • Documentation required for this feature

fluent/fluent-bit-docs#1235

Backporting

  • Backport to latest stable release.

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

@cosmo0920 cosmo0920 temporarily deployed to pr October 17, 2023 02:17 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr October 17, 2023 02:17 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr October 17, 2023 02:17 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr October 17, 2023 02:45 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 force-pushed the cosmo0920-implement-nvme-metrics-on-node_exporter_metrics branch from c7eb188 to 6070107 Compare October 17, 2023 06:15
@cosmo0920 cosmo0920 temporarily deployed to pr October 17, 2023 06:15 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr October 17, 2023 06:15 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr October 17, 2023 06:15 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 temporarily deployed to pr October 17, 2023 06:45 — with GitHub Actions Inactive
@cosmo0920 cosmo0920 marked this pull request as ready for review October 17, 2023 07:57
@edsiper edsiper added this to the Fluent Bit v2.2.0 milestone Oct 17, 2023
@cosmo0920 cosmo0920 force-pushed the cosmo0920-implement-nvme-metrics-on-node_exporter_metrics branch from 6070107 to e728baf Compare November 5, 2023 13:23
@cosmo0920
Copy link
Contributor Author

cosmo0920 commented Nov 5, 2023

I rebased off master. Hmm..., some of the unit tests for filter plugins are failed...

Signed-off-by: Hiroshi Hatake <hatake@calyptia.com>
@cosmo0920 cosmo0920 force-pushed the cosmo0920-implement-nvme-metrics-on-node_exporter_metrics branch from e728baf to e56c0dc Compare November 6, 2023 18:06
@edsiper edsiper merged commit 880c715 into master Nov 7, 2023
44 checks passed
@edsiper edsiper deleted the cosmo0920-implement-nvme-metrics-on-node_exporter_metrics branch November 7, 2023 03:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement nvme metrics on node_exporter_metrics
2 participants