feat(inputs.icinga2): Support collecting hosts,services and endpoint metrics

[deric]

• Updated associated README.md.
• Wrote appropriate unit tests.
• Pull request title or commits are in conventional commit format

This PR adds possibility to collect both hosts and services metrics from Icinga 2 API at the same time. Furthermore it adds possibility to parse Icinga2 endpoint metrics. These metrics are crucial for evaluating bottlenecks in Icinga2 setup.

Currently only partial stats endpoints support:

• /v1/status/ApiListener
• /v1/status/CIB
• /v1/status/IdoMysqlConnection
• /v1/status/IdoPgsqlConnection

Test output:

> icinga2_status,component=CIB,host=test active_host_checks=4.483333333333333,active_host_checks_15min=3496,
active_host_checks_1min=269,active_host_checks_5min=1191,active_service_checks=620.85,
active_service_checks_15min=579580,active_service_checks_1min=37251,
active_service_checks_5min=192239,avg_execution_time=0.03036015721774434,
avg_latency=0.000002201153386722349,current_concurrent_checks=1,
current_pending_callbacks=0,max_execution_time=38.768075942993164,
max_latency=0.0030307769775390625,min_execution_time=0,
min_latency=0,num_hosts_acknowledged=0,num_hosts_down=0,
num_hosts_flapping=0,num_hosts_handled=0,num_hosts_in_downtime=0,
num_hosts_pending=0,num_hosts_problem=0,num_hosts_unreachable=0,
num_hosts_up=457,num_services_acknowledged=0,num_services_critical=58,
num_services_flapping=0,num_services_handled=0,num_services_in_downtime=0,
num_services_ok=114404,num_services_pending=1,num_services_problem=1189,
num_services_unknown=184,num_services_unreachable=0,num_services_warning=948,
passive_host_checks=0,passive_host_checks_15min=0,passive_host_checks_1min=0,
passive_host_checks_5min=0,passive_service_checks=0,passive_service_checks_15min=0,
passive_service_checks_1min=0,passive_service_checks_5min=0,remote_check_queue=0,
uptime=314454.3722131252 1673859991000000000
> icinga2_status,component=IdoPgsqlConnection,host=test pgsql_queries_15mins=588245,pgsql_queries_1min=38875,
pgsql_queries_5mins=195997,pgsql_queries_rate=647.9166666666666,
pgsql_query_queue_item_rate=1291.1333333333334,pgsql_query_queue_items=0 1673859991000000000

[powersj]

Thanks for taking the time to put a PR. I think the big things are 1) we have a deprecation framework that you should use and 2) the default values need to be consistent and documented.

Once you make some updates we can give it a more complete review

[powersj]

Thanks for the follow-up, just some final small items.

[deric]

@powersj Thanks, let me know if you need more changes. The CI failures should be unrelated.

[powersj]

Thank you!

[srebhan]

This looks quite nice @deric, however I do have a few comments. Can you please take a look?

[deric]

@srebhan @Hipska Thanks for review! Hopefully all issues have been addressed.

[srebhan]

@deric I do have two cosmetic suggestions/comments. Let me know if you want to apply those. Otherwise I will merge in about 7h...

[telegraf-tiger]

Download PR build artifacts for linux_amd64.tar.gz, darwin_amd64.tar.gz, and windows_amd64.zip.
Downloads for additional architectures and packages are available below.

☺️ This pull request doesn't significantly change the Telegraf binary size (less than 1%)

📦 Click here to get additional PR build artifacts

Artifact URLs

DEB	RPM	TAR GZ	ZIP
amd64.deb	aarch64.rpm	darwin_amd64.tar.gz	windows_amd64.zip
arm64.deb	armel.rpm	darwin_arm64.tar.gz	windows_arm64.zip
armel.deb	armv6hl.rpm	freebsd_amd64.tar.gz	windows_i386.zip
armhf.deb	i386.rpm	freebsd_armv7.tar.gz
i386.deb	ppc64le.rpm	freebsd_i386.tar.gz
mips.deb	riscv64.rpm	linux_amd64.tar.gz
mipsel.deb	s390x.rpm	linux_arm64.tar.gz
ppc64el.deb	x86_64.rpm	linux_armel.tar.gz
riscv64.deb		linux_armhf.tar.gz
s390x.deb		linux_i386.tar.gz
		linux_mips.tar.gz
		linux_mipsel.tar.gz
		linux_ppc64le.tar.gz
		linux_riscv64.tar.gz
		linux_s390x.tar.gz
		static_linux_amd64.tar.gz

[srebhan]

Looks good to me. Thanks for your contribution @deric!