Prometheus checks failing in Camel 4

[squakez]

See https://github.com/apache/camel-k/actions/runs/6250483200/job/16992484666?pr=4762

prometheus_test.go:65: 
          	Error Trace:	/home/runner/work/camel-k/camel-k/e2e/common/traits/prometheus_test.go:65
          	Error:      	"# TYPE system_load_average_1m gauge\n# HELP system_load_average_1m The sum of the number of runnable entities queued to available processors and the number of runnable entities running on the available processors averaged over a period of time\nsystem_load_average_1m 2.1923828125\n# TYPE http_server_bytes_read summary\n# HELP http_server_bytes_read Number of bytes received by the server\nhttp_server_bytes_read_count 0.0\nhttp_server_bytes_read_sum 0.0\n# TYPE http_server_bytes_read_max gauge\n# HELP http_server_bytes_read_max Number of bytes received by the server\nhttp_server_bytes_read_max 0.0\n# TYPE camel_routes_running_routes gauge\n# HELP camel_routes_running_routes  \ncamel_routes_running_routes{camelContext=\"camel-1\",eventType=\"RouteEvent\",serviceName=\"MicrometerEventNotifierService\"} 1.0\n# TYPE jvm_threads_live_threads gauge\n# HELP jvm_threads_live_threads The current number of live threads including both daemon and non-daemon threads\njvm_threads_live_threads 13.0\n# T
  YPE system_cpu_usage gauge\n# HELP system_cpu_usage The \\\"recent cpu usage\\\" of the system the application is running in\nsystem_cpu_usage 6.681187741237206E-4\n# TYPE jvm_classes_unloaded_classes counter\n# HELP jvm_classes_unloaded_classes The total number of classes unloaded since the Java virtual machine has started execution\njvm_classes_unloaded_classes_total 0.0\n# TYPE camel_exchanges counter\n# HELP camel_exchanges Total number of processed exchanges\ncamel_exchanges_total{camelContext=\"camel-1\",routeId=\"route1\",serviceName=\"MicrometerRoutePolicyService\"} 1.0\n# TYPE jvm_gc_overhead_percent gauge\n# HELP jvm_gc_overhead_percent An approximation of the percent of CPU time used by GC activities over the last lookback period or since monitoring began, whichever is shorter, in the range [0..1]\njvm_gc_overhead_percent 0.015644135917346293\n# TYPE camel_exchanges_inflight gauge\n# HELP camel_exchanges_inflight Route inflight messages\ncamel_exchanges_inflight{camelContext=\"camel-1\",routeId=\"r
  oute1\",serviceName=\"MicrometerEventNotifierService\"} 0.0\n# TYPE process_files_max_files gauge\n# HELP process_files_max_files The maximum file descriptor count\nprocess_files_max_files 1048576.0\n# TYPE camel_exchanges_external_redeliveries counter\n# HELP camel_exchanges_external_redeliveries Number of external initiated redeliveries (such as from JMS broker)\ncamel_exchanges_external_redeliveries_total{camelContext=\"camel-1\",routeId=\"route1\",serviceName=\"MicrometerRoutePolicyService\"} 0.0\n# TYPE process_files_open_files gauge\n# HELP process_files_open_files The open file descriptor count\nprocess_files_open_files 427.0\n# TYPE jvm_gc_pause_seconds summary\n# HELP jvm_gc_pause_seconds Time spent in GC pause\njvm_gc_pause_seconds_count{action=\"end of minor GC\",cause=\"Metadata GC Threshold\",gc=\"G1 Young Generation\"} 1.0\njvm_gc_pause_seconds_sum{action=\"end of minor GC\",cause=\"Metadata GC Threshold\",gc=\"G1 Young Generation\"} 0.007\njvm_gc_pause_seconds_count{action=\"end of minor GC\",c
  ause=\"G1 Evacuation Pause\",gc=\"G1 Young Generation\"} 5.0\njvm_gc_pause_seconds_sum{action=\"end of minor GC\",cause=\"G1 Evacuation Pause\",gc=\"G1 Young Generation\"} 0.031\n# TYPE jvm_gc_pause_seconds_max gauge\n# HELP jvm_gc_pause_seconds_max Time spent in GC pause\njvm_gc_pause_seconds_max{action=\"end of minor GC\",cause=\"Metadata GC Threshold\",gc=\"G1 Young Generation\"} 0.007\njvm_gc_pause_seconds_max{action=\"end of minor GC\",cause=\"G1 Evacuation Pause\",gc=\"G1 Young Generation\"} 0.009\n# TYPE jvm_buffer_count_buffers gauge\n# HELP jvm_buffer_count_buffers An estimate of the number of buffers in the pool\njvm_buffer_count_buffers{id=\"mapped - 'non-volatile memory'\"} 0.0\njvm_buffer_count_buffers{id=\"mapped\"} 0.0\njvm_buffer_count_buffers{id=\"direct\"} 7.0\n# TYPE process_start_time_seconds gauge\n# HELP process_start_time_seconds Start time of the process since unix epoch.\nprocess_start_time_seconds 1.695285191841E9\n# TYPE camel_exchanges_failures_handled counter\n# HELP camel_exchang
  es_failures_handled Number of failures handled\ncamel_exchanges_failures_handled_total{camelContext=\"camel-1\",routeId=\"route1\",serviceName=\"MicrometerRoutePolicyService\"} 0.0\n# TYPE http_server_connections_seconds_max gauge\n# HELP http_server_connections_seconds_max The duration of the connections\nhttp_server_connections_seconds_max 0.035788216\n# TYPE http_server_connections_seconds summary\n# HELP http_server_connections_seconds The duration of the connections\nhttp_server_connections_seconds_active_count 1.0\nhttp_server_connections_seconds_duration_sum 0.035651715\n# TYPE jvm_threads_peak_threads gauge\n# HELP jvm_threads_peak_threads The peak live thread count since the Java virtual machine started or peak was reset\njvm_threads_peak_threads 13.0\n# TYPE worker_pool_queue_delay_seconds_max gauge\n# HELP worker_pool_queue_delay_seconds_max Time spent in the waiting queue before being processed\nworker_pool_queue_delay_seconds_max{pool_name=\"vert.x-internal-blocking\",pool_type=\"worker\"} 0.0\nw
  orker_pool_queue_delay_seconds_max{pool_name=\"vert.x-worker-thread\",pool_type=\"worker\"} 0.00174111\n# TYPE worker_pool_queue_delay_seconds summary\n# HELP worker_pool_queue_delay_seconds Time spent in the waiting queue before being processed\nworker_pool_queue_delay_seconds_count{pool_name=\"vert.x-internal-blocking\",pool_type=\"worker\"} 0.0\nworker_pool_queue_delay_seconds_sum{pool_name=\"vert.x-internal-blocking\",pool_type=\"worker\"} 0.0\nworker_pool_queue_delay_seconds_count{pool_name=\"vert.x-worker-thread\",pool_type=\"worker\"} 1.0\nworker_pool_queue_delay_seconds_sum{pool_name=\"vert.x-worker-thread\",pool_type=\"worker\"} 0.00174111\n# TYPE jvm_memory_used_bytes gauge\n# HELP jvm_memory_used_bytes The amount of used memory\njvm_memory_used_bytes{area=\"nonheap\",id=\"CodeHeap 'profiled nmethods'\"} 7538816.0\njvm_memory_used_bytes{area=\"heap\",id=\"G1 Survivor Space\"} 2228208.0\njvm_memory_used_bytes{area=\"heap\",id=\"G1 Old Gen\"} 4.6693888E7\njvm_memory_used_bytes{area=\"nonheap\",id=\"Me
  taspace\"} 5.0203272E7\njvm_memory_used_bytes{area=\"nonheap\",id=\"CodeHeap 'non-nmethods'\"} 1326336.0\njvm_memory_used_bytes{area=\"heap\",id=\"G1 Eden Space\"} 7340032.0\njvm_memory_used_bytes{area=\"nonheap\",id=\"Compressed Class Space\"} 6440376.0\njvm_memory_used_bytes{area=\"nonheap\",id=\"CodeHeap 'non-profiled nmethods'\"} 1551232.0\n# TYPE camel_exchanges_succeeded counter\n# HELP camel_exchanges_succeeded Number of successfully completed exchanges\ncamel_exchanges_succeeded_total{camelContext=\"camel-1\",routeId=\"route1\",serviceName=\"MicrometerRoutePolicyService\"} 1.0\n# TYPE jvm_buffer_memory_used_bytes gauge\n# HELP jvm_buffer_memory_used_bytes An estimate of the memory that the Java virtual machine is using for this buffer pool\njvm_buffer_memory_used_bytes{id=\"mapped - 'non-volatile memory'\"} 0.0\njvm_buffer_memory_used_bytes{id=\"mapped\"} 0.0\njvm_buffer_memory_used_bytes{id=\"direct\"} 2432220.0\n# TYPE jvm_threads_started_threads counter\n# HELP jvm_threads_started_threads The total
   number of application threads started in the JVM\njvm_threads_started_threads_total 14.0\n# TYPE jvm_info counter\n# HELP jvm_info JVM version info\njvm_info_total{runtime=\"OpenJDK Runtime Environment\",vendor=\"Eclipse Adoptium\",version=\"17.0.8.1+1\"} 1.0\n# TYPE jvm_memory_committed_bytes gauge\n# HELP jvm_memory_committed_bytes The amount of memory in bytes that is committed for the Java virtual machine to use\njvm_memory_committed_bytes{area=\"nonheap\",id=\"CodeHeap 'profiled nmethods'\"} 7602176.0\njvm_memory_committed_bytes{area=\"heap\",id=\"G1 Survivor Space\"} 3145728.0\njvm_memory_committed_bytes{area=\"heap\",id=\"G1 Old Gen\"} 7.4448896E7\njvm_memory_committed_bytes{area=\"nonheap\",id=\"Metaspace\"} 5.0659328E7\njvm_memory_committed_bytes{area=\"nonheap\",id=\"CodeHeap 'non-nmethods'\"} 2555904.0\njvm_memory_committed_bytes{area=\"heap\",id=\"G1 Eden Space\"} 1.572864E7\njvm_memory_committed_bytes{area=\"nonheap\",id=\"Compressed Class Space\"} 6619136.0\njvm_memory_committed_bytes{area=\"no
  nheap\",id=\"CodeHeap 'non-profiled nmethods'\"} 2555904.0\n# TYPE process_cpu_usage gauge\n# HELP process_cpu_usage The \\\"recent cpu usage\\\" for the Java Virtual Machine process\nprocess_cpu_usage 6.635049151165598E-4\n# TYPE camel_route_policy_seconds summary\n# HELP camel_route_policy_seconds Route performance metrics\ncamel_route_policy_seconds_count{camelContext=\"camel-1\",routeId=\"route1\",serviceName=\"MicrometerRoutePolicyService\"} 1.0\ncamel_route_policy_seconds_sum{camelContext=\"camel-1\",routeId=\"route1\",serviceName=\"MicrometerRoutePolicyService\"} 0.006433839\n# TYPE camel_route_policy_seconds_max gauge\n# HELP camel_route_policy_seconds_max Route performance metrics\ncamel_route_policy_seconds_max{camelContext=\"camel-1\",routeId=\"route1\",serviceName=\"MicrometerRoutePolicyService\"} 0.006433839\n# TYPE process_uptime_seconds gauge\n# HELP process_uptime_seconds The uptime of the Java virtual machine\nprocess_uptime_seconds 4.317\n# TYPE jvm_classes_loaded_classes gauge\n# HELP jvm_c
  lasses_loaded_classes The number of classes that are currently loaded in the Java virtual machine\njvm_classes_loaded_classes 9267.0\n# TYPE jvm_memory_max_bytes gauge\n# HELP jvm_memory_max_bytes The maximum amount of memory in bytes that can be used for memory management\njvm_memory_max_bytes{area=\"nonheap\",id=\"CodeHeap 'profiled nmethods'\"} 1.22912768E8\njvm_memory_max_bytes{area=\"heap\",id=\"G1 Survivor Space\"} -1.0\njvm_memory_max_bytes{area=\"heap\",id=\"G1 Old Gen\"} 1.816133632E9\njvm_memory_max_bytes{area=\"nonheap\",id=\"Metaspace\"} -1.0\njvm_memory_max_bytes{area=\"nonheap\",id=\"CodeHeap 'non-nmethods'\"} 5828608.0\njvm_memory_max_bytes{area=\"heap\",id=\"G1 Eden Space\"} -1.0\njvm_memory_max_bytes{area=\"nonheap\",id=\"Compressed Class Space\"} 1.073741824E9\njvm_memory_max_bytes{area=\"nonheap\",id=\"CodeHeap 'non-profiled nmethods'\"} 1.22916864E8\n# TYPE camel_routes_added_routes gauge\n# HELP camel_routes_added_routes  \ncamel_routes_added_routes{camelContext=\"camel-1\",eventType=\"Ro
  uteEvent\",serviceName=\"MicrometerEventNotifierService\"} 1.0\n# TYPE jvm_gc_live_data_size_bytes gauge\n# HELP jvm_gc_live_data_size_bytes Size of long-lived heap memory pool after reclamation\njvm_gc_live_data_size_bytes 0.0\n# TYPE system_cpu_count gauge\n# HELP system_cpu_count The number of processors available to the Java virtual machine\nsystem_cpu_count 2.0\n# TYPE jvm_threads_daemon_threads gauge\n# HELP jvm_threads_daemon_threads The current number of live daemon threads\njvm_threads_daemon_threads 8.0\n# TYPE camel_exchange_event_notifier_seconds_max gauge\n# HELP camel_exchange_event_notifier_seconds_max  \ncamel_exchange_event_notifier_seconds_max{camelContext=\"camel-1\",endpointName=\"timer://tick\",eventType=\"ExchangeCompletedEvent\",failed=\"false\",serviceName=\"MicrometerEventNotifierService\"} 0.008085349\n# TYPE camel_exchange_event_notifier_seconds summary\n# HELP camel_exchange_event_notifier_seconds  \ncamel_exchange_event_notifier_seconds_count{camelContext=\"camel-1\",endpointName=
  \"timer://tick\",eventType=\"ExchangeCompletedEvent\",failed=\"false\",serviceName=\"MicrometerEventNotifierService\"} 1.0\ncamel_exchange_event_notifier_seconds_sum{camelContext=\"camel-1\",endpointName=\"timer://tick\",eventType=\"ExchangeCompletedEvent\",failed=\"false\",serviceName=\"MicrometerEventNotifierService\"} 0.008085349\n# TYPE jvm_threads_states_threads gauge\n# HELP jvm_threads_states_threads The current number of threads\njvm_threads_states_threads{state=\"runnable\"} 7.0\njvm_threads_states_threads{state=\"blocked\"} 0.0\njvm_threads_states_threads{state=\"waiting\"} 3.0\njvm_threads_states_threads{state=\"timed-waiting\"} 3.0\njvm_threads_states_threads{state=\"new\"} 0.0\njvm_threads_states_threads{state=\"terminated\"} 0.0\n# TYPE worker_pool_idle gauge\n# HELP worker_pool_idle The number of resources from the pool currently used\nworker_pool_idle{pool_name=\"vert.x-internal-blocking\",pool_type=\"worker\"} 20.0\nworker_pool_idle{pool_name=\"vert.x-worker-thread\",pool_type=\"worker\"} 19.
  0\n# TYPE jvm_buffer_total_capacity_bytes gauge\n# HELP jvm_buffer_total_capacity_bytes An estimate of the total capacity of the buffers in this pool\njvm_buffer_total_capacity_bytes{id=\"mapped - 'non-volatile memory'\"} 0.0\njvm_buffer_total_capacity_bytes{id=\"mapped\"} 0.0\njvm_buffer_total_capacity_bytes{id=\"direct\"} 2432219.0\n# TYPE worker_pool_usage_seconds summary\n# HELP worker_pool_usage_seconds Time spent using resources from the pool\nworker_pool_usage_seconds_count{pool_name=\"vert.x-internal-blocking\",pool_type=\"worker\"} 0.0\nworker_pool_usage_seconds_sum{pool_name=\"vert.x-internal-blocking\",pool_type=\"worker\"} 0.0\nworker_pool_usage_seconds_count{pool_name=\"vert.x-worker-thread\",pool_type=\"worker\"} 0.0\nworker_pool_usage_seconds_sum{pool_name=\"vert.x-worker-thread\",pool_type=\"worker\"} 0.0\n# TYPE worker_pool_usage_seconds_max gauge\n# HELP worker_pool_usage_seconds_max Time spent using resources from the pool\nworker_pool_usage_seconds_max{pool_name=\"vert.x-internal-blocking\
  ",pool_type=\"worker\"} 0.0\nworker_pool_usage_seconds_max{pool_name=\"vert.x-worker-thread\",pool_type=\"worker\"} 0.0\n# TYPE worker_pool_active gauge\n# HELP worker_pool_active The number of resources from the pool currently used\nworker_pool_active{pool_name=\"vert.x-internal-blocking\",pool_type=\"worker\"} 0.0\nworker_pool_active{pool_name=\"vert.x-worker-thread\",pool_type=\"worker\"} 1.0\n# TYPE jvm_gc_max_data_size_bytes gauge\n# HELP jvm_gc_max_data_size_bytes Max size of long-lived heap memory pool\njvm_gc_max_data_size_bytes 1.816133632E9\n# TYPE jvm_gc_memory_allocated_bytes counter\n# HELP jvm_gc_memory_allocated_bytes Incremented for an increase in the size of the (young) heap memory pool after one GC to before the next\njvm_gc_memory_allocated_bytes_total 1.048576E8\n# TYPE http_server_active_requests gauge\n# HELP http_server_active_requests  \nhttp_server_active_requests 1.0\n# TYPE worker_pool_queue_size gauge\n# HELP worker_pool_queue_size Number of pending elements in the waiting queue\nw
  orker_pool_queue_size{pool_name=\"vert.x-internal-blocking\",pool_type=\"worker\"} 0.0\nworker_pool_queue_size{pool_name=\"vert.x-worker-thread\",pool_type=\"worker\"} 0.0\n# TYPE http_server_bytes_written_max gauge\n# HELP http_server_bytes_written_max Number of bytes sent by the server\nhttp_server_bytes_written_max 0.0\n# TYPE http_server_bytes_written summary\n# HELP http_server_bytes_written Number of bytes sent by the server\nhttp_server_bytes_written_count 0.0\nhttp_server_bytes_written_sum 0.0\n# TYPE worker_pool_completed counter\n# HELP worker_pool_completed Number of times resources from the pool have been acquired\nworker_pool_completed_total{pool_name=\"vert.x-internal-blocking\",pool_type=\"worker\"} 0.0\nworker_pool_completed_total{pool_name=\"vert.x-worker-thread\",pool_type=\"worker\"} 0.0\n# TYPE jvm_gc_memory_promoted_bytes counter\n# HELP jvm_gc_memory_promoted_bytes Count of positive increases in the size of the old generation memory pool before GC to after GC\njvm_gc_memory_promoted_byte
  s_total 2.4179712E7\n# TYPE camel_exchanges_failed counter\n# HELP camel_exchanges_failed Number of failed exchanges\ncamel_exchanges_failed_total{camelContext=\"camel-1\",routeId=\"route1\",serviceName=\"MicrometerRoutePolicyService\"} 0.0\n# TYPE jvm_memory_usage_after_gc_percent gauge\n# HELP jvm_memory_usage_after_gc_percent The percentage of long-lived heap pool used after the last GC event, in the range [0..1]\njvm_memory_usage_after_gc_percent{area=\"heap\",pool=\"long-lived\"} 0.02571060145424365\n# TYPE worker_pool_ratio gauge\n# HELP worker_pool_ratio Pool usage ratio\nworker_pool_ratio{pool_name=\"vert.x-internal-blocking\",pool_type=\"worker\"} NaN\nworker_pool_ratio{pool_name=\"vert.x-worker-thread\",pool_type=\"worker\"} 0.05\n# EOF\n" does not contain "CamelExchangesTotal_total"

Originally posted by @squakez in #4762 (comment)

[gansheer]

I'll take care of it