2024-04-29 11:54:56,596 INFO [kafka-admin-client-thread | kafka-ui-admin-1714391696-80] o.a.k.c.u.AppInfoParser: App info kafka.admin.client for kafka-ui-admin-1714391696-80 unregistered 2024-04-29 11:54:56,597 INFO [kafka-admin-client-thread | kafka-ui-admin-1714391696-80] o.a.k.c.a.i.AdminMetadataManager: [AdminClient clientId=kafka-ui-admin-1714391696-80] Metadata update failed org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: fetchMetadata 2024-04-29 11:54:56,597 INFO [kafka-admin-client-thread | kafka-ui-admin-1714391696-80] o.a.k.c.a.KafkaAdminClient: [AdminClient clientId=kafka-ui-admin-1714391696-80] Timed out 2 remaining operation(s) during close. 2024-04-29 11:54:56,598 INFO [kafka-admin-client-thread | kafka-ui-admin-1714391696-80] o.a.k.c.m.Metrics: Metrics scheduler closed 2024-04-29 11:54:56,599 INFO [kafka-admin-client-thread | kafka-ui-admin-1714391696-80] o.a.k.c.m.Metrics: Closing reporter org.apache.kafka.common.metrics.JmxReporter 2024-04-29 11:54:56,599 INFO [kafka-admin-client-thread | kafka-ui-admin-1714391696-80] o.a.k.c.m.Metrics: Metrics reporters closed 2024-04-29 11:54:56,599 ERROR [kafka-admin-client-thread | kafka-ui-admin-1714391696-80] o.a.k.c.u.KafkaThread: Uncaught exception in thread 'kafka-admin-client-thread | kafka-ui-admin-1714391696-80': java.lang.OutOfMemoryError: Java heap space 2024-04-29 11:54:56,600 ERROR [parallel-1] c.p.k.u.s.StatisticsService: Failed to collect cluster lacp-development-msk info java.lang.IllegalStateException: Error while creating AdminClient for Cluster lacp-development-msk at com.provectus.kafka.ui.service.AdminClientServiceImpl.lambda$createAdminClient$5(AdminClientServiceImpl.java:56) at reactor.core.publisher.Mono.lambda$onErrorMap$28(Mono.java:3783) at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94) at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onError(MonoPeekTerminal.java:258) at reactor.core.publisher.FluxMap$MapConditionalSubscriber.onError(FluxMap.java:265) at reactor.core.publisher.Operators$MonoSubscriber.onError(Operators.java:1886) at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.signalCached(MonoCacheTime.java:340) at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.onError(MonoCacheTime.java:363) at reactor.core.publisher.MonoFlatMap$FlatMapMain.onError(MonoFlatMap.java:180) at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onError(MonoIgnoreThen.java:278) at reactor.core.publisher.MonoPublishOn$PublishOnSubscriber.run(MonoPublishOn.java:187) at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68) at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: org.apache.kafka.common.errors.TimeoutException: The AdminClient thread has exited. Call: listNodes 2024-04-29 11:54:56,601 DEBUG [parallel-1] c.p.k.u.s.ClustersStatisticsScheduler: Metrics updated for cluster: lacp-development-msk [root@ip-172-6-48-126 ~]# docker logs --tail=100 c6301f32567c sasl.oauthbearer.expected.audience = null sasl.oauthbearer.expected.issuer = null sasl.oauthbearer.jwks.endpoint.refresh.ms = 3600000 sasl.oauthbearer.jwks.endpoint.retry.backoff.max.ms = 10000 sasl.oauthbearer.jwks.endpoint.retry.backoff.ms = 100 sasl.oauthbearer.jwks.endpoint.url = null sasl.oauthbearer.scope.claim.name = scope sasl.oauthbearer.sub.claim.name = sub sasl.oauthbearer.token.endpoint.url = null security.protocol = SASL_PLAINTEXT security.providers = null send.buffer.bytes = 131072 socket.connection.setup.timeout.max.ms = 30000 socket.connection.setup.timeout.ms = 10000 ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2, TLSv1.3] ssl.endpoint.identification.algorithm = https ssl.engine.factory.class = null ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.certificate.chain = null ssl.keystore.key = null ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLSv1.3 ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.certificates = null ssl.truststore.location = null ssl.truststore.password = null ssl.truststore.type = JKS 2024-04-29 12:20:54,863 INFO [parallel-1] o.a.k.c.s.a.AbstractLogin: Successfully logged in. 2024-04-29 12:20:54,866 INFO [parallel-1] o.a.k.c.u.AppInfoParser: Kafka version: 3.5.0 2024-04-29 12:20:54,870 INFO [parallel-1] o.a.k.c.u.AppInfoParser: Kafka commitId: c97b88d5db4de28d 2024-04-29 12:20:54,871 INFO [parallel-1] o.a.k.c.u.AppInfoParser: Kafka startTimeMs: 1714393254866 2024-04-29 12:20:55,072 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393254-137] o.a.k.c.u.AppInfoParser: App info kafka.admin.client for kafka-ui-admin-1714393254-137 unregistered 2024-04-29 12:20:55,077 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393254-137] o.a.k.c.a.i.AdminMetadataManager: [AdminClient clientId=kafka-ui-admin-1714393254-137] Metadata update failed org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: fetchMetadata 2024-04-29 12:20:55,077 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393254-137] o.a.k.c.a.KafkaAdminClient: [AdminClient clientId=kafka-ui-admin-1714393254-137] Timed out 2 remaining operation(s) during close. 2024-04-29 12:20:55,079 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393254-137] o.a.k.c.m.Metrics: Metrics scheduler closed 2024-04-29 12:20:55,081 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393254-137] o.a.k.c.m.Metrics: Closing reporter org.apache.kafka.common.metrics.JmxReporter 2024-04-29 12:20:55,081 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393254-137] o.a.k.c.m.Metrics: Metrics reporters closed 2024-04-29 12:20:55,082 ERROR [kafka-admin-client-thread | kafka-ui-admin-1714393254-137] o.a.k.c.u.KafkaThread: Uncaught exception in thread 'kafka-admin-client-thread | kafka-ui-admin-1714393254-137': java.lang.OutOfMemoryError: Java heap space 2024-04-29 12:20:55,082 ERROR [parallel-1] c.p.k.u.s.StatisticsService: Failed to collect cluster lacp-development-msk info java.lang.IllegalStateException: Error while creating AdminClient for Cluster lacp-development-msk at com.provectus.kafka.ui.service.AdminClientServiceImpl.lambda$createAdminClient$5(AdminClientServiceImpl.java:56) at reactor.core.publisher.Mono.lambda$onErrorMap$28(Mono.java:3783) at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94) at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onError(MonoPeekTerminal.java:258) at reactor.core.publisher.FluxMap$MapConditionalSubscriber.onError(FluxMap.java:265) at reactor.core.publisher.Operators$MonoSubscriber.onError(Operators.java:1886) at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.signalCached(MonoCacheTime.java:340) at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.onError(MonoCacheTime.java:363) at reactor.core.publisher.MonoFlatMap$FlatMapMain.onError(MonoFlatMap.java:180) at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onError(MonoIgnoreThen.java:278) at reactor.core.publisher.MonoPublishOn$PublishOnSubscriber.run(MonoPublishOn.java:187) at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68) at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: org.apache.kafka.common.errors.TimeoutException: The AdminClient thread has exited. Call: listNodes 2024-04-29 12:20:55,082 DEBUG [parallel-1] c.p.k.u.s.ClustersStatisticsScheduler: Metrics updated for cluster: lacp-development-msk 2024-04-29 12:20:55,270 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393254-138] o.a.k.c.u.AppInfoParser: App info kafka.admin.client for kafka-ui-admin-1714393254-138 unregistered 2024-04-29 12:20:55,271 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393254-138] o.a.k.c.a.i.AdminMetadataManager: [AdminClient clientId=kafka-ui-admin-1714393254-138] Metadata update failed org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: fetchMetadata 2024-04-29 12:20:55,272 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393254-138] o.a.k.c.a.KafkaAdminClient: [AdminClient clientId=kafka-ui-admin-1714393254-138] Timed out 2 remaining operation(s) during close. 2024-04-29 12:20:55,273 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393254-138] o.a.k.c.m.Metrics: Metrics scheduler closed 2024-04-29 12:20:55,274 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393254-138] o.a.k.c.m.Metrics: Closing reporter org.apache.kafka.common.metrics.JmxReporter 2024-04-29 12:20:55,274 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393254-138] o.a.k.c.m.Metrics: Metrics reporters closed 2024-04-29 12:20:55,275 ERROR [kafka-admin-client-thread | kafka-ui-admin-1714393254-138] o.a.k.c.u.KafkaThread: Uncaught exception in thread 'kafka-admin-client-thread | kafka-ui-admin-1714393254-138': java.lang.OutOfMemoryError: Java heap space 2024-04-29 12:20:55,276 ERROR [parallel-1] c.p.k.u.s.StatisticsService: Failed to collect cluster test info java.lang.IllegalStateException: Error while creating AdminClient for Cluster test at com.provectus.kafka.ui.service.AdminClientServiceImpl.lambda$createAdminClient$5(AdminClientServiceImpl.java:56) [root@ip-172-6-48-126 ~]# docker logs --tail=100 c6301f32567c sasl.oauthbearer.expected.audience = null sasl.oauthbearer.expected.issuer = null sasl.oauthbearer.jwks.endpoint.refresh.ms = 3600000 sasl.oauthbearer.jwks.endpoint.retry.backoff.max.ms = 10000 sasl.oauthbearer.jwks.endpoint.retry.backoff.ms = 100 sasl.oauthbearer.jwks.endpoint.url = null sasl.oauthbearer.scope.claim.name = scope sasl.oauthbearer.sub.claim.name = sub sasl.oauthbearer.token.endpoint.url = null security.protocol = SASL_PLAINTEXT security.providers = null send.buffer.bytes = 131072 socket.connection.setup.timeout.max.ms = 30000 socket.connection.setup.timeout.ms = 10000 ssl.cipher.suites = null ssl.enabled.protocols = [TLSv1.2, TLSv1.3] ssl.endpoint.identification.algorithm = https ssl.engine.factory.class = null ssl.key.password = null ssl.keymanager.algorithm = SunX509 ssl.keystore.certificate.chain = null ssl.keystore.key = null ssl.keystore.location = null ssl.keystore.password = null ssl.keystore.type = JKS ssl.protocol = TLSv1.3 ssl.provider = null ssl.secure.random.implementation = null ssl.trustmanager.algorithm = PKIX ssl.truststore.certificates = null ssl.truststore.location = null ssl.truststore.password = null ssl.truststore.type = JKS 2024-04-29 12:21:24,864 INFO [parallel-1] o.a.k.c.s.a.AbstractLogin: Successfully logged in. 2024-04-29 12:21:24,867 INFO [parallel-1] o.a.k.c.u.AppInfoParser: Kafka version: 3.5.0 2024-04-29 12:21:24,874 INFO [parallel-1] o.a.k.c.u.AppInfoParser: Kafka commitId: c97b88d5db4de28d 2024-04-29 12:21:24,874 INFO [parallel-1] o.a.k.c.u.AppInfoParser: Kafka startTimeMs: 1714393284867 2024-04-29 12:21:25,068 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393284-139] o.a.k.c.u.AppInfoParser: App info kafka.admin.client for kafka-ui-admin-1714393284-139 unregistered 2024-04-29 12:21:25,069 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393284-139] o.a.k.c.a.i.AdminMetadataManager: [AdminClient clientId=kafka-ui-admin-1714393284-139] Metadata update failed org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: fetchMetadata 2024-04-29 12:21:25,070 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393284-139] o.a.k.c.a.KafkaAdminClient: [AdminClient clientId=kafka-ui-admin-1714393284-139] Timed out 2 remaining operation(s) during close. 2024-04-29 12:21:25,071 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393284-139] o.a.k.c.m.Metrics: Metrics scheduler closed 2024-04-29 12:21:25,072 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393284-139] o.a.k.c.m.Metrics: Closing reporter org.apache.kafka.common.metrics.JmxReporter 2024-04-29 12:21:25,072 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393284-139] o.a.k.c.m.Metrics: Metrics reporters closed 2024-04-29 12:21:25,073 ERROR [kafka-admin-client-thread | kafka-ui-admin-1714393284-139] o.a.k.c.u.KafkaThread: Uncaught exception in thread 'kafka-admin-client-thread | kafka-ui-admin-1714393284-139': java.lang.OutOfMemoryError: Java heap space 2024-04-29 12:21:25,074 ERROR [parallel-1] c.p.k.u.s.StatisticsService: Failed to collect cluster lacp-development-msk info java.lang.IllegalStateException: Error while creating AdminClient for Cluster lacp-development-msk at com.provectus.kafka.ui.service.AdminClientServiceImpl.lambda$createAdminClient$5(AdminClientServiceImpl.java:56) at reactor.core.publisher.Mono.lambda$onErrorMap$28(Mono.java:3783) at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94) at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onError(MonoPeekTerminal.java:258) at reactor.core.publisher.FluxMap$MapConditionalSubscriber.onError(FluxMap.java:265) at reactor.core.publisher.Operators$MonoSubscriber.onError(Operators.java:1886) at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.signalCached(MonoCacheTime.java:340) at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.onError(MonoCacheTime.java:363) at reactor.core.publisher.MonoFlatMap$FlatMapMain.onError(MonoFlatMap.java:180) at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onError(MonoIgnoreThen.java:278) at reactor.core.publisher.MonoPublishOn$PublishOnSubscriber.run(MonoPublishOn.java:187) at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68) at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: org.apache.kafka.common.errors.TimeoutException: The AdminClient thread has exited. Call: listNodes 2024-04-29 12:21:25,077 DEBUG [parallel-1] c.p.k.u.s.ClustersStatisticsScheduler: Metrics updated for cluster: lacp-development-msk 2024-04-29 12:21:25,262 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393284-140] o.a.k.c.u.AppInfoParser: App info kafka.admin.client for kafka-ui-admin-1714393284-140 unregistered 2024-04-29 12:21:25,263 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393284-140] o.a.k.c.a.i.AdminMetadataManager: [AdminClient clientId=kafka-ui-admin-1714393284-140] Metadata update failed org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call. Call: fetchMetadata 2024-04-29 12:21:25,264 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393284-140] o.a.k.c.a.KafkaAdminClient: [AdminClient clientId=kafka-ui-admin-1714393284-140] Timed out 2 remaining operation(s) during close. 2024-04-29 12:21:25,265 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393284-140] o.a.k.c.m.Metrics: Metrics scheduler closed 2024-04-29 12:21:25,266 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393284-140] o.a.k.c.m.Metrics: Closing reporter org.apache.kafka.common.metrics.JmxReporter 2024-04-29 12:21:25,266 INFO [kafka-admin-client-thread | kafka-ui-admin-1714393284-140] o.a.k.c.m.Metrics: Metrics reporters closed 2024-04-29 12:21:25,267 ERROR [kafka-admin-client-thread | kafka-ui-admin-1714393284-140] o.a.k.c.u.KafkaThread: Uncaught exception in thread 'kafka-admin-client-thread | kafka-ui-admin-1714393284-140': java.lang.OutOfMemoryError: Java heap space 2024-04-29 12:21:25,268 ERROR [parallel-1] c.p.k.u.s.StatisticsService: Failed to collect cluster test info java.lang.IllegalStateException: Error while creating AdminClient for Cluster test at com.provectus.kafka.ui.service.AdminClientServiceImpl.lambda$createAdminClient$5(AdminClientServiceImpl.java:56) at reactor.core.publisher.Mono.lambda$onErrorMap$28(Mono.java:3783) at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94) at reactor.core.publisher.MonoPeekTerminal$MonoTerminalPeekSubscriber.onError(MonoPeekTerminal.java:258) at reactor.core.publisher.FluxMap$MapConditionalSubscriber.onError(FluxMap.java:265) at reactor.core.publisher.Operators$MonoSubscriber.onError(Operators.java:1886) at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.signalCached(MonoCacheTime.java:340) at reactor.core.publisher.MonoCacheTime$CoordinatorSubscriber.onError(MonoCacheTime.java:363) at reactor.core.publisher.MonoFlatMap$FlatMapMain.onError(MonoFlatMap.java:180) at reactor.core.publisher.MonoIgnoreThen$ThenIgnoreMain.onError(MonoIgnoreThen.java:278) at reactor.core.publisher.MonoPublishOn$PublishOnSubscriber.run(MonoPublishOn.java:187) at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68) at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) Caused by: org.apache.kafka.common.errors.TimeoutException: The AdminClient thread has exited. Call: listNodes