Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

AM + Kubernetes (Minikube) gives me "io.vertx.core.VertxException: Thread blocked!" #6303

Closed
andrearossi90 opened this issue Oct 4, 2021 · 8 comments
Closed

AM + Kubernetes (Minikube) gives me "io.vertx.core.VertxException: Thread blocked!" #6303

andrearossi90 opened this issue Oct 4, 2021 · 8 comments
Labels
project: AM

Comments

@andrearossi90
Copy link

Hi,
I'm not new to Gravitee, but I'm pretty new to Kubernetes. My current setup is:

  • OS: Fedora 34 (Linux)
  • Kubernetes distribution: Minikube 1.23.2
  • Kubernetes version: 1.21.5
  • Helm version: 3.6 or 3.7 (I'll check it later)
  • Docker version: 20 (using docker as Minikube driver)
  • Helm charts: official AM 3.11 Helm charts (no editing)

Here the steps to reproduce the problem:

  1. Install Minikube
  2. Install Helm via script
  3. Install and start Docker
  4. Add user to Docker group so it can use docker even if not root
  5. Start Minikube minikube start --addons ingress --kubernetes-version=v1.21.5
  6. Bring up Gravitee AM helm install graviteeio-am graviteeio/am --set ui.service.type=NodePort --set mongodb-replicaset.enabled=true --set "mongo.uri=mongodb+srv://graviteeio-am-mongodb-replicaset.default.svc.cluster.local/?tls=false&ssl=false" --set api.service.type=NodePort --set gateway.service.type=NodePort
  7. Gateway and Management API cannot properly boot due to a long stack trace with root io.vertx.core.VertxException: Thread blocked

What can I do?

Thanks in advance.
@leleueri
Copy link
Contributor

leleueri commented Oct 4, 2021

Hi,

Could you provide to us the logs output to see the full stacktrace and previous logs messages ?

Thanks

@andrearossi90
Copy link
Author

andrearossi90 commented Oct 4, 2021 •
edited

@leleueri

Management API (attaching the complete log):

management.log

=========================================================================



  Gravitee.io - Access Management - Management API



  GRAVITEE_HOME: /opt/graviteeio-am-management-api



  GRAVITEE_OPTS: 



  JAVA: /opt/java/openjdk/bin/java



  JAVA_OPTS:  -Djava.net.preferIPv4Stack=true -Xms256m -Xmx256m -Djava.awt.headless=true -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8


[management.log](https://github.com/gravitee-io/issues/files/7277800/management.log)

  CLASSPATH: /opt/graviteeio-am-management-api/lib/gravitee-am-management-api-standalone-bootstrap-3.11.0.jar



=========================================================================



11:19:30.063 [graviteeio-node] [] INFO  i.g.n.c.s.e.PropertiesConfiguration - Loading Gravitee configuration.

11:19:30.237 [graviteeio-node] [] INFO  i.g.n.c.s.e.PropertiesConfiguration - 	Gravitee configuration loaded from /opt/graviteeio-am-management-api/config/gravitee.yml

11:19:30.912 [graviteeio-node] [] INFO  i.g.n.c.s.e.PropertiesConfiguration - Loading Gravitee configuration. DONE

11:19:31.046 [graviteeio-node] [] INFO  i.g.p.c.s.PluginHandlerBeanFactoryPostProcessor - Loading plugin handlers

11:19:31.238 [graviteeio-node] [] INFO  i.g.p.c.s.PluginHandlerBeanFactoryPostProcessor - Find 15 plugin handler(s):

11:19:31.242 [graviteeio-node] [] INFO  i.g.p.c.s.PluginHandlerBeanFactoryPostProcessor - 	io.gravitee.am.plugins.resource.plugin.ResourcePluginHandler

11:19:31.254 [graviteeio-node] [] INFO  i.g.p.c.s.PluginHandlerBeanFactoryPostProcessor - 	io.gravitee.plugin.policy.internal.PolicyPluginHandler

11:19:31.375 [graviteeio-node] [] INFO  i.g.p.c.s.PluginHandlerBeanFactoryPostProcessor - 	io.gravitee.node.plugins.service.handler.ServicePluginHandler

11:19:31.442 [graviteeio-node] [] INFO  i.g.p.c.s.PluginHandlerBeanFactoryPostProcessor - 	io.gravitee.node.tracing.plugin.TracingPluginHandler

11:19:31.503 [graviteeio-node] [] INFO  i.g.p.c.s.PluginHandlerBeanFactoryPostProcessor - 	io.gravitee.plugin.notifier.internal.NotifierPluginHandler

11:19:31.516 [graviteeio-node] [] INFO  i.g.p.c.s.PluginHandlerBeanFactoryPostProcessor - 	io.gravitee.am.plugins.botdetection.plugin.BotDetectionPluginHandler

11:19:31.532 [graviteeio-node] [] INFO  i.g.p.c.s.PluginHandlerBeanFactoryPostProcessor - 	io.gravitee.am.plugins.certificate.plugin.CertificatePluginHandler

11:19:31.551 [graviteeio-node] [] INFO  i.g.p.c.s.PluginHandlerBeanFactoryPostProcessor - 	io.gravitee.plugin.repository.internal.RepositoryPluginHandler

11:19:31.554 [graviteeio-node] [] INFO  i.g.p.c.s.PluginHandlerBeanFactoryPostProcessor - 	io.gravitee.node.reporter.plugin.ReporterPluginHandler

11:19:31.573 [graviteeio-node] [] INFO  i.g.p.c.s.PluginHandlerBeanFactoryPostProcessor - 	io.gravitee.plugin.cockpit.CockpitPluginHandler

11:19:31.615 [graviteeio-node] [] INFO  i.g.p.c.s.PluginHandlerBeanFactoryPostProcessor - 	io.gravitee.am.plugins.factor.plugin.FactorPluginHandler

11:19:31.620 [graviteeio-node] [] INFO  i.g.p.c.s.PluginHandlerBeanFactoryPostProcessor - 	io.gravitee.am.plugins.idp.plugin.IdentityProviderPluginHandler

11:19:31.624 [graviteeio-node] [] INFO  i.g.p.c.s.PluginHandlerBeanFactoryPostProcessor - 	io.gravitee.am.plugins.reporter.plugin.ReporterPluginHandler

11:19:31.681 [graviteeio-node] [] INFO  i.g.p.c.s.PluginHandlerBeanFactoryPostProcessor - 	io.gravitee.am.plugins.extensiongrant.plugin.ExtensionGrantPluginHandler

11:19:31.763 [graviteeio-node] [] INFO  i.g.p.c.s.PluginHandlerBeanFactoryPostProcessor - 	io.gravitee.plugin.alert.internal.AlertPluginHandler

11:19:40.724 [graviteeio-node] [] INFO  i.g.n.c.NodeDeployerFactoriesLoader - Loading instances for type io.gravitee.node.api.NodeDeployer

11:19:40.910 [graviteeio-node] [] INFO  i.g.node.tracing.TracingService - Tracing support is enabled with tracer: name[null]

11:19:51.216 [graviteeio-node] [] INFO  i.g.p.r.i.RepositoryPluginHandler - Loading repository for scope MANAGEMENT: mongodb

11:19:51.216 [graviteeio-node] [] INFO  i.g.p.r.i.RepositoryPluginHandler - Loading repository for scope OAUTH2: mongodb

11:19:53.121 [graviteeio-node] [] INFO  org.eclipse.jetty.util.log - Logging initialized @46888ms to org.eclipse.jetty.util.log.Slf4jLog

11:19:54.999 [graviteeio-node] [] INFO  i.g.node.container.AbstractContainer - Starting Gravitee.io - AM Management API...

11:19:55.080 [graviteeio-node] [] INFO  i.g.a.m.s.node.ManagementNode - Gravitee.io - AM Management API is now starting...

11:19:55.103 [graviteeio-node] [] INFO  i.g.a.m.s.node.ManagementNode - 	Starting component: PluginEventListener

11:19:55.122 [graviteeio-node] [] INFO  i.g.common.service.AbstractService - Initializing service io.gravitee.plugin.core.internal.PluginEventListener

11:19:55.130 [graviteeio-node] [] INFO  i.g.c.event.impl.EventManagerImpl - Register new listener PluginEventListener for event type class io.gravitee.plugin.core.api.PluginEvent

11:19:55.131 [graviteeio-node] [] INFO  i.g.a.m.s.node.ManagementNode - 	Starting component: PluginRegistry

11:19:55.178 [graviteeio-node] [] INFO  i.g.common.service.AbstractService - Initializing service io.gravitee.plugin.core.internal.PluginRegistryImpl

11:19:55.178 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Initializing plugin registry.

11:19:55.178 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugins from /opt/graviteeio-am-management-api/plugins

11:19:55.183 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-botdetection-recaptcha-v3-3.11.0.zip

11:19:55.359 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-botdetection-recaptcha-v3-3.11.0/gravitee-am-botdetection-recaptcha-v3-3.11.0.jar

11:19:55.417 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-certificate-javakeystore-3.11.0.zip

11:19:55.505 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-certificate-javakeystore-3.11.0/gravitee-am-certificate-javakeystore-3.11.0.jar

11:19:55.524 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-certificate-pkcs12-3.11.0.zip

11:19:55.527 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-certificate-pkcs12-3.11.0/gravitee-am-certificate-pkcs12-3.11.0.jar

11:19:55.545 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-extensiongrant-jwtbearer-3.11.0.zip

11:19:55.574 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-extensiongrant-jwtbearer-3.11.0/gravitee-am-extensiongrant-jwtbearer-3.11.0.jar

11:19:55.577 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-factor-email-3.11.0.zip

11:19:55.631 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-factor-email-3.11.0/gravitee-am-factor-email-3.11.0.jar

11:19:55.635 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-factor-otp-3.11.0.zip

11:19:55.765 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-factor-otp-3.11.0/gravitee-am-factor-otp-3.11.0.jar

11:19:55.802 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-factor-sms-3.11.0.zip

11:19:55.849 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-factor-sms-3.11.0/gravitee-am-factor-sms-3.11.0.jar

11:19:55.853 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-identityprovider-azure-ad-3.11.0.zip

11:19:55.906 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-identityprovider-azure-ad-3.11.0/gravitee-am-identityprovider-azure-ad-3.11.0.jar

11:19:55.910 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-identityprovider-facebook-3.11.0.zip

11:19:55.917 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-identityprovider-facebook-3.11.0/gravitee-am-identityprovider-facebook-3.11.0.jar

11:19:55.920 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-identityprovider-franceconnect-3.11.0.zip

11:19:55.928 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-identityprovider-franceconnect-3.11.0/gravitee-am-identityprovider-franceconnect-3.11.0.jar

11:19:55.932 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-identityprovider-github-3.11.0.zip

11:19:56.007 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-identityprovider-github-3.11.0/gravitee-am-identityprovider-github-3.11.0.jar

11:19:56.010 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-identityprovider-google-3.11.0.zip

11:19:56.020 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-identityprovider-google-3.11.0/gravitee-am-identityprovider-google-3.11.0.jar

11:19:56.026 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-identityprovider-gravitee-3.11.0.zip

11:19:56.099 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-identityprovider-gravitee-3.11.0/gravitee-am-identityprovider-gravitee-3.11.0.jar

11:19:56.101 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-identityprovider-http-3.11.0.zip

11:19:56.106 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-identityprovider-http-3.11.0/gravitee-am-identityprovider-http-3.11.0.jar

11:19:56.108 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-identityprovider-inline-3.11.0.zip

11:19:56.113 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-identityprovider-inline-3.11.0/gravitee-am-identityprovider-inline-3.11.0.jar

11:19:56.116 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-identityprovider-jdbc-3.11.0.zip

11:19:56.499 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-identityprovider-jdbc-3.11.0/gravitee-am-identityprovider-jdbc-3.11.0.jar

11:19:56.501 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-identityprovider-ldap-3.11.0.zip

11:19:56.718 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-identityprovider-ldap-3.11.0/gravitee-am-identityprovider-ldap-3.11.0.jar

11:19:56.738 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-identityprovider-linkedin-3.11.0.zip

11:19:56.746 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-identityprovider-linkedin-3.11.0/gravitee-am-identityprovider-linkedin-3.11.0.jar

11:19:56.799 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-identityprovider-mongo-3.11.0.zip

11:19:56.915 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-identityprovider-mongo-3.11.0/gravitee-am-identityprovider-mongo-3.11.0.jar

11:19:56.917 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-identityprovider-oauth2-generic-3.11.0.zip

11:19:56.922 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-identityprovider-oauth2-generic-3.11.0/gravitee-am-identityprovider-oauth2-generic-3.11.0.jar

11:19:56.924 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-identityprovider-salesforce-3.11.0.zip

11:19:56.927 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-identityprovider-salesforce-3.11.0/gravitee-am-identityprovider-salesforce-3.11.0.jar

11:19:57.003 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-identityprovider-twitter-3.11.0.zip

11:19:57.024 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-identityprovider-twitter-3.11.0/gravitee-am-identityprovider-twitter-3.11.0.jar

11:19:57.036 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-management-api-services-purge-3.11.0.zip

11:19:57.041 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-management-api-services-purge-3.11.0/gravitee-am-management-api-services-purge-3.11.0.jar

11:19:57.114 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-management-api-services-sync-3.11.0.zip

11:19:57.127 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-management-api-services-sync-3.11.0/gravitee-am-management-api-services-sync-3.11.0.jar

11:19:57.134 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-policy-enrich-auth-flow-3.11.0.zip

11:19:57.212 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-policy-enrich-auth-flow-3.11.0/gravitee-am-policy-enrich-auth-flow-3.11.0.jar

11:19:57.214 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-policy-enrich-profile-3.11.0.zip

11:19:57.218 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-policy-enrich-profile-3.11.0/gravitee-am-policy-enrich-profile-3.11.0.jar

11:19:57.220 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-reporter-file-3.11.0.zip

11:19:57.228 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-reporter-file-3.11.0/gravitee-am-reporter-file-3.11.0.jar

11:19:57.230 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-reporter-jdbc-3.11.0.zip

11:19:57.653 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-reporter-jdbc-3.11.0/gravitee-am-reporter-jdbc-3.11.0.jar

11:19:57.660 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-reporter-kafka-3.11.0.zip

11:19:57.745 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-reporter-kafka-3.11.0/gravitee-am-reporter-kafka-3.11.0.jar

11:19:57.806 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-reporter-mongodb-3.11.0.zip

11:19:57.823 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-reporter-mongodb-3.11.0/gravitee-am-reporter-mongodb-3.11.0.jar

11:19:57.827 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-repository-jdbc-3.11.0.zip

11:19:58.109 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-repository-jdbc-3.11.0/gravitee-am-repository-jdbc-3.11.0.jar

11:19:58.112 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-repository-mongodb-3.11.0.zip

11:19:58.314 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-repository-mongodb-3.11.0/gravitee-am-repository-mongodb-3.11.0.jar

11:19:58.316 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-resource-infobip-3.11.0.zip

11:19:58.509 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-resource-infobip-3.11.0/gravitee-am-resource-infobip-3.11.0.jar

11:19:58.515 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-resource-smtp-3.11.0.zip

11:19:58.521 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-resource-smtp-3.11.0/gravitee-am-resource-smtp-3.11.0.jar

11:19:58.524 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-am-resource-twilio-3.11.0.zip

11:19:58.982 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-am-resource-twilio-3.11.0/gravitee-am-resource-twilio-3.11.0.jar

11:19:59.008 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-cockpit-connectors-ws-2.0.0.zip

11:19:59.023 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-cockpit-connectors-ws-2.0.0/gravitee-cockpit-connectors-ws-2.0.0.jar

11:19:59.023 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-notifier-email-1.3.0.zip

11:19:59.140 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-notifier-email-1.3.0/gravitee-notifier-email-1.3.0.jar

11:19:59.144 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-notifier-webhook-1.1.0.zip

11:19:59.148 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-notifier-webhook-1.1.0/gravitee-notifier-webhook-1.1.0.jar

11:19:59.150 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-policy-callout-http-1.13.0.zip

11:19:59.157 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-policy-callout-http-1.13.0/gravitee-policy-callout-http-1.13.0.jar

11:19:59.159 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-policy-groovy-1.14.1.zip

11:19:59.601 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-policy-groovy-1.14.1/gravitee-policy-groovy-1.14.1.jar

11:19:59.607 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-policy-ipfiltering-1.8.0.zip

11:19:59.639 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-policy-ipfiltering-1.8.0/gravitee-policy-ipfiltering-1.8.0.jar

11:19:59.713 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Loading plugin from /opt/graviteeio-am-management-api/plugins/gravitee-policy-request-validation-1.12.0.zip

11:19:59.813 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - A plugin manifest has been loaded from: /opt/graviteeio-am-management-api/plugins/.work/gravitee-policy-request-validation-1.12.0/gravitee-policy-request-validation-1.12.0.jar

11:19:59.819 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - List of available bot_detection: 

11:19:59.837 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> google-recaptcha-v3-am-bot-detection [3.11.0] has been loaded

11:19:59.838 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - List of available certificate: 

11:19:59.838 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> javakeystore-am-certificate [3.11.0] has been loaded

11:19:59.838 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> pkcs12-am-certificate [3.11.0] has been loaded

11:19:59.839 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - List of available extension_grant: 

11:19:59.839 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> jwtbearer-am-extension-grant [3.11.0] has been loaded

11:19:59.840 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - List of available factor: 

11:19:59.840 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> email-am-factor [3.11.0] has been loaded

11:19:59.840 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> otp-am-factor [3.11.0] has been loaded

11:19:59.841 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> sms-am-factor [3.11.0] has been loaded

11:19:59.841 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - List of available identity_provider: 

11:19:59.842 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> azure-ad-am-idp [3.11.0] has been loaded

11:19:59.842 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> facebook-am-idp [3.11.0] has been loaded

11:19:59.842 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> franceconnect-am-idp [3.11.0] has been loaded

11:19:59.842 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> github-am-idp [3.11.0] has been loaded

11:19:59.842 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> google-am-idp [3.11.0] has been loaded

11:19:59.842 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> gravitee-am-idp [3.11.0] has been loaded

11:19:59.843 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> http-am-idp [3.11.0] has been loaded

11:19:59.843 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> inline-am-idp [3.11.0] has been loaded

11:19:59.843 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> jdbc-am-idp [3.11.0] has been loaded

11:19:59.843 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> ldap-am-idp [3.11.0] has been loaded

11:19:59.843 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> linkedin-am-idp [3.11.0] has been loaded

11:19:59.843 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> mongo-am-idp [3.11.0] has been loaded

11:19:59.843 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> oauth2-generic-am-idp [3.11.0] has been loaded

11:19:59.843 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> salesforce-am-idp [3.11.0] has been loaded

11:19:59.843 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> twitter-am-idp [3.11.0] has been loaded

11:19:59.900 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - List of available service: 

11:19:59.902 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> purge-service [3.11.0] has been loaded

11:19:59.907 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> sync-service [3.11.0] has been loaded

11:19:59.907 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - List of available policy: 

11:19:59.908 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> policy-am-enrich-auth-flow [3.11.0] has been loaded

11:19:59.908 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> policy-am-enrich-profile [3.11.0] has been loaded

11:19:59.908 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> policy-http-callout [1.13.0] has been loaded

11:19:59.909 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> groovy [1.14.1] has been loaded

11:19:59.917 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> ip-filtering [1.8.0] has been loaded

11:19:59.918 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> policy-request-validation [1.12.0] has been loaded

11:19:59.918 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - List of available am-reporter: 

11:19:59.918 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> reporter-am-file [3.11.0] has been loaded

11:19:59.919 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> reporter-am-jdbc [3.11.0] has been loaded

11:19:59.920 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> reporter-am-kafka [3.11.0] has been loaded

11:19:59.920 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> mongodb [3.11.0] has been loaded

11:19:59.921 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - List of available repository: 

11:19:59.921 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> repository-am-jdbc [3.11.0] has been loaded

11:19:59.921 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> repository-am-mongodb [3.11.0] has been loaded

11:19:59.921 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - List of available resource: 

11:19:59.921 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> infobip-am-resource [3.11.0] has been loaded

11:19:59.924 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> smtp-am-resource [3.11.0] has been loaded

11:19:59.925 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> twilio-verify-am-resource [3.11.0] has been loaded

11:19:59.925 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - List of available cockpit: 

11:19:59.925 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> cockpit-connectors-ws [2.0.0] has been loaded

11:19:59.933 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - List of available notifier: 

11:19:59.936 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> email-notifier [1.3.0] has been loaded

11:19:59.937 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - 	> webhook-notifier [1.1.0] has been loaded

11:19:59.938 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginEventListener - All plugins have been loaded. Installing...

11:20:00.004 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginEventListener - Installing repository plugins...

11:20:00.060 [graviteeio-node] [] INFO  i.g.p.r.i.RepositoryPluginHandler - Register a new repository: repository-am-jdbc [io.gravitee.am.repository.jdbc.JdbcRepositoryProvider]

11:20:00.107 [graviteeio-node] [] INFO  i.g.p.r.i.RepositoryPluginHandler - Register a new repository: repository-am-mongodb [io.gravitee.am.repository.mongodb.MongoRepositoryProvider]

11:20:00.112 [graviteeio-node] [] INFO  i.g.p.r.i.RepositoryPluginHandler - Repository [MANAGEMENT] loaded by mongodb

11:20:17.524 [graviteeio-node] [] INFO  i.g.p.r.i.RepositoryPluginHandler - Repository [OAUTH2] loaded by mongodb

11:20:19.500 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginEventListener - Installing cockpit plugins...

11:20:19.502 [graviteeio-node] [] INFO  i.g.p.cockpit.CockpitPluginHandler - Install plugin: cockpit-connectors-ws [io.gravitee.cockpit.connectors.ws.WebSocketCockpitConnector]

11:20:20.805 [graviteeio-node] [] INFO  i.g.c.c.c.s.MonitoringCollectorService - Starting monitoring collector service

11:20:20.816 [graviteeio-node] [] INFO  i.g.c.c.c.s.AlertManagerService - Starting alert manager service

11:20:20.818 [graviteeio-node] [] INFO  i.g.c.c.ws.WebSocketCockpitConnector - Cockpit connector is disabled.

11:20:20.818 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginEventListener - Installing service plugins...

11:20:20.818 [graviteeio-node] [] INFO  i.g.n.p.s.h.ServicePluginHandler - Install plugin: purge-service [io.gravitee.am.management.services.purge.ScheduledPurgeService]

WARNING: An illegal reflective access operation has occurred

WARNING: Illegal reflective access by org.springframework.cglib.core.ReflectUtils (file:/opt/graviteeio-am-management-api/lib/ext/spring-core-5.2.14.RELEASE.jar) to method java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain)

WARNING: Please consider reporting this to the maintainers of org.springframework.cglib.core.ReflectUtils

WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations

WARNING: All illegal access operations will be denied in a future release

11:20:21.303 [graviteeio-node] [] INFO  i.g.n.p.s.h.ServicePluginHandler - Install plugin: sync-service [io.gravitee.am.management.services.sync.ScheduledSyncService]

11:20:21.424 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginEventListener - Installing resource plugins...

11:20:21.431 [graviteeio-node] [] INFO  i.g.a.p.r.p.ResourcePluginHandler - Register a new resource plugin: smtp-am-resource [io.gravitee.am.resource.smtp.SmtpResource]

11:20:21.509 [graviteeio-node] [] INFO  i.g.a.p.r.p.ResourcePluginHandler - Register a new resource plugin: infobip-am-resource [io.gravitee.am.resource.infobip.InfobipResource]

11:20:21.512 [graviteeio-node] [] INFO  i.g.a.p.r.p.ResourcePluginHandler - Register a new resource plugin: twilio-verify-am-resource [io.gravitee.am.resource.twilio.TwilioVerifyResource]

11:20:21.513 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginEventListener - Installing am-reporter plugins...

11:20:21.630 [graviteeio-node] [] INFO  i.g.a.p.r.p.ReporterPluginHandler - Register a new reporter plugin: reporter-am-jdbc [io.gravitee.am.reporter.jdbc.JdbcReporter]

11:20:21.652 [graviteeio-node] [] INFO  i.g.a.p.r.p.ReporterPluginHandler - Register a new reporter plugin: reporter-am-kafka [io.gravitee.am.reporter.kafka.KafkaReporter]

11:20:21.700 [graviteeio-node] [] INFO  i.g.a.p.r.p.ReporterPluginHandler - Register a new reporter plugin: reporter-am-file [io.gravitee.am.reporter.file.FileReporter]

11:20:21.703 [graviteeio-node] [] INFO  i.g.a.p.r.p.ReporterPluginHandler - Register a new reporter plugin: mongodb [io.gravitee.am.reporter.mongodb.MongoReporter]

11:20:21.704 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginEventListener - Installing factor plugins...

11:20:21.706 [graviteeio-node] [] INFO  i.g.a.p.f.plugin.FactorPluginHandler - Register a new factor plugin: sms-am-factor [io.gravitee.am.factor.sms.SMSFactor]

11:20:21.707 [graviteeio-node] [] INFO  i.g.a.p.f.plugin.FactorPluginHandler - Register a new factor plugin: email-am-factor [io.gravitee.am.factor.email.EmailFactor]

11:20:21.709 [graviteeio-node] [] INFO  i.g.a.p.f.plugin.FactorPluginHandler - Register a new factor plugin: otp-am-factor [io.gravitee.am.factor.otp.OTPFactor]

11:20:21.710 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginEventListener - Installing identity_provider plugins...

11:20:21.717 [graviteeio-node] [] INFO  i.g.a.p.i.p.IdentityProviderPluginHandler - Register a new identity provider plugin: facebook-am-idp [io.gravitee.am.identityprovider.facebook.FacebookIdentityProvider]

11:20:21.719 [graviteeio-node] [] INFO  i.g.a.p.i.p.IdentityProviderPluginHandler - Register a new identity provider plugin: google-am-idp [io.gravitee.am.identityprovider.google.GoogleIdentityProvider]

11:20:21.721 [graviteeio-node] [] INFO  i.g.a.p.i.p.IdentityProviderPluginHandler - Register a new identity provider plugin: franceconnect-am-idp [io.gravitee.am.identityprovider.franceconnect.FranceConnectIdentityProvider]

11:20:21.722 [graviteeio-node] [] INFO  i.g.a.p.i.p.IdentityProviderPluginHandler - Register a new identity provider plugin: linkedin-am-idp [io.gravitee.am.identityprovider.linkedin.LinkedinIdentityProvider]

11:20:21.723 [graviteeio-node] [] INFO  i.g.a.p.i.p.IdentityProviderPluginHandler - Register a new identity provider plugin: mongo-am-idp [io.gravitee.am.identityprovider.mongo.MongoIdentityProvider]

11:20:21.725 [graviteeio-node] [] INFO  i.g.a.p.i.p.IdentityProviderPluginHandler - Register a new identity provider plugin: azure-ad-am-idp [io.gravitee.am.identityprovider.azure.AzureADIdentityProvider]

11:20:21.802 [graviteeio-node] [] INFO  i.g.a.p.i.p.IdentityProviderPluginHandler - Register a new identity provider plugin: github-am-idp [io.gravitee.am.identityprovider.github.GithubIdentityProvider]

11:20:21.805 [graviteeio-node] [] INFO  i.g.a.p.i.p.IdentityProviderPluginHandler - Register a new identity provider plugin: twitter-am-idp [io.gravitee.am.identityprovider.twitter.TwitterIdentityProvider]

11:20:21.807 [graviteeio-node] [] INFO  i.g.a.p.i.p.IdentityProviderPluginHandler - Register a new identity provider plugin: http-am-idp [io.gravitee.am.identityprovider.http.HttpIdentityProvider]

11:20:21.809 [graviteeio-node] [] INFO  i.g.a.p.i.p.IdentityProviderPluginHandler - Register a new identity provider plugin: oauth2-generic-am-idp [io.gravitee.am.identityprovider.oauth2.OAuth2GenericIdentityProvider]

11:20:21.812 [graviteeio-node] [] INFO  i.g.a.p.i.p.IdentityProviderPluginHandler - Register a new identity provider plugin: inline-am-idp [io.gravitee.am.identityprovider.inline.InlineIdentityProvider]

11:20:21.915 [graviteeio-node] [] INFO  i.g.a.p.i.p.IdentityProviderPluginHandler - Register a new identity provider plugin: salesforce-am-idp [io.gravitee.am.identityprovider.salesforce.SalesForceIdentityProvider]

11:20:21.927 [graviteeio-node] [] INFO  i.g.a.p.i.p.IdentityProviderPluginHandler - Register a new identity provider plugin: ldap-am-idp [io.gravitee.am.identityprovider.ldap.LdapIdentityProvider]

11:20:21.938 [graviteeio-node] [] INFO  i.g.a.p.i.p.IdentityProviderPluginHandler - Register a new identity provider plugin: gravitee-am-idp [io.gravitee.am.identityprovider.gravitee.GraviteeIdentityProvider]

11:20:21.959 [graviteeio-node] [] INFO  i.g.a.p.i.p.IdentityProviderPluginHandler - Register a new identity provider plugin: jdbc-am-idp [io.gravitee.am.identityprovider.jdbc.JdbcIdentityProvider]

11:20:21.963 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginEventListener - Installing policy plugins...

11:20:21.965 [graviteeio-node] [] INFO  i.g.p.p.internal.PolicyPluginHandler - Install plugin: policy-request-validation [io.gravitee.policy.requestvalidation.RequestValidationPolicy]

11:20:22.158 [graviteeio-node] [] INFO  i.g.p.p.internal.PolicyPluginHandler - Install plugin: groovy [io.gravitee.policy.groovy.GroovyPolicy]

11:20:23.116 [graviteeio-node] [] INFO  i.g.p.p.internal.PolicyPluginHandler - Install plugin: policy-am-enrich-auth-flow [io.gravitee.am.policy.enrich.auth.EnrichAuthFlowPolicy]

11:20:23.121 [graviteeio-node] [] INFO  i.g.p.p.internal.PolicyPluginHandler - Install plugin: ip-filtering [io.gravitee.policy.ipfiltering.IPFilteringPolicy]

11:20:23.151 [graviteeio-node] [] INFO  i.g.p.p.internal.PolicyPluginHandler - Install plugin: policy-am-enrich-profile [io.gravitee.am.policy.enrich.profile.EnrichProfilePolicy]

11:20:23.155 [graviteeio-node] [] INFO  i.g.p.p.internal.PolicyPluginHandler - Install plugin: policy-http-callout [io.gravitee.policy.callout.CalloutHttpPolicy]

11:20:23.181 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginEventListener - Installing bot_detection plugins...

11:20:23.202 [graviteeio-node] [] INFO  i.g.a.p.b.p.BotDetectionPluginHandler - Register a new bot detection plugin: google-recaptcha-v3-am-bot-detection [io.gravitee.am.botdetection.google.recaptcha.GoogleReCaptchaV3]

11:20:23.209 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginEventListener - Installing certificate plugins...

11:20:23.214 [graviteeio-node] [] INFO  i.g.a.p.c.p.CertificatePluginHandler - Register a new certificate plugin: javakeystore-am-certificate [io.gravitee.am.certificate.javakeystore.JavaKeyStore]

11:20:23.218 [graviteeio-node] [] INFO  i.g.a.p.c.p.CertificatePluginHandler - Register a new certificate plugin: pkcs12-am-certificate [io.gravitee.am.certificate.pkcs12.PKCS12Certificate]

11:20:23.219 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginEventListener - Installing notifier plugins...

11:20:23.220 [graviteeio-node] [] INFO  i.g.p.n.i.NotifierPluginHandler - Install plugin: webhook-notifier [io.gravitee.notifier.webhook.WebhookNotifier]

11:20:23.336 [graviteeio-node] [] INFO  i.g.p.n.i.NotifierPluginHandler - Install plugin: email-notifier [io.gravitee.notifier.email.EmailNotifier]

11:20:23.424 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginEventListener - Installing extension_grant plugins...

11:20:23.427 [graviteeio-node] [] INFO  i.g.a.p.e.p.ExtensionGrantPluginHandler - Register a new extension grant plugin: jwtbearer-am-extension-grant [io.gravitee.am.extensiongrant.jwtbearer.JWTBearerExtensionGrant]

11:20:23.441 [graviteeio-node] [] INFO  i.g.p.c.internal.PluginRegistryImpl - Plugins have been loaded and installed.

11:20:23.442 [graviteeio-node] [] INFO  i.g.a.m.s.node.ManagementNode - 	Starting component: ServiceManager

11:20:23.442 [graviteeio-node] [] INFO  i.g.common.service.AbstractService - Initializing service Plugins - Services Manager

11:20:23.442 [graviteeio-node] [] WARN  i.g.a.m.s.p.ScheduledPurgeService - Purge service has been disabled

11:20:23.442 [graviteeio-node] [] INFO  i.g.common.service.AbstractService - Initializing service Sync Service

11:20:23.447 [graviteeio-node] [] INFO  i.g.a.m.s.sync.ScheduledSyncService - Sync service has been initialized with cron [*/5 * * * * *]

11:20:23.558 [graviteeio-node] [] INFO  i.g.a.m.s.node.ManagementNode - 	Starting component: ManagementService

11:20:23.562 [graviteeio-node] [] INFO  i.g.common.service.AbstractService - Initializing service Node Management API service

11:20:23.636 [graviteeio-node] [] INFO  i.g.a.m.s.node.ManagementNode - 	Starting component: NodeMonitoringEventHandler

11:20:23.647 [graviteeio-node] [] INFO  i.g.common.service.AbstractService - Initializing service io.gravitee.node.monitoring.handler.NodeMonitoringEventHandler

11:20:23.753 [graviteeio-node] [] INFO  i.g.a.m.s.node.ManagementNode - 	Starting component: NodeInfosService

11:20:23.763 [graviteeio-node] [] INFO  i.g.n.m.infos.NodeInfosService - Starting node infos service

11:20:23.763 [graviteeio-node] [] INFO  i.g.common.service.AbstractService - Initializing service Node Infos Service

11:20:23.807 [vert.x-eventloop-thread-1] [] INFO  i.g.n.m.h.v.v.ManagementVerticle - Start HTTP listener for Node Management API

11:20:24.204 [graviteeio-node] [] INFO  i.g.n.m.infos.NodeInfosService - Start node infos service: DONE

11:20:24.205 [graviteeio-node] [] INFO  i.g.a.m.s.node.ManagementNode - 	Starting component: NodeHealthCheckService

11:20:24.233 [graviteeio-node] [] INFO  i.g.common.service.AbstractService - Initializing service Node Health-check service

11:20:24.234 [graviteeio-node] [] INFO  i.g.n.m.healthcheck.ProbeManagerImpl - Loading instances for type io.gravitee.node.api.healthcheck.Probe

11:20:24.524 [graviteeio-node] [] INFO  i.g.n.m.h.v.e.ManagementEndpointManagerImpl - Register a new endpoint for Management API: GET /health [io.gravitee.node.monitoring.healthcheck.NodeHealthCheckManagementEndpoint]

11:20:25.886 [graviteeio-node] [] INFO  i.g.a.m.s.node.ManagementNode - 	Starting component: NodeMonitorService

11:20:25.886 [graviteeio-node] [] INFO  i.g.common.service.AbstractService - Initializing service Node Monitor Service

11:20:26.035 [graviteeio-node] [] INFO  i.g.n.m.monitor.NodeMonitorService - Node monitoring scheduled with fixed delay 5000 MILLISECONDS 

11:20:26.061 [graviteeio-node] [] INFO  i.g.n.m.h.v.e.ManagementEndpointManagerImpl - Register a new endpoint for Management API: GET /monitor [io.gravitee.node.monitoring.monitor.NodeMonitorManagementEndpoint]

11:20:26.061 [graviteeio-node] [] INFO  i.g.a.m.s.node.ManagementNode - 	Starting component: ReporterManager

11:20:26.070 [graviteeio-node] [] INFO  i.g.common.service.AbstractService - Initializing service Reporter service

11:20:26.222 [graviteeio-node] [] INFO  i.g.a.m.s.node.ManagementNode - 	Starting component: JettyHttpServer

11:20:26.852 [vertx-blocked-thread-checker] [] WARN  i.v.core.impl.BlockedThreadChecker - Thread Thread[vert.x-eventloop-thread-1,5,main] has been blocked for 3029 ms, time limit is 2000 ms

11:20:26.853 [vertx-blocked-thread-checker] [] WARN  i.v.core.impl.BlockedThreadChecker - Thread Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 2613 ms, time limit is 2000 ms

11:20:27.837 [vertx-blocked-thread-checker] [] WARN  i.v.core.impl.BlockedThreadChecker - Thread Thread[vert.x-eventloop-thread-1,5,main] has been blocked for 4029 ms, time limit is 2000 ms

11:20:27.837 [vertx-blocked-thread-checker] [] WARN  i.v.core.impl.BlockedThreadChecker - Thread Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 3613 ms, time limit is 2000 ms

11:20:28.908 [vertx-blocked-thread-checker] [] WARN  i.v.core.impl.BlockedThreadChecker - Thread Thread[vert.x-eventloop-thread-1,5,main] has been blocked for 5031 ms, time limit is 2000 ms

io.vertx.core.VertxException: Thread blocked

	at java.base@11.0.11/java.lang.ClassLoader.defineClass1(Native Method)

	at java.base@11.0.11/java.lang.ClassLoader.defineClass(Unknown Source)

	at java.base@11.0.11/java.security.SecureClassLoader.defineClass(Unknown Source)

	at java.base@11.0.11/java.net.URLClassLoader.defineClass(Unknown Source)

	at java.base@11.0.11/java.net.URLClassLoader$1.run(Unknown Source)

	at java.base@11.0.11/java.net.URLClassLoader$1.run(Unknown Source)

	at java.base@11.0.11/java.security.AccessController.doPrivileged(Native Method)

	at java.base@11.0.11/java.net.URLClassLoader.findClass(Unknown Source)

	at java.base@11.0.11/java.lang.ClassLoader.loadClass(Unknown Source)

	at java.base@11.0.11/java.lang.ClassLoader.loadClass(Unknown Source)

	at io.netty.channel.AbstractChannel.<init>(AbstractChannel.java:52)

	at io.netty.channel.nio.AbstractNioChannel.<init>(AbstractNioChannel.java:80)

	at io.netty.channel.nio.AbstractNioMessageChannel.<init>(AbstractNioMessageChannel.java:42)

	at io.netty.channel.socket.nio.NioDatagramChannel.<init>(NioDatagramChannel.java:151)

	at io.netty.channel.socket.nio.NioDatagramChannel.<init>(NioDatagramChannel.java:119)

	at io.vertx.core.net.impl.transport.Transport.datagramChannel(Transport.java:160)

	at io.vertx.core.impl.resolver.DnsResolverProvider$1.lambda$newResolver$0(DnsResolverProvider.java:122)

	at io.vertx.core.impl.resolver.DnsResolverProvider$1$$Lambda$436/0x0000000100659040.newChannel(Unknown Source)

	at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:310)

	at io.netty.bootstrap.AbstractBootstrap.register(AbstractBootstrap.java:227)

	at io.netty.resolver.dns.DnsNameResolver.<init>(DnsNameResolver.java:497)

	at io.netty.resolver.dns.DnsNameResolverBuilder.build(DnsNameResolverBuilder.java:488)

	at io.vertx.core.impl.resolver.DnsResolverProvider$1$1.newNameResolver(DnsResolverProvider.java:181)

	at io.netty.resolver.dns.DnsAddressResolverGroup.newResolver(DnsAddressResolverGroup.java:92)

	at io.netty.resolver.dns.DnsAddressResolverGroup.newResolver(DnsAddressResolverGroup.java:77)

	at io.netty.resolver.AddressResolverGroup.getResolver(AddressResolverGroup.java:70)

	at io.vertx.core.impl.resolver.DnsResolverProvider$1.newResolver(DnsResolverProvider.java:185)

	at io.netty.resolver.AddressResolverGroup.getResolver(AddressResolverGroup.java:70)

	at io.vertx.core.impl.AddressResolver.resolveHostname(AddressResolver.java:90)

	at io.vertx.core.impl.AddressResolver.resolveHostname(AddressResolver.java:83)

	at io.vertx.core.impl.VertxImpl.resolveAddress(VertxImpl.java:837)

	at io.vertx.core.net.impl.AsyncResolveConnectHelper.doBind(AsyncResolveConnectHelper.java:73)

	at io.vertx.core.net.impl.TCPServerBase.listen(TCPServerBase.java:130)

	at io.vertx.core.http.impl.HttpServerImpl.listen(HttpServerImpl.java:213)

	at io.vertx.core.http.impl.HttpServerImpl.listen(HttpServerImpl.java:143)

	at io.vertx.core.http.impl.HttpServerImpl.listen(HttpServerImpl.java:129)

	at io.vertx.core.http.impl.HttpServerImpl.listen(HttpServerImpl.java:134)

	at io.gravitee.node.management.http.vertx.verticle.ManagementVerticle.doStart(ManagementVerticle.java:148)

	at io.gravitee.node.management.http.vertx.verticle.ManagementVerticle.start(ManagementVerticle.java:91)

	at io.vertx.core.impl.DeploymentManager.lambda$doDeploy$5(DeploymentManager.java:196)

	at io.vertx.core.impl.DeploymentManager$$Lambda$401/0x0000000100625840.handle(Unknown Source)

	at io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:96)

	at io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:59)

	at io.vertx.core.impl.EventLoopContext.lambda$runOnContext$0(EventLoopContext.java:37)

	at io.vertx.core.impl.EventLoopContext$$Lambda$407/0x0000000100627840.run(Unknown Source)

	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)

	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)

	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)

	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)

	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)

	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)

	at java.base@11.0.11/java.lang.Thread.run(Unknown Source)

11:20:28.909 [vertx-blocked-thread-checker] [] WARN  i.v.core.impl.BlockedThreadChecker - Thread Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 4615 ms, time limit is 2000 ms

11:20:29.313 [graviteeio-node] [] INFO  org.eclipse.jetty.server.Server - jetty-9.4.36.v20210114; built: 2021-01-14T16:44:28.689Z; git: 238ec6997c7806b055319a6d11f8ae7564adc0de; jvm 11.0.11+9

11:20:29.810 [graviteeio-node] [] INFO  o.e.jetty.server.RequestLogWriter - Opened /opt/graviteeio-am-management-api/logs/gravitee_accesslog_2021_10_04.log

11:20:29.902 [vertx-blocked-thread-checker] [] WARN  i.v.core.impl.BlockedThreadChecker - Thread Thread[vert.x-eventloop-thread-1,5,main] has been blocked for 6031 ms, time limit is 2000 ms

io.vertx.core.VertxException: Thread blocked

	at java.base@11.0.11/java.util.ArrayDeque.poll(Unknown Source)

	at java.base@11.0.11/java.util.zip.ZipFile$CleanableResource.getInflater(Unknown Source)

	at java.base@11.0.11/java.util.zip.ZipFile$ZipFileInflaterInputStream.<init>(Unknown Source)

	at java.base@11.0.11/java.util.zip.ZipFile.getInputStream(Unknown Source)

	at java.base@11.0.11/java.util.jar.JarFile.getInputStream(Unknown Source)

	at java.base@11.0.11/jdk.internal.loader.URLClassPath$JarLoader$2.getInputStream(Unknown Source)

	at java.base@11.0.11/jdk.internal.loader.Resource.cachedInputStream(Unknown Source)

	at java.base@11.0.11/jdk.internal.loader.Resource.getByteBuffer(Unknown Source)

	at java.base@11.0.11/java.net.URLClassLoader.defineClass(Unknown Source)

	at java.base@11.0.11/java.net.URLClassLoader$1.run(Unknown Source)

	at java.base@11.0.11/java.net.URLClassLoader$1.run(Unknown Source)

	at java.base@11.0.11/java.security.AccessController.doPrivileged(Native Method)

	at java.base@11.0.11/java.net.URLClassLoader.findClass(Unknown Source)

	at java.base@11.0.11/java.lang.ClassLoader.loadClass(Unknown Source)

	at java.base@11.0.11/java.lang.ClassLoader.loadClass(Unknown Source)

	at io.netty.buffer.ByteBufUtil.<clinit>(ByteBufUtil.java:1615)

	at io.netty.buffer.ByteBufAllocator.<clinit>(ByteBufAllocator.java:24)

	at io.netty.channel.DefaultChannelConfig.<init>(DefaultChannelConfig.java:60)

	at io.netty.channel.socket.DefaultDatagramChannelConfig.<init>(DefaultDatagramChannelConfig.java:55)

	at io.netty.channel.socket.nio.NioDatagramChannelConfig.<init>(NioDatagramChannelConfig.java:117)

	at io.netty.channel.socket.nio.NioDatagramChannel.<init>(NioDatagramChannel.java:152)

	at io.netty.channel.socket.nio.NioDatagramChannel.<init>(NioDatagramChannel.java:119)

	at io.vertx.core.net.impl.transport.Transport.datagramChannel(Transport.java:160)

	at io.vertx.core.impl.resolver.DnsResolverProvider$1.lambda$newResolver$0(DnsResolverProvider.java:122)

	at io.vertx.core.impl.resolver.DnsResolverProvider$1$$Lambda$436/0x0000000100659040.newChannel(Unknown Source)

	at io.netty.bootstrap.AbstractBootstrap.initAndRegister(AbstractBootstrap.java:310)

	at io.netty.bootstrap.AbstractBootstrap.register(AbstractBootstrap.java:227)

	at io.netty.resolver.dns.DnsNameResolver.<init>(DnsNameResolver.java:497)

	at io.netty.resolver.dns.DnsNameResolverBuilder.build(DnsNameResolverBuilder.java:488)

	at io.vertx.core.impl.resolver.DnsResolverProvider$1$1.newNameResolver(DnsResolverProvider.java:181)

	at io.netty.resolver.dns.DnsAddressResolverGroup.newResolver(DnsAddressResolverGroup.java:92)

	at io.netty.resolver.dns.DnsAddressResolverGroup.newResolver(DnsAddressResolverGroup.java:77)

	at io.netty.resolver.AddressResolverGroup.getResolver(AddressResolverGroup.java:70)

	at io.vertx.core.impl.resolver.DnsResolverProvider$1.newResolver(DnsResolverProvider.java:185)

	at io.netty.resolver.AddressResolverGroup.getResolver(AddressResolverGroup.java:70)

	at io.vertx.core.impl.AddressResolver.resolveHostname(AddressResolver.java:90)

	at io.vertx.core.impl.AddressResolver.resolveHostname(AddressResolver.java:83)

	at io.vertx.core.impl.VertxImpl.resolveAddress(VertxImpl.java:837)

	at io.vertx.core.net.impl.AsyncResolveConnectHelper.doBind(AsyncResolveConnectHelper.java:73)

	at io.vertx.core.net.impl.TCPServerBase.listen(TCPServerBase.java:130)

	at io.vertx.core.http.impl.HttpServerImpl.listen(HttpServerImpl.java:213)

	at io.vertx.core.http.impl.HttpServerImpl.listen(HttpServerImpl.java:143)

	at io.vertx.core.http.impl.HttpServerImpl.listen(HttpServerImpl.java:129)

	at io.vertx.core.http.impl.HttpServerImpl.listen(HttpServerImpl.java:134)

	at io.gravitee.node.management.http.vertx.verticle.ManagementVerticle.doStart(ManagementVerticle.java:148)

	at io.gravitee.node.management.http.vertx.verticle.ManagementVerticle.start(ManagementVerticle.java:91)

	at io.vertx.core.impl.DeploymentManager.lambda$doDeploy$5(DeploymentManager.java:196)

	at io.vertx.core.impl.DeploymentManager$$Lambda$401/0x0000000100625840.handle(Unknown Source)

	at io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:96)

	at io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:59)

	at io.vertx.core.impl.EventLoopContext.lambda$runOnContext$0(EventLoopContext.java:37)

	at io.vertx.core.impl.EventLoopContext$$Lambda$407/0x0000000100627840.run(Unknown Source)

	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)

	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)

	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)

	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)

	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)

	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)

	at java.base@11.0.11/java.lang.Thread.run(Unknown Source)

11:20:30.419 [graviteeio-node] [] INFO  o.e.j.s.h.ContextHandler.management - Initializing Spring root WebApplicationContext

11:20:30.923 [vert.x-eventloop-thread-1] [] INFO  i.g.n.m.h.v.e.ManagementEndpointManagerImpl - Register a new endpoint for Management API: GET / [io.gravitee.node.management.http.node.NodeEndpoint]

11:20:30.923 [vert.x-eventloop-thread-1] [] INFO  i.g.n.m.h.v.e.ManagementEndpointManagerImpl - Register a new endpoint for Management API: GET /configuration [io.gravitee.node.management.http.configuration.ConfigurationEndpoint]

11:20:30.924 [vert.x-eventloop-thread-1] [] INFO  i.g.n.m.h.v.v.ManagementVerticle - HTTP listener for Node Management bind to port TCP:18093

11:20:31.514 [vert.x-eventloop-thread-0] [] INFO  i.g.n.r.vertx.ReporterManagerImpl - 	There is no reporter to start

11:20:33.409 [graviteeio-node] [] INFO  i.g.a.m.h.m.a.a.m.i.i.IdentityProviderManagerImpl - Register event listener for identity provider events for all organizations

11:20:33.411 [graviteeio-node] [] INFO  i.g.c.event.impl.EventManagerImpl - Register new listener IdentityProviderManagerImpl for event type class io.gravitee.am.common.event.IdentityProviderEvent

11:20:33.411 [graviteeio-node] [] INFO  i.g.a.m.h.m.a.a.m.i.i.IdentityProviderManagerImpl - Initializing identity providers for all organizations

11:20:33.499 [graviteeio-node] [] INFO  i.g.a.m.h.m.a.a.m.i.i.IdentityProviderManagerImpl - Identity providers loaded for all organizations

11:20:34.048 [graviteeio-node] [] WARN  i.g.a.m.h.m.a.a.p.g.JWTGenerator - 

11:20:34.055 [graviteeio-node] [] WARN  i.g.a.m.h.m.a.a.p.g.JWTGenerator - ##############################################################

11:20:34.058 [graviteeio-node] [] WARN  i.g.a.m.h.m.a.a.p.g.JWTGenerator - #                      SECURITY WARNING                      #

11:20:34.058 [graviteeio-node] [] WARN  i.g.a.m.h.m.a.a.p.g.JWTGenerator - ##############################################################

11:20:34.058 [graviteeio-node] [] WARN  i.g.a.m.h.m.a.a.p.g.JWTGenerator - 

11:20:34.058 [graviteeio-node] [] WARN  i.g.a.m.h.m.a.a.p.g.JWTGenerator - You still use the default jwt secret.

11:20:34.058 [graviteeio-node] [] WARN  i.g.a.m.h.m.a.a.p.g.JWTGenerator - This known secret can be used to impersonate anyone.

11:20:34.058 [graviteeio-node] [] WARN  i.g.a.m.h.m.a.a.p.g.JWTGenerator - Please change this value, or ask your administrator to do it !

11:20:34.058 [graviteeio-node] [] WARN  i.g.a.m.h.m.a.a.p.g.JWTGenerator - 

11:20:34.058 [graviteeio-node] [] WARN  i.g.a.m.h.m.a.a.p.g.JWTGenerator - ##############################################################

11:20:34.058 [graviteeio-node] [] WARN  i.g.a.m.h.m.a.a.p.g.JWTGenerator - 

11:20:41.731 [graviteeio-node] [] INFO  o.s.s.web.DefaultSecurityFilterChain - Creating filter chain: Ant [pattern='/swagger.json'], []

11:20:42.314 [graviteeio-node] [] INFO  o.s.s.web.DefaultSecurityFilterChain - Creating filter chain: OrRequestMatcher [requestMatchers=[Ant [pattern='/auth/authorize'], Ant [pattern='/auth/login'], Ant [pattern='/auth/cockpit'], Ant [pattern='/auth/login/callback'], Ant [pattern='/auth/logout'], Ant [pattern='/auth/assets/**']]], [org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@1a67bde5, org.springframework.security.web.context.SecurityContextPersistenceFilter@498f1f63, org.springframework.security.web.header.HeaderWriterFilter@4d2f8ee7, org.springframework.web.filter.CorsFilter@5c648e38, org.springframework.security.web.csrf.CsrfFilter@30b29f55, io.gravitee.am.management.handlers.management.api.authentication.filter.CsrfIncludeFilter@6bc8d8bd, org.springframework.security.web.authentication.logout.LogoutFilter@2fd409c1, io.gravitee.am.management.handlers.management.api.authentication.filter.CockpitAuthenticationFilter@395c21ee, io.gravitee.am.management.handlers.management.api.authentication.filter.RecaptchaFilter@1e733bb5, io.gravitee.am.management.handlers.management.api.authentication.filter.CheckRedirectionCookieFilter@669e730d, io.gravitee.am.management.handlers.management.api.authentication.filter.BuiltInAuthenticationFilter@4b3e4253, io.gravitee.am.management.handlers.management.api.authentication.filter.SocialAuthenticationFilter@13b5598b, io.gravitee.am.management.handlers.management.api.authentication.filter.CheckAuthenticationCookieFilter@141c66db, org.springframework.security.web.authentication.UsernamePasswordAuthenticationFilter@5980e29f, org.springframework.security.web.savedrequest.RequestCacheAwareFilter@2a7b4bb6, org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@1c76b2fe, org.springframework.security.web.authentication.AnonymousAuthenticationFilter@18f794e, org.springframework.security.web.session.SessionManagementFilter@3a1e5fa7, org.springframework.security.web.access.ExceptionTranslationFilter@79735611, org.springframework.security.web.access.intercept.FilterSecurityInterceptor@3df54fd5]

11:20:42.465 [graviteeio-node] [] INFO  o.s.s.web.DefaultSecurityFilterChain - Creating filter chain: Ant [pattern='/auth/token'], [org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@30259c61, org.springframework.security.web.context.SecurityContextPersistenceFilter@5aec151b, org.springframework.security.web.header.HeaderWriterFilter@57f6fc09, org.springframework.security.web.authentication.logout.LogoutFilter@7e577eed, org.springframework.security.web.authentication.www.BasicAuthenticationFilter@5dc8448b, org.springframework.security.web.savedrequest.RequestCacheAwareFilter@74cf2d5f, org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@14b9817b, org.springframework.security.web.authentication.AnonymousAuthenticationFilter@33d08a24, org.springframework.security.web.session.SessionManagementFilter@ae5d544, org.springframework.security.web.access.ExceptionTranslationFilter@4e3c0627, org.springframework.security.web.access.intercept.FilterSecurityInterceptor@5bfeb9ba]

11:20:42.472 [graviteeio-node] [] INFO  o.s.s.web.DefaultSecurityFilterChain - Creating filter chain: OrRequestMatcher [requestMatchers=[Ant [pattern='/organizations/**'], Ant [pattern='/user/**'], Ant [pattern='/platform/**']]], [org.springframework.security.web.context.request.async.WebAsyncManagerIntegrationFilter@5af7e2f, org.springframework.security.web.context.SecurityContextPersistenceFilter@175c26ff, org.springframework.security.web.header.HeaderWriterFilter@1196537d, org.springframework.web.filter.CorsFilter@2df24f19, org.springframework.security.web.csrf.CsrfFilter@11c40a65, io.gravitee.am.management.handlers.management.api.authentication.filter.CsrfIncludeFilter@c76762, org.springframework.security.web.authentication.logout.LogoutFilter@45bbb3e8, io.gravitee.am.management.handlers.management.api.authentication.filter.JWTAuthenticationFilter@5318731f, org.springframework.security.web.savedrequest.RequestCacheAwareFilter@623bdc46, org.springframework.security.web.servletapi.SecurityContextHolderAwareRequestFilter@411576c7, org.springframework.security.web.authentication.AnonymousAuthenticationFilter@2db82155, org.springframework.security.web.session.SessionManagementFilter@267a8777, org.springframework.security.web.access.ExceptionTranslationFilter@68c3aa38, org.springframework.security.web.access.intercept.FilterSecurityInterceptor@80b70fd]

11:20:43.049 [graviteeio-node] [] INFO  i.g.a.m.h.m.a.a.m.f.i.FormManagerImpl - Register event listener for form events

11:20:43.066 [graviteeio-node] [] INFO  i.g.c.event.impl.EventManagerImpl - Register new listener FormManagerImpl for event type class io.gravitee.am.common.event.FormEvent

11:20:43.072 [graviteeio-node] [] INFO  i.g.a.m.h.m.a.a.m.f.i.FormManagerImpl - Initializing forms

11:20:43.432 [graviteeio-node] [] INFO  o.e.j.s.h.ContextHandler.management - Initializing Spring DispatcherServlet 'org.springframework.web.servlet.DispatcherServlet-362c48c'

@andrearossi90
Copy link
Author

andrearossi90 commented Oct 4, 2021 •
edited

Attaching the complete log
gateway.log

@leleueri
Copy link
Contributor

leleueri commented Oct 4, 2021 •
edited

Problem seems to come from the DNS resolution.
Could you please try to disable the vertx DNSRresolver using the following JVM option : -Dvertx.disableDnsResolver=true

gateway:
  env:
    - name: JAVA_OPTS
      value: -Dvertx.disableDnsResolver=true 

api:
  env:
    - name: JAVA_OPTS
      value: -Dvertx.disableDnsResolver=true

@andrearossi90
Copy link
Author

andrearossi90 commented Oct 4, 2021 •
edited

Hi, I'm using this command and the attached file helm upgrade --values am/values.yaml graviteeio-am graviteeio/am

values.yaml.txt

Edited for better clarity @leleueri

@andrearossi90
Copy link
Author

andrearossi90 commented Oct 4, 2021 •
edited

Edited for better clarity @leleueri , here the logs
gateway3.log
management3.log

In the last one can't catch the exception, it resets before I can download the log. Howerber we can see that it takes JAVA_OPTS correctly. Still not working.

=========================================================================

  Gravitee.io - Access Management - Management API
  GRAVITEE_HOME: /opt/graviteeio-am-management-api
  GRAVITEE_OPTS: 
  JAVA: /opt/java/openjdk/bin/java
  JAVA_OPTS: -Dvertx.disableDnsResolver=true -Djava.net.preferIPv4Stack=true -Xms256m -Xmx256m -Djava.awt.headless=true -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8
  CLASSPATH: /opt/graviteeio-am-management-api/lib/gravitee-am-management-api-standalone-bootstrap-3.11.0.jar

=========================================================================

=========================================================================

  Gravitee.io - Access Management - Gateway
  GRAVITEE_HOME: /opt/graviteeio-am-gateway
  GRAVITEE_OPTS: 
  JAVA: /opt/java/openjdk/bin/java
  JAVA_OPTS: -Dvertx.disableDnsResolver=true -Djava.net.preferIPv4Stack=true -Xms256m -Xmx256m -Djava.awt.headless=true -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8
  CLASSPATH: /opt/graviteeio-am-gateway/lib/gravitee-am-gateway-standalone-bootstrap-3.11.0.jar

=========================================================================

Gateway:

16:43:38.739 [cluster-ClusterId{value='615b2f31523437448262fe4e', description='null'}-graviteeio-am-mongodb-replicaset-2.graviteeio-am-mongodb-replicaset.default.svc.cluster.local:27017] [] INFO  org.mongodb.driver.cluster - Exception in monitor thread while connecting to server graviteeio-am-mongodb-replicaset-2.graviteeio-am-mongodb-replicaset.default.svc.cluster.local:27017
com.mongodb.MongoSocketWriteException: Exception sending message
	at com.mongodb.internal.connection.InternalStreamConnection.translateWriteException(InternalStreamConnection.java:659)
	at com.mongodb.internal.connection.InternalStreamConnection.sendMessage(InternalStreamConnection.java:536)
	at com.mongodb.internal.connection.InternalStreamConnection.sendCommandMessage(InternalStreamConnection.java:367)
	at com.mongodb.internal.connection.InternalStreamConnection.sendAndReceive(InternalStreamConnection.java:317)
	at com.mongodb.internal.connection.CommandHelper.sendAndReceive(CommandHelper.java:88)
	at com.mongodb.internal.connection.CommandHelper.executeCommand(CommandHelper.java:36)
	at com.mongodb.internal.connection.InternalStreamConnectionInitializer.initializeConnectionDescription(InternalStreamConnectionInitializer.java:129)
	at com.mongodb.internal.connection.InternalStreamConnectionInitializer.startHandshake(InternalStreamConnectionInitializer.java:71)
	at com.mongodb.internal.connection.InternalStreamConnection.open(InternalStreamConnection.java:167)
	at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.lookupServerDescription(DefaultServerMonitor.java:195)
	at com.mongodb.internal.connection.DefaultServerMonitor$ServerMonitorRunnable.run(DefaultServerMonitor.java:151)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.nio.channels.ClosedChannelException: null
	at com.mongodb.internal.connection.tlschannel.impl.TlsChannelImpl.handshake(TlsChannelImpl.java:524)
	at com.mongodb.internal.connection.tlschannel.impl.TlsChannelImpl.write(TlsChannelImpl.java:378)
	at com.mongodb.internal.connection.tlschannel.ClientTlsChannel.write(ClientTlsChannel.java:184)
	at com.mongodb.internal.connection.tlschannel.async.AsynchronousTlsChannelGroup.writeHandlingTasks(AsynchronousTlsChannelGroup.java:540)
	at com.mongodb.internal.connection.tlschannel.async.AsynchronousTlsChannelGroup.doWrite(AsynchronousTlsChannelGroup.java:498)
	at com.mongodb.internal.connection.tlschannel.async.AsynchronousTlsChannelGroup.lambda$processWrite$4(AsynchronousTlsChannelGroup.java:459)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	... 1 common frames omitted

@andrearossi90
Copy link
Author

andrearossi90 commented Oct 4, 2021 •
edited

@leleueri Using these JAVA_OPTS=-Dvertx.disableDnsResolver=true -Djdk.tls.client.protocols=TLSv1.2 and connection uri mongodb+srv://graviteeio-am-mongodb-replicaset.default.svc.cluster.local/?**tls=false**&ssl=false&serverSelectionTimeoutMS=5000&connectTimeoutMS=5000&socketTimeoutMS=5000&maxIdleTimeMS=30000
there are no more DNS o connection issues. The Thread blocked came back however.

Gateway:

17:42:17.310 [vertx-blocked-thread-checker] [] WARN  i.v.core.impl.BlockedThreadChecker - Thread Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 5722 ms, time limit is 2000 ms

io.vertx.core.VertxException: Thread blocked

	at java.base@11.0.11/java.lang.ClassLoader.defineClass1(Native Method)

	at java.base@11.0.11/java.lang.ClassLoader.defineClass(Unknown Source)

	at java.base@11.0.11/java.security.SecureClassLoader.defineClass(Unknown Source)

	at java.base@11.0.11/java.net.URLClassLoader.defineClass(Unknown Source)

	at java.base@11.0.11/java.net.URLClassLoader$1.run(Unknown Source)

	at java.base@11.0.11/java.net.URLClassLoader$1.run(Unknown Source)

	at java.base@11.0.11/java.security.AccessController.doPrivileged(Native Method)

	at java.base@11.0.11/java.net.URLClassLoader.findClass(Unknown Source)

	at java.base@11.0.11/java.lang.ClassLoader.loadClass(Unknown Source)

	at java.base@11.0.11/java.lang.ClassLoader.loadClass(Unknown Source)

	at io.gravitee.am.repository.mongodb.management.MongoNodeMonitoringRepository.findByNodeIdAndType(MongoNodeMonitoringRepository.java:58)

	at java.base@11.0.11/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

	at java.base@11.0.11/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

	at java.base@11.0.11/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

	at java.base@11.0.11/java.lang.reflect.Method.invoke(Unknown Source)

	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)

	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:205)

	at com.sun.proxy.$Proxy20.findByNodeIdAndType(Unknown Source)

	at io.gravitee.node.monitoring.NodeMonitoringService.createOrUpdate(NodeMonitoringService.java:69)

	at io.gravitee.node.monitoring.handler.NodeMonitoringEventHandler.handleNodeInfos(NodeMonitoringEventHandler.java:84)

	at io.gravitee.node.monitoring.handler.NodeMonitoringEventHandler.handleNodeInfosMessage(NodeMonitoringEventHandler.java:69)

	at io.gravitee.node.monitoring.handler.NodeMonitoringEventHandler$$Lambda$287/0x00000001004e8c40.handle(Unknown Source)

	at io.vertx.core.impl.EventLoopContext.emit(EventLoopContext.java:49)

	at io.vertx.core.impl.DuplicatedContext.emit(DuplicatedContext.java:167)

	at io.vertx.core.eventbus.impl.MessageConsumerImpl.dispatch(MessageConsumerImpl.java:177)

	at io.vertx.core.eventbus.impl.HandlerRegistration$InboundDeliveryContext.next(HandlerRegistration.java:163)

	at io.vertx.core.eventbus.impl.HandlerRegistration$InboundDeliveryContext.dispatch(HandlerRegistration.java:128)

	at io.vertx.core.eventbus.impl.HandlerRegistration$$Lambda$303/0x00000001004ffc40.run(Unknown Source)

	at io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:107)

	at io.vertx.core.eventbus.impl.HandlerRegistration.dispatch(HandlerRegistration.java:104)

	at io.vertx.core.eventbus.impl.MessageConsumerImpl.deliver(MessageConsumerImpl.java:183)

	at io.vertx.core.eventbus.impl.MessageConsumerImpl.doReceive(MessageConsumerImpl.java:168)

	at io.vertx.core.eventbus.impl.HandlerRegistration.lambda$receive$0(HandlerRegistration.java:54)

	at io.vertx.core.eventbus.impl.HandlerRegistration$$Lambda$302/0x00000001004f9c40.run(Unknown Source)

	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)

	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)

	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)

	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)

	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)

	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)

	at java.base@11.0.11/java.lang.Thread.run(Unknown Source)

17:42:18.486 [vertx-blocked-thread-checker] [] WARN  i.v.core.impl.BlockedThreadChecker - Thread Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 6745 ms, time limit is 2000 ms

io.vertx.core.VertxException: Thread blocked

	at io.gravitee.am.repository.mongodb.management.MongoNodeMonitoringRepository.findByNodeIdAndType(MongoNodeMonitoringRepository.java:58)

	at java.base@11.0.11/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

	at java.base@11.0.11/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

	at java.base@11.0.11/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

	at java.base@11.0.11/java.lang.reflect.Method.invoke(Unknown Source)

	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)

	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:205)

	at com.sun.proxy.$Proxy20.findByNodeIdAndType(Unknown Source)

	at io.gravitee.node.monitoring.NodeMonitoringService.createOrUpdate(NodeMonitoringService.java:69)

	at io.gravitee.node.monitoring.handler.NodeMonitoringEventHandler.handleNodeInfos(NodeMonitoringEventHandler.java:84)

	at io.gravitee.node.monitoring.handler.NodeMonitoringEventHandler.handleNodeInfosMessage(NodeMonitoringEventHandler.java:69)

	at io.gravitee.node.monitoring.handler.NodeMonitoringEventHandler$$Lambda$287/0x00000001004e8c40.handle(Unknown Source)

	at io.vertx.core.impl.EventLoopContext.emit(EventLoopContext.java:49)

	at io.vertx.core.impl.DuplicatedContext.emit(DuplicatedContext.java:167)

	at io.vertx.core.eventbus.impl.MessageConsumerImpl.dispatch(MessageConsumerImpl.java:177)

	at io.vertx.core.eventbus.impl.HandlerRegistration$InboundDeliveryContext.next(HandlerRegistration.java:163)

	at io.vertx.core.eventbus.impl.HandlerRegistration$InboundDeliveryContext.dispatch(HandlerRegistration.java:128)

	at io.vertx.core.eventbus.impl.HandlerRegistration$$Lambda$303/0x00000001004ffc40.run(Unknown Source)

	at io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:107)

	at io.vertx.core.eventbus.impl.HandlerRegistration.dispatch(HandlerRegistration.java:104)

	at io.vertx.core.eventbus.impl.MessageConsumerImpl.deliver(MessageConsumerImpl.java:183)

	at io.vertx.core.eventbus.impl.MessageConsumerImpl.doReceive(MessageConsumerImpl.java:168)

	at io.vertx.core.eventbus.impl.HandlerRegistration.lambda$receive$0(HandlerRegistration.java:54)

	at io.vertx.core.eventbus.impl.HandlerRegistration$$Lambda$302/0x00000001004f9c40.run(Unknown Source)

	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)

	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)

	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)

	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)

	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)

	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)

	at java.base@11.0.11/java.lang.Thread.run(Unknown Source)

17:42:19.599 [vertx-blocked-thread-checker] [] WARN  i.v.core.impl.BlockedThreadChecker - Thread Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 7753 ms, time limit is 2000 ms

io.vertx.core.VertxException: Thread blocked

	at java.base@11.0.11/java.io.RandomAccessFile.readBytes(Native Method)

	at java.base@11.0.11/java.io.RandomAccessFile.read(Unknown Source)

	at java.base@11.0.11/java.io.RandomAccessFile.readFully(Unknown Source)

	at java.base@11.0.11/java.util.zip.ZipFile$Source.readFullyAt(Unknown Source)

	at java.base@11.0.11/java.util.zip.ZipFile$ZipFileInputStream.initDataOffset(Unknown Source)

	at java.base@11.0.11/java.util.zip.ZipFile$ZipFileInputStream.read(Unknown Source)

	at java.base@11.0.11/java.util.zip.ZipFile$ZipFileInflaterInputStream.fill(Unknown Source)

	at java.base@11.0.11/java.util.zip.InflaterInputStream.read(Unknown Source)

	at java.base@11.0.11/jdk.internal.loader.Resource.getBytes(Unknown Source)

	at java.base@11.0.11/java.net.URLClassLoader.defineClass(Unknown Source)

	at java.base@11.0.11/java.net.URLClassLoader$1.run(Unknown Source)

	at java.base@11.0.11/java.net.URLClassLoader$1.run(Unknown Source)

	at java.base@11.0.11/java.security.AccessController.doPrivileged(Native Method)

	at java.base@11.0.11/java.net.URLClassLoader.findClass(Unknown Source)

	at java.base@11.0.11/java.lang.ClassLoader.loadClass(Unknown Source)

	at java.base@11.0.11/java.lang.ClassLoader.loadClass(Unknown Source)

	at io.gravitee.am.repository.mongodb.management.MongoNodeMonitoringRepository.findByNodeIdAndType(MongoNodeMonitoringRepository.java:58)

	at java.base@11.0.11/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

	at java.base@11.0.11/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

	at java.base@11.0.11/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

	at java.base@11.0.11/java.lang.reflect.Method.invoke(Unknown Source)

	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)

	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:205)

	at com.sun.proxy.$Proxy20.findByNodeIdAndType(Unknown Source)

	at io.gravitee.node.monitoring.NodeMonitoringService.createOrUpdate(NodeMonitoringService.java:69)

	at io.gravitee.node.monitoring.handler.NodeMonitoringEventHandler.handleNodeInfos(NodeMonitoringEventHandler.java:84)

	at io.gravitee.node.monitoring.handler.NodeMonitoringEventHandler.handleNodeInfosMessage(NodeMonitoringEventHandler.java:69)

	at io.gravitee.node.monitoring.handler.NodeMonitoringEventHandler$$Lambda$287/0x00000001004e8c40.handle(Unknown Source)

	at io.vertx.core.impl.EventLoopContext.emit(EventLoopContext.java:49)

	at io.vertx.core.impl.DuplicatedContext.emit(DuplicatedContext.java:167)

	at io.vertx.core.eventbus.impl.MessageConsumerImpl.dispatch(MessageConsumerImpl.java:177)

	at io.vertx.core.eventbus.impl.HandlerRegistration$InboundDeliveryContext.next(HandlerRegistration.java:163)

	at io.vertx.core.eventbus.impl.HandlerRegistration$InboundDeliveryContext.dispatch(HandlerRegistration.java:128)

	at io.vertx.core.eventbus.impl.HandlerRegistration$$Lambda$303/0x00000001004ffc40.run(Unknown Source)

	at io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:107)

	at io.vertx.core.eventbus.impl.HandlerRegistration.dispatch(HandlerRegistration.java:104)

	at io.vertx.core.eventbus.impl.MessageConsumerImpl.deliver(MessageConsumerImpl.java:183)

	at io.vertx.core.eventbus.impl.MessageConsumerImpl.doReceive(MessageConsumerImpl.java:168)

	at io.vertx.core.eventbus.impl.HandlerRegistration.lambda$receive$0(HandlerRegistration.java:54)

	at io.vertx.core.eventbus.impl.HandlerRegistration$$Lambda$302/0x00000001004f9c40.run(Unknown Source)

	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)

	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)

	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)

	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)

	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)

	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)

	at java.base@11.0.11/java.lang.Thread.run(Unknown Source)

17:42:20.353 [vertx-blocked-thread-checker] [] WARN  i.v.core.impl.BlockedThreadChecker - Thread Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 8768 ms, time limit is 2000 ms

io.vertx.core.VertxException: Thread blocked

	at java.base@11.0.11/java.lang.Throwable.fillInStackTrace(Native Method)

	at java.base@11.0.11/java.lang.Throwable.fillInStackTrace(Unknown Source)

	at java.base@11.0.11/java.lang.Throwable.<init>(Unknown Source)

	at java.base@11.0.11/java.lang.Exception.<init>(Unknown Source)

	at java.base@11.0.11/java.lang.ReflectiveOperationException.<init>(Unknown Source)

	at java.base@11.0.11/java.lang.ClassNotFoundException.<init>(Unknown Source)

	at java.base@11.0.11/jdk.internal.loader.BuiltinClassLoader.loadClass(Unknown Source)

	at java.base@11.0.11/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(Unknown Source)

	at java.base@11.0.11/java.lang.ClassLoader.loadClass(Unknown Source)

	at java.base@11.0.11/java.lang.ClassLoader.loadClass(Unknown Source)

	at io.reactivex.Observable.elementAt(Observable.java:8341)

	at io.reactivex.Observable.firstElement(Observable.java:8440)

	at io.gravitee.am.repository.mongodb.management.MongoNodeMonitoringRepository.findByNodeIdAndType(MongoNodeMonitoringRepository.java:59)

	at java.base@11.0.11/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

	at java.base@11.0.11/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)

	at java.base@11.0.11/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)

	at java.base@11.0.11/java.lang.reflect.Method.invoke(Unknown Source)

	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344)

	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:205)

	at com.sun.proxy.$Proxy20.findByNodeIdAndType(Unknown Source)

	at io.gravitee.node.monitoring.NodeMonitoringService.createOrUpdate(NodeMonitoringService.java:69)

	at io.gravitee.node.monitoring.handler.NodeMonitoringEventHandler.handleNodeInfos(NodeMonitoringEventHandler.java:84)

	at io.gravitee.node.monitoring.handler.NodeMonitoringEventHandler.handleNodeInfosMessage(NodeMonitoringEventHandler.java:69)

	at io.gravitee.node.monitoring.handler.NodeMonitoringEventHandler$$Lambda$287/0x00000001004e8c40.handle(Unknown Source)

	at io.vertx.core.impl.EventLoopContext.emit(EventLoopContext.java:49)

	at io.vertx.core.impl.DuplicatedContext.emit(DuplicatedContext.java:167)

	at io.vertx.core.eventbus.impl.MessageConsumerImpl.dispatch(MessageConsumerImpl.java:177)

	at io.vertx.core.eventbus.impl.HandlerRegistration$InboundDeliveryContext.next(HandlerRegistration.java:163)

	at io.vertx.core.eventbus.impl.HandlerRegistration$InboundDeliveryContext.dispatch(HandlerRegistration.java:128)

	at io.vertx.core.eventbus.impl.HandlerRegistration$$Lambda$303/0x00000001004ffc40.run(Unknown Source)

	at io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:107)

	at io.vertx.core.eventbus.impl.HandlerRegistration.dispatch(HandlerRegistration.java:104)

	at io.vertx.core.eventbus.impl.MessageConsumerImpl.deliver(MessageConsumerImpl.java:183)

	at io.vertx.core.eventbus.impl.MessageConsumerImpl.doReceive(MessageConsumerImpl.java:168)

	at io.vertx.core.eventbus.impl.HandlerRegistration.lambda$receive$0(HandlerRegistration.java:54)

	at io.vertx.core.eventbus.impl.HandlerRegistration$$Lambda$302/0x00000001004f9c40.run(Unknown Source)

	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)

	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)

	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)

	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)

	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)

	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)

	at java.base@11.0.11/java.lang.Thread.run(Unknown Source)

Management

17:46:05.963 [vertx-blocked-thread-checker] [] WARN  i.v.core.impl.BlockedThreadChecker - Thread Thread[vert.x-eventloop-thread-0,5,main] has been blocked for 5767 ms, time limit is 2000 ms

io.vertx.core.VertxException: Thread blocked

	at java.base@11.0.11/jdk.internal.misc.Unsafe.unpark(Native Method)

	at java.base@11.0.11/java.util.concurrent.locks.LockSupport.unpark(Unknown Source)

	at java.base@11.0.11/java.util.concurrent.locks.AbstractQueuedSynchronizer.unparkSuccessor(Unknown Source)

	at java.base@11.0.11/java.util.concurrent.locks.AbstractQueuedSynchronizer.release(Unknown Source)

	at java.base@11.0.11/java.util.concurrent.locks.ReentrantLock.unlock(Unknown Source)

	at java.base@11.0.11/java.util.concurrent.LinkedBlockingQueue.signalNotEmpty(Unknown Source)

	at java.base@11.0.11/java.util.concurrent.LinkedBlockingQueue.offer(Unknown Source)

	at java.base@11.0.11/java.util.AbstractQueue.add(Unknown Source)

	at com.mongodb.internal.connection.DefaultConnectionPool$AsyncWorkManager.enqueue(DefaultConnectionPool.java:1178)

	at com.mongodb.internal.connection.DefaultConnectionPool.getAsync(DefaultConnectionPool.java:179)

	at com.mongodb.internal.connection.DefaultServer.getConnectionAsync(DefaultServer.java:105)

	at com.mongodb.internal.binding.AsyncClusterBinding$AsyncClusterBindingConnectionSource.getConnection(AsyncClusterBinding.java:153)

	at com.mongodb.internal.async.client.ClientSessionBinding$SessionBindingAsyncConnectionSource.getConnection(ClientSessionBinding.java:179)

	at com.mongodb.internal.operation.OperationHelper.withAsyncConnectionSource(OperationHelper.java:731)

	at com.mongodb.internal.operation.OperationHelper.access$200(OperationHelper.java:69)

	at com.mongodb.internal.operation.OperationHelper$AsyncCallableWithConnectionAndSourceCallback.onResult(OperationHelper.java:751)

	at com.mongodb.internal.operation.OperationHelper$AsyncCallableWithConnectionAndSourceCallback.onResult(OperationHelper.java:739)

	at com.mongodb.internal.async.ErrorHandlingResultCallback.onResult(ErrorHandlingResultCallback.java:48)

	at com.mongodb.internal.async.client.ClientSessionBinding$WrappingCallback.onResult(ClientSessionBinding.java:267)

	at com.mongodb.internal.async.client.ClientSessionBinding$WrappingCallback.onResult(ClientSessionBinding.java:255)

	at com.mongodb.internal.binding.AsyncClusterBinding$1.onResult(AsyncClusterBinding.java:119)

	at com.mongodb.internal.binding.AsyncClusterBinding$1.onResult(AsyncClusterBinding.java:113)

	at com.mongodb.internal.connection.BaseCluster$ServerSelectionRequest.onResult(BaseCluster.java:430)

	at com.mongodb.internal.connection.BaseCluster.handleServerSelectionRequest(BaseCluster.java:294)

	at com.mongodb.internal.connection.BaseCluster.selectServerAsync(BaseCluster.java:155)

	at com.mongodb.internal.connection.AbstractMultiServerCluster.selectServerAsync(AbstractMultiServerCluster.java:50)

	at com.mongodb.internal.binding.AsyncClusterBinding.getAsyncClusterBindingConnectionSource(AsyncClusterBinding.java:113)

	at com.mongodb.internal.binding.AsyncClusterBinding.getReadConnectionSource(AsyncClusterBinding.java:98)

	at com.mongodb.internal.async.client.ClientSessionBinding.getReadConnectionSource(ClientSessionBinding.java:62)

	at com.mongodb.internal.operation.OperationHelper.withAsyncReadConnection(OperationHelper.java:678)

	at com.mongodb.internal.operation.FindOperation.executeAsync(FindOperation.java:692)

	at com.mongodb.reactivestreams.client.internal.OperationExecutorImpl.lambda$execute$3(OperationExecutorImpl.java:74)

	at com.mongodb.reactivestreams.client.internal.OperationExecutorImpl$$Lambda$468/0x00000001006bac40.accept(Unknown Source)

	at reactor.core.publisher.MonoCreate.subscribe(MonoCreate.java:57)

	at reactor.core.publisher.InternalMonoOperator.subscribe(InternalMonoOperator.java:64)

	at reactor.core.publisher.MonoFlatMap$FlatMapMain.onNext(MonoFlatMap.java:150)

	at reactor.core.publisher.FluxSwitchIfEmpty$SwitchIfEmptySubscriber.onNext(FluxSwitchIfEmpty.java:67)

	at reactor.core.publisher.FluxMapFuseable$MapFuseableSubscriber.onNext(FluxMapFuseable.java:121)

	at reactor.core.publisher.Operators$MonoSubscriber.complete(Operators.java:1812)

	at reactor.core.publisher.MonoCallable.subscribe(MonoCallable.java:61)

	at reactor.core.publisher.Mono.subscribe(Mono.java:4252)

	at io.reactivex.internal.operators.observable.ObservableFromPublisher.subscribeActual(ObservableFromPublisher.java:31)

	at io.reactivex.Observable.subscribe(Observable.java:12284)

	at io.reactivex.internal.operators.observable.ObservableElementAtMaybe.subscribeActual(ObservableElementAtMaybe.java:32)

	at io.reactivex.Maybe.subscribe(Maybe.java:4290)

	at io.reactivex.internal.operators.maybe.MaybeMap.subscribeActual(MaybeMap.java:40)

	at io.reactivex.Maybe.subscribe(Maybe.java:4290)

	at io.reactivex.internal.operators.maybe.MaybeFlatten.subscribeActual(MaybeFlatten.java:42)

	at io.reactivex.Maybe.subscribe(Maybe.java:4290)

	at io.reactivex.internal.operators.maybe.MaybeSwitchIfEmptySingle.subscribeActual(MaybeSwitchIfEmptySingle.java:45)

	at io.reactivex.Single.subscribe(Single.java:3666)

	at io.reactivex.internal.operators.single.SingleDoOnSuccess.subscribeActual(SingleDoOnSuccess.java:35)

	at io.reactivex.Single.subscribe(Single.java:3666)

	at io.reactivex.internal.operators.single.SingleDoFinally.subscribeActual(SingleDoFinally.java:44)

	at io.reactivex.Single.subscribe(Single.java:3666)

	at io.reactivex.Single.subscribe(Single.java:3652)

	at io.reactivex.Single.subscribe(Single.java:3565)

	at io.gravitee.node.monitoring.handler.NodeMonitoringEventHandler.handleNodeInfos(NodeMonitoringEventHandler.java:84)

	at io.gravitee.node.monitoring.handler.NodeMonitoringEventHandler.handleNodeInfosMessage(NodeMonitoringEventHandler.java:69)

	at io.gravitee.node.monitoring.handler.NodeMonitoringEventHandler$$Lambda$399/0x000000010061cc40.handle(Unknown Source)

	at io.vertx.core.impl.EventLoopContext.emit(EventLoopContext.java:49)

	at io.vertx.core.impl.DuplicatedContext.emit(DuplicatedContext.java:167)

	at io.vertx.core.eventbus.impl.MessageConsumerImpl.dispatch(MessageConsumerImpl.java:177)

	at io.vertx.core.eventbus.impl.HandlerRegistration$InboundDeliveryContext.next(HandlerRegistration.java:163)

	at io.vertx.core.eventbus.impl.HandlerRegistration$InboundDeliveryContext.dispatch(HandlerRegistration.java:128)

	at io.vertx.core.eventbus.impl.HandlerRegistration$$Lambda$411/0x000000010062d040.run(Unknown Source)

	at io.vertx.core.impl.AbstractContext.dispatch(AbstractContext.java:107)

	at io.vertx.core.eventbus.impl.HandlerRegistration.dispatch(HandlerRegistration.java:104)

	at io.vertx.core.eventbus.impl.MessageConsumerImpl.deliver(MessageConsumerImpl.java:183)

	at io.vertx.core.eventbus.impl.MessageConsumerImpl.doReceive(MessageConsumerImpl.java:168)

	at io.vertx.core.eventbus.impl.HandlerRegistration.lambda$receive$0(HandlerRegistration.java:54)

	at io.vertx.core.eventbus.impl.HandlerRegistration$$Lambda$408/0x000000010062c440.run(Unknown Source)

	at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:164)

	at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:472)

	at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:500)

	at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)

	at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)

	at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)

	at java.base@11.0.11/java.lang.Thread.run(Unknown Source)

I'm attaching the full logs, what can I try?

Thanks in advance

gateway4.log
management4.log

@andrearossi90
Copy link
Author

I switched to a clean environment. At the moment it works with:

  • mongodb://graviteeio-am-mongodb-replicaset-client:27017/?replicaSet=rs0&tls=false&ssl=false
  • JAVA_OPTS=-Dvertx.disableDnsResolver=true -Djdk.tls.client.protocols=TLSv1.2

Thank you for your support

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
project: AM
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@leleueri @andrearossi90