Skip to content
This repository has been archived by the owner on Feb 20, 2023. It is now read-only.

StageMonitor not able to connect to elasticSearch via ElasticsearchClient #475

Open
shubham-cashify opened this issue May 4, 2020 · 4 comments

Comments

@shubham-cashify
Copy link

shubham-cashify commented May 4, 2020

Following is my stagemonitor.properties

stagemonitor.applicationName=uzi-csh-appcandy
stagemonitor.instanceName=dev
stagemonitor.instrument.include=in.cashify.uzi.appcandy
stagemonitor.instrument.exclude=in.cashify.uzi.appcandy.dao.model
stagemonitor.reporting.interval.elasticsearch = 10
stagemonitor.reporting.elasticsearch.url= http://localhost:9200
stagemonitor.elasticsearch.numberOfReplicas= 0
stagemonitor.elasticsearch.numberOfShards= 1
stagemonitor.password=
stagemonitor.grafana.url = http://localhost:5601
stagemonitor.grafana.apiKey=
stagemonitor.alerts.frequency=10
stagemonitor.profiler.minExecutionTimeNanos=0
stagemonitor.requestmonitor.http.parseUserAgent=true
stagemonitor.eum.enabled=true

@shubham-cashify
Copy link
Author

shubham-cashify commented May 4, 2020

18:59:35.198 [main] INFO org.stagemonitor.tracing.soap.SoapHandlerTransformer - Adding SOAPHandlers [org.stagemonitor.tracing.soap.TracingServerSOAPHandler@3c0be339, org.stagemonitor.tracing.soap.TracingClientSOAPHandler@15ca7889]
18:59:35.414 [main] INFO org.stagemonitor.core.Stagemonitor - Measurement Session is initialized: [application=uzi-csh-appcandy] [instance=dev] [host=shb05-ThinkPad-E490]
18:59:35.427 [main] DEBUG org.stagemonitor.core.grafana.GrafanaClient - Not requesting grafana, because the url or the api key is not configured.
18:59:35.464 [elasticsearch-tasks] INFO org.stagemonitor.core.elasticsearch.ElasticsearchClient - wait 1s until core plugin is initialized
18:59:35.467 [main] DEBUG org.stagemonitor.core.grafana.GrafanaClient - Not requesting grafana, because the url or the api key is not configured.
18:59:35.468 [main] INFO org.stagemonitor.core.CorePlugin - Not sending metrics to InfluxDB (url=null, interval=60s)
18:59:35.473 [main] DEBUG org.stagemonitor.core.grafana.GrafanaClient - Not requesting grafana, because the url or the api key is not configured.
18:59:35.477 [main] DEBUG org.stagemonitor.core.grafana.GrafanaClient - Not requesting grafana, because the url or the api key is not configured.
18:59:36.062 [main] DEBUG org.stagemonitor.core.grafana.GrafanaClient - Not requesting grafana, because the url or the api key is not configured.
18:59:36.134 [main] INFO org.stagemonitor.alerting.AlertingPlugin - Using ElasticsearchIncidentRepository for storing incidents.
18:59:36.146 [main] DEBUG org.stagemonitor.core.grafana.GrafanaClient - Not requesting grafana, because the url or the api key is not configured.
18:59:36.149 [main] DEBUG org.stagemonitor.core.grafana.GrafanaClient - Not requesting grafana, because the url or the api key is not configured.
18:59:36.151 [main] INFO org.stagemonitor.web.servlet.ServletPlugin - Initializing ClientSpanExtensions [org.stagemonitor.web.servlet.eum.WeaselClientSpanExtension@7da10b5b]
18:59:36.152 [main] INFO org.stagemonitor.core.Stagemonitor - # stagemonitor status
18:59:36.152 [main] INFO org.stagemonitor.core.Stagemonitor - System information: Java 1.8.0_242 (Private Build) Linux 5.3.0-46-generic
18:59:36.153 [main] INFO org.stagemonitor.core.Stagemonitor - OK - Agent attachment
18:59:36.153 [main] INFO org.stagemonitor.core.Stagemonitor - OK - AlertingPlugin (version 0.89.1)
18:59:36.153 [main] INFO org.stagemonitor.core.Stagemonitor - OK - CorePlugin (version 0.89.1)
18:59:36.154 [main] INFO org.stagemonitor.core.Stagemonitor - OK - ElasticsearchTracingPlugin (version 0.89.1)
18:59:36.154 [main] INFO org.stagemonitor.core.Stagemonitor - OK - JdbcPlugin (version 0.89.1)
18:59:36.154 [main] INFO org.stagemonitor.core.Stagemonitor - OK - JvmPlugin (version 0.89.1)
18:59:36.154 [main] INFO org.stagemonitor.core.Stagemonitor - OK - LoggingPlugin (version 0.89.1)
18:59:36.154 [main] INFO org.stagemonitor.core.Stagemonitor - OK - OsPlugin (version 0.89.1)
18:59:36.154 [main] INFO org.stagemonitor.core.Stagemonitor - OK - ServletPlugin (version 0.89.1)
18:59:36.154 [main] INFO org.stagemonitor.core.Stagemonitor - OK - SoapTracingPlugin (version 0.89.1)
18:59:36.154 [main] INFO org.stagemonitor.core.Stagemonitor - OK - Startup
18:59:36.154 [main] INFO org.stagemonitor.core.Stagemonitor - OK - TracingPlugin (version 0.89.1)
18:59:36.154 [main] INFO org.stagemonitor.core.configuration.ConfigurationLogger - # stagemonitor configuration, listing non-default values:
18:59:36.154 [main] INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.requestmonitor.http.parseUserAgent: true (source: stagemonitor.properties)
18:59:36.154 [main] WARN org.stagemonitor.core.configuration.ConfigurationLogger - Detected usage of deprecated configuration option 'stagemonitor.requestmonitor.http.parseUserAgent'. This option might be removed in the future. Please refer to the documentation about alternatives.
18:59:36.154 [main] INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.eum.enabled: true (source: stagemonitor.properties)
18:59:36.154 [main] INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.alerts.frequency: 10 (source: stagemonitor.properties)
18:59:36.154 [main] INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.profiler.minExecutionTimeNanos: 0 (source: stagemonitor.properties)
18:59:36.155 [main] INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.reporting.interval.elasticsearch: 10 (source: stagemonitor.properties)
18:59:36.155 [main] INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.applicationName: uzi-csh-appcandy (source: stagemonitor.properties)
18:59:36.155 [main] INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.instanceName: dev (source: stagemonitor.properties)
18:59:36.155 [main] INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.reporting.elasticsearch.url: http://localhost:9200 (source: stagemonitor.properties)
18:59:36.155 [main] INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.instrument.exclude: in.cashify.uzi.appcandy.dao.model (source: stagemonitor.properties)
18:59:36.155 [main] INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.instrument.include: in.cashify.uzi.appcandy (source: stagemonitor.properties)
18:59:36.155 [main] INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.grafana.url: http://localhost:5601 (source: stagemonitor.properties)
18:59:36.155 [main] INFO org.stagemonitor.core.configuration.ConfigurationLogger - stagemonitor.grafana.apiKey: XXXX (source: stagemonitor.properties)

. ____ _ __ _ _
/\ / ' __ _ () __ __ _ \ \ \
( ( )_
_ | '_ | '| | ' / ` | \ \ \
\/ )| |)| | | | | || (| | ) ) ) )
' |
| .__|| ||| |_, | / / / /
=========|
|==============|/=////
:: Spring Boot :: (v1.5.4.RELEASE)

2020-05-04 18:59:36 [main] INFO i.c.uzi.appcandy.api.Application - Starting Application on shb05-ThinkPad-E490 with PID 22214 (/home/shb05/intellijWorkspace/uzi-csh-appcandy/uzi-appcandy-api/build/classes/main started by shb05 in /home/shb05/intellijWorkspace/uzi-csh-appcandy)
2020-05-04 18:59:36 [main] DEBUG i.c.uzi.appcandy.api.Application - Running with Spring Boot v1.5.4.RELEASE, Spring v4.3.9.RELEASE

ElasticSearch not mapped with stageMonitor

Also, In application.java
@component
public static class StageMonitorInitializer implements ServletContextInitializer {

    @Override
    public void onStartup(ServletContext servletContext) {
        // necessary for spring boot 2.0.0 until stagemonitor supports it natively
        ServletContainerInitializerUtil.registerStagemonitorServletContainerInitializers(servletContext);
    }
}

Here, I am using stageMonitor 0.89.1 version and byte-buddy 1.10.10 version.

Could you please suggest some solution for the same.

@thaarbach
Copy link
Contributor

@shubham-cashify can you tell me which es version you are using? There are missing es template directories for es 5.

@shubham-cashify
Copy link
Author

@shubham-cashify can you tell me which es version you are using? There are missing es template directories for es 5.

Hi, The above mentioned problem is resolved now.
However, I am facing this issue now.

13:47:19.831 [main] INFO org.stagemonitor.core.Stagemonitor - Measurement Session is initialized: [application=appcandy] [instance=dev] [host=shb05-ThinkPad-E490]
13:47:19.901 [async-elasticsearch] WARN org.stagemonitor.core.util.http.ErrorLoggingResponseHandler - http://localhost:9200/.kibana/_mapping/search: 400 {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, search]"}],"type":"illegal_argument_exception","reason":"Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, search]"},"status":400}
13:47:19.912 [async-elasticsearch] WARN org.stagemonitor.core.util.http.ErrorLoggingResponseHandler - http://localhost:9200/.kibana/_mapping/dashboard: 400 {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, dashboard]"}],"type":"illegal_argument_exception","reason":"Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, dashboard]"},"status":400}
13:47:19.925 [async-elasticsearch] WARN org.stagemonitor.core.util.http.ErrorLoggingResponseHandler - http://localhost:9200/.kibana/_mapping/visualization: 400 {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]"}],"type":"illegal_argument_exception","reason":"Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]"},"status":400}
13:47:19.928 [main] INFO org.stagemonitor.core.CorePlugin - Not sending metrics to InfluxDB (url=null, interval=60s)
13:47:19.930 [main] INFO org.stagemonitor.core.CorePlugin - Sending metrics to Elasticsearch ([http://localhost:9200]) every 10s
13:47:20.219 [async-elasticsearch] WARN org.stagemonitor.core.elasticsearch.ElasticsearchClient$BulkErrorReportingResponseHandler - Error(s) while sending a _bulk request to elasticsearch:

  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, dashboard]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
    13:47:20.565 [async-elasticsearch] WARN org.stagemonitor.core.elasticsearch.ElasticsearchClient$BulkErrorReportingResponseHandler - Error(s) while sending a _bulk request to elasticsearch:
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, dashboard]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
    13:47:20.651 [main] INFO org.stagemonitor.alerting.AlertingPlugin - Using ElasticsearchIncidentRepository for storing incidents.
    13:47:20.653 [main] INFO org.stagemonitor.core.Stagemonitor - # stagemonitor status
    13:47:20.654 [main] INFO org.stagemonitor.core.Stagemonitor - System information: Java 1.8.0_242 (Private Build) Linux 5.3.0-46-generic
    13:47:20.655 [main] INFO org.stagemonitor.core.Stagemonitor - OK - Agent attachment
    13:47:20.655 [main] INFO org.stagemonitor.core.Stagemonitor - OK - AlertingPlugin (version 0.84.0)
    13:47:20.655 [main] INFO org.stagemonitor.core.Stagemonitor - OK - CorePlugin (version 0.84.0)
    13:47:20.655 [main] INFO org.stagemonitor.core.Stagemonitor - OK - Elasticsearch
    13:47:20.656 [main] INFO org.stagemonitor.core.Stagemonitor - OK - ElasticsearchTracingPlugin (version 0.84.0)
    13:47:20.656 [main] INFO org.stagemonitor.core.Stagemonitor - OK - JdbcPlugin (version 0.84.0)
    13:47:20.656 [main] INFO org.stagemonitor.core.Stagemonitor - OK - JvmPlugin (version 0.84.0)
    13:47:20.656 [main] INFO org.stagemonitor.core.Stagemonitor - OK - LoggingPlugin (version 0.84.0)
    13:47:20.656 [main] INFO org.stagemonitor.core.Stagemonitor - OK - OsPlugin (version 0.84.0)
    13:47:20.656 [main] INFO org.stagemonitor.core.Stagemonitor - OK - ServletPlugin (version 0.84.0)
    13:47:20.656 [main] INFO org.stagemonitor.core.Stagemonitor - OK - SoapTracingPlugin (version 0.84.0)
    13:47:20.656 [main] INFO org.stagemonitor.core.Stagemonitor - OK - Startup
    13:47:20.656 [main] INFO org.stagemonitor.core.Stagemonitor - OK - TracingPlugin (version 0.84.0)
    13:47:20.656 [main] INFO org.stagemonitor.core.Stagemonitor - # stagemonitor configuration, listing non-default values:
    13:47:20.657 [main] INFO org.stagemonitor.core.Stagemonitor - stagemonitor.requestmonitor.http.parseUserAgent: true (source: stagemonitor.properties)
    13:47:20.657 [main] INFO org.stagemonitor.core.Stagemonitor - stagemonitor.web.paths.excluded: /resources, /webjars, /dandelion (source: stagemonitor.properties)
    13:47:20.657 [main] INFO org.stagemonitor.core.Stagemonitor - stagemonitor.eum.enabled: true (source: stagemonitor.properties)
    13:47:20.657 [main] INFO org.stagemonitor.core.Stagemonitor - stagemonitor.alerts.frequency: 10 (source: stagemonitor.properties)
    13:47:20.657 [main] INFO org.stagemonitor.core.Stagemonitor - stagemonitor.reporting.interval.elasticsearch: 10 (source: stagemonitor.properties)
    13:47:20.658 [main] INFO org.stagemonitor.core.Stagemonitor - stagemonitor.applicationName: appcandy (source: stagemonitor.properties)
    13:47:20.658 [main] INFO org.stagemonitor.core.Stagemonitor - stagemonitor.instanceName: dev (source: stagemonitor.properties)
    13:47:20.658 [main] INFO org.stagemonitor.core.Stagemonitor - stagemonitor.reporting.elasticsearch.url: http://localhost:9200 (source: stagemonitor.properties)
    13:47:20.658 [main] INFO org.stagemonitor.core.Stagemonitor - stagemonitor.instrument.exclude: in.cashify.uzi.appcandy.dao.model (source: stagemonitor.properties)
    13:47:20.658 [main] INFO org.stagemonitor.core.Stagemonitor - stagemonitor.instrument.include: in.cashify.uzi.appcandy (source: stagemonitor.properties)
    13:47:20.658 [main] INFO org.stagemonitor.core.Stagemonitor - stagemonitor.grafana.url: http://localhost:3000 (source: stagemonitor.properties)
    13:47:20.658 [main] INFO org.stagemonitor.core.Stagemonitor - stagemonitor.grafana.apiKey: XXXX (source: stagemonitor.properties)
    13:47:20.665 [async-elasticsearch] WARN org.stagemonitor.core.elasticsearch.ElasticsearchClient$BulkErrorReportingResponseHandler - Error(s) while sending a _bulk request to elasticsearch:
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, dashboard]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
    13:47:20.722 [async-elasticsearch] WARN org.stagemonitor.core.elasticsearch.ElasticsearchClient$BulkErrorReportingResponseHandler - Error(s) while sending a _bulk request to elasticsearch:
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, dashboard]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
    13:47:20.775 [async-elasticsearch] WARN org.stagemonitor.core.elasticsearch.ElasticsearchClient$BulkErrorReportingResponseHandler - Error(s) while sending a _bulk request to elasticsearch:
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, dashboard]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]
  • Rejecting mapping update to [.kibana] as the final mapping would have more than 1 type: [index-pattern, visualization]

. ____ _ __ _ _
/\ / ' __ _ () __ __ _ \ \ \
( ( )_
_ | '_ | '| | ' / ` | \ \ \
\/ )| |)| | | | | || (| | ) ) ) )
' |
| .__|| ||| |_, | / / / /
=========|
|==============|/=////
:: Spring Boot :: (v1.5.4.RELEASE)

2020-05-07 13:47:21 [main] INFO i.c.uzi.appcandy.api.Application - Starting Application on shb05-ThinkPad-E490 with PID 30760 (/home/shb05/intellijWorkspace/uzi-csh-appcandy/uzi-appcandy-api/build/classes/main started by shb05 in /home/shb05/intellijWorkspace/uzi-csh-appcandy)
2020-05-07 13:47:21 [main] DEBUG i.c.uzi.appcandy.api.Application - Running with Spring Boot v1.5.4.RELEASE, Spring v4.3.9.RELEASE

I am using ES 6.5 version alongwith grafana for visulization

@thaarbach
Copy link
Contributor

thaarbach commented May 7, 2020

@shubham-cashify can you try to delete the .kibana index. No worries, stagemonitor will create it again. If you have upgrade es / kibana this is known problem as mentioned here
https://discuss.elastic.co/t/fresh-install-kibana-rejecting-mapping-update-to-kibana-as-the-final-mapping-would-have-more-than-1-type-log-doc/117413

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants