CPU usage rising with elasticsearch-jdbc 2.1.0.0 #752
Comments
|
Hi Arthur, hi everybody, |
|
Can you post somewhere the output of a thread dump created by |
|
Hi Jörg, |
|
Very strange, threads do not exit as they should. What Linux kernel version is this? |
|
it's a CentOS release 6.7 (Final) |
|
Thanks. In May I have some days off and can work on this - seems like a regression. |
|
Ok thanks! |
|
+1 on windows 7 this time - memory usage >4gb constant cpu time 80% with everyminute schedule and empty results sets (that is expected) |
|
I will work on this and it will be fixed next release. |
|
I have the same issue. My configuration:
After about 5h of working my server is dead and I need to reset it. I think that a temporary workaround it can be to use windows scheduler to execute jdbc importer bat file every one minute. |
|
is it the same like 842? and it s fixed now? |
|
It looks familiar I will deploy it tommorow and verify whether it works. |
|
@adamlepkowski ok, great. Can you please share your results. Thank you! |
|
@tipler79 everything work like a charm. I have currently about 10 jdbc services executed every 1 minute and after 12h of server work everything works perfectly. CPU usage jumps only for execution time but nothing special. |
|
Perfect! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Hi!
Previously working on jdbc importer and elasticsearch with version 1.7.0.1 and 1.7.0, everything was working fine with your elasticsearch-jdbc.
Nevertheless, I'm now using elasticsearch version 2.1.0 and JDBC Importer 2.1.0.0 and I am facing an issue with the CPU usage of the JDBC Importer.
You can see the cpu_usage_per_process acquired through topbeat but I can see the same fact (the cpu usage is linearly growing) with other tools.
Here is part of my configuration:
{
"type": "jdbc",
"jdbc": {
"driver": "org.postgresql.Driver",
"strategy": "simple",
"schedule": "30 0/1 * ? * *",
"url": "%DB_URI%",
"user": "%DB_USER%",
"password": "%DB_PASSWORD%",
"index": "test",
"type": "test_product",
"sql": {
"statement": "/usr/share/elasticsearch/scripts/update.sql",
"parameter": ["$metrics.lastexecutionstart", "$metrics.lastexecutionstart"]
}
}
}
And I know that the sql script don't get any rows. So we can eliminate that from cpu usage.
Is that a known issue? Am I the only one facing that phenomenon?
I could provide the logging in debug mode but I didn't see anything which seems abnormal. If you want some data don't hesitate!
PS: Here is some of the INFO of the logger:
[14:16:30,003][INFO ][importer.jdbc ][pool-2-thread-2] index name = test, concrete index name = test
[14:16:30,008][INFO ][importer.jdbc ][pool-516-thread-1] strategy simple: settings = "******" (my_settings :))
[14:16:30,008][INFO ][importer.jdbc.context.standard][pool-516-thread-1] found sink class org.xbib.elasticsearch.jdbc.strategy.standard.StandardSink@30f223c2
[14:16:30,009][INFO ][importer.jdbc.context.standard][pool-516-thread-1] found source class org.xbib.elasticsearch.jdbc.strategy.standard.StandardSource@a3be548
[14:16:30,009][INFO ][org.xbib.elasticsearch.helper.client.BaseTransportClient][pool-516-thread-1] creating transport client on Linux OpenJDK 64-Bit Server VM Oracle Corporation 1.8.0_66-internal-b17 25.66-b17 with effective settings {autodiscover=false, client.transport.ignore_cluster_name=false, client.transport.nodes_sampler_interval=5s, client.transport.ping_timeout=5s,
[14:16:30,010][INFO ][org.elasticsearch.plugins][pool-516-thread-1] [importer] loaded [helper], sites []
[14:16:30,109][INFO ][org.xbib.elasticsearch.helper.client.BaseTransportClient][pool-516-thread-1] trying to connect to [localhost/127.0.0.1:9300]
[14:16:30,136][INFO ][org.xbib.elasticsearch.helper.client.BaseTransportClient][pool-516-thread-1] connected to [{Outrage}{YOoML0EdQ5aQyEh8FOWm8A}{172.17.4.47}{localhost/127.0.0.1:9300}]
[14:16:30,142][INFO ][importer.jdbc.sink.standard][pool-516-thread-1] creating index test with settings = and mappings =
[14:16:30,160][WARN ][importer.jdbc.sink.standard][pool-516-thread-1] already exists
Arthur
The text was updated successfully, but these errors were encountered: