Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Issue with NettyRuntime$AvailableProcessorsHolder: number of available processors #6956

Closed
balteo opened this issue Jul 9, 2017 · 7 comments
Closed

Issue with NettyRuntime$AvailableProcessorsHolder: number of available processors #6956

balteo opened this issue Jul 9, 2017 · 7 comments

Comments

@balteo
Copy link

balteo commented Jul 9, 2017

Actual behavior

Exception is thrown when both Redis/lettuce & Elasticsearch clients try to set number of available processors on netty.

Stacktrace

Caused by: java.lang.IllegalStateException: availableProcessors is already set to [8], rejecting [8]
        at io.netty.util.NettyRuntime$AvailableProcessorsHolder.setAvailableProcessors(NettyRuntime.java:51) ~[netty-common-4.1.12.Final.jar:4.1.12.Final]
        at io.netty.util.NettyRuntime.setAvailableProcessors(NettyRuntime.java:87) ~[netty-common-4.1.12.Final.jar:4.1.12.Final]
        at org.elasticsearch.transport.netty4.Netty4Utils.setAvailableProcessors(Netty4Utils.java:82) ~[transport-netty4-client-5.5.0.jar:5.5.0]
        at org.elasticsearch.transport.netty4.Netty4Transport.<init>(Netty4Transport.java:138) ~[transport-netty4-client-5.5.0.jar:5.5.0]
        at org.elasticsearch.xpack.security.transport.netty4.SecurityNetty4Transport.<init>(SecurityNetty4Transport.java:71) ~[x-pack-api-5.5.0.jar:5.5.0]
        at org.elasticsearch.xpack.security.Security.lambda$getTransports$13(Security.java:842) ~[x-pack-api-5.5.0.jar:5.5.0]
        at org.elasticsearch.client.transport.TransportClient.buildTemplate(TransportClient.java:174) ~[elasticsearch-5.5.0.jar:5.5.0]
        at org.elasticsearch.client.transport.TransportClient.<init>(TransportClient.java:265) ~[elasticsearch-5.5.0.jar:5.5.0]
        at org.elasticsearch.transport.client.PreBuiltTransportClient.<init>(PreBuiltTransportClient.java:130) ~[transport-5.5.0.jar:5.5.0]
        at org.elasticsearch.xpack.client.PreBuiltXPackTransportClient.<init>(PreBuiltXPackTransportClient.java:55) ~[x-pack-transport-5.5.0.jar:5.5.0]
        at org.elasticsearch.xpack.client.PreBuiltXPackTransportClient.<init>(PreBuiltXPackTransportClient.java:50) ~[x-pack-transport-5.5.0.jar:5.5.0]
        at org.elasticsearch.xpack.client.PreBuiltXPackTransportClient.<init>(PreBuiltXPackTransportClient.java:46) ~[x-pack-transport-5.5.0.jar:5.5.0]
        at com.bignibou.configuration.search.cloud.connector.ElasticsearchTransportClientCreator.create(ElasticsearchTransportClientCreator.java:38) ~[bignibou-common.jar:?]
        at com.bignibou.configuration.search.cloud.connector.ElasticsearchTransportClientCreator.create(ElasticsearchTransportClientCreator.java:23) ~[bignibou-common.jar:?]
        at org.springframework.cloud.Cloud.getServiceConnector(Cloud.java:245) ~[spring-cloud-core-2.0.0.M1.jar:?]
        at org.springframework.cloud.Cloud.getSingletonServiceConnector(Cloud.java:155) ~[spring-cloud-core-2.0.0.M1.jar:?]
        at org.springframework.cloud.config.java.CloudServiceConnectionFactory.service(CloudServiceConnectionFactory.java:327) ~[spring-cloud-spring-service-connector-2.0.0.M1.jar:?]
        at com.bignibou.configuration.search.ElasticsearchConfiguration.client(ElasticsearchConfiguration.java:18) ~[bignibou-common.jar:?]
        at com.bignibou.configuration.search.ElasticsearchConfiguration$$EnhancerBySpringCGLIB$$7ea6662d.CGLIB$client$0(<generated>) ~[bignibou-common.jar:?]
        at com.bignibou.configuration.search.ElasticsearchConfiguration$$EnhancerBySpringCGLIB$$7ea6662d$$FastClassBySpringCGLIB$$e0f8345b.invoke(<generated>) ~[bignibou-common.jar:?]
        at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:228) ~[spring-core-5.0.0.RC2.jar:5.0.0.RC2]
        at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:359) ~[spring-context-5.0.0.RC2.

Netty version

4.1.12.Final

JVM version (e.g. java -version)

java version "1.8.0_102"
Java(TM) SE Runtime Environment (build 1.8.0_102-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)

OS version (e.g. uname -a)

Darwin Kernel Version 16.4.0
@jasontedor
Copy link
Contributor

This is not a Netty problem. You either need to ensure in your application that the Elasticsearch client is initialized first, or set the system property es.set.netty.runtime.available.processors to false so that the client does not try to set the number of processors when the client initializes.

@balteo balteo closed this as completed Jul 9, 2017
@seut seut mentioned this issue Nov 17, 2017
Merged
@niaomingjian
Copy link

@jasontedor
Why not check in org.elasticsearch.transport.netty4.Netty4Utils to prevent setting AvailableProcessors again?
If so, developers don't need to configure this by themselves.

andreidan added a commit to crate/jepsen that referenced this issue Nov 21, 2017
@andreidan
Avoid "availableProcessors is already set" exception.
d840748 
netty/netty#6956
@andreidan andreidan mentioned this issue Nov 21, 2017
Merged
andreidan pushed a commit to crate/jepsen that referenced this issue Nov 23, 2017
@seut @andreidan
Avoid "availableProcessors is already set" exception.
0c1ad22 
Ensures that the ES client does not try to set the number of processors
when the client initializes. it could be already set at inside the same
JVM which raises an exception.
Although this setting is just an upper limit to prevent too many threads
on machines with lot of cores (>32), this is not relevant for our tests
here.

netty/netty#6956
andreidan pushed a commit to crate/jepsen that referenced this issue Nov 23, 2017
@seut @andreidan
Avoid "availableProcessors is already set" exception.
379f6e4 
Ensures that the ES client does not try to set the number of processors
when the client initializes. it could be already set at inside the same
JVM which raises an exception.
Although this setting is just an upper limit to prevent too many threads
on machines with lot of cores (>32), this is not relevant for our tests
here.

netty/netty#6956
andreidan pushed a commit to crate/jepsen that referenced this issue Nov 23, 2017
@seut @andreidan
Avoid "availableProcessors is already set" exception.
691781a 
Ensures that the ES client does not try to set the number of processors
when the client initializes. it could be already set at inside the same
JVM which raises an exception.
Although this setting is just an upper limit to prevent too many threads
on machines with lot of cores (>32), this is not relevant for our tests
here.

netty/netty#6956
@andreidan andreidan mentioned this issue Nov 23, 2017
Merged
@porunov
Copy link

porunov commented Jun 12, 2018

I am getting this exception with Elasticsearch java Transport client v. 6.2.4 if I don't set the system property:
es.set.netty.runtime.available.processors=false.

@imrimt
Copy link

imrimt commented Nov 20, 2018

Still getting this exception with Elasticsearch 6.4.2 as well. Setting the system property like @porunov mentioned above fixes the issue -- however, this is indeed a hack that could be avoid by checking for existing value before setting the # of processors in the transport client's initialization.

@Qk1hpys
Copy link

Qk1hpys commented Mar 7, 2019 •
edited

spring-boot-starter-data-redis&spring-boot-starter-data-elasticsearch still getting this problem now. And setting -Des.set.netty.runtime.available.processors=false when starting jvm can resolve that.

@normanmaurer
Copy link
Member

@jasontedor any idea ?

@wakedeer
Copy link

wakedeer commented Sep 13, 2019 •
edited

spring-boot-starter-data-redis&spring-boot-starter-data-elasticsearch still getting this problem now. And setting -Des.set.netty.runtime.available.processors=false when starting jvm can resolve that.

It's just a hack, but I didn't find any elegant solutions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
8 participants
@normanmaurer @balteo @jasontedor @niaomingjian @imrimt @wakedeer @porunov @Qk1hpys