Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Setting xpack.security.enabled=false does not stop the master from trying to authenticate with tribe node #23515

Closed
dforste opened this issue Mar 9, 2017 · 3 comments
Closed

Setting xpack.security.enabled=false does not stop the master from trying to authenticate with tribe node #23515

dforste opened this issue Mar 9, 2017 · 3 comments

Comments

@dforste
Copy link

dforste commented Mar 9, 2017 •
edited

When running elasticsearch in docker per the documentation with xpack.security.enabled=false tribe nodes connect to the master and are immediately disconnected.

Elasticsearch version:
Docker image: docker.elastic.co/elasticsearch/elasticsearch:5.2.2
Plugins installed: [x-pack]

Description of the problem including expected versus actual behavior:
Expected tribe node to be able to connect to master.

Steps to reproduce:

  1. This should be the minimal docker-compose.yaml to reproduce:
version: '2'
services:
  master:
    image: docker.elastic.co/elasticsearch/elasticsearch:5.2.2
    environment:
      - cluster.name=logs.es
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - node.data=false
      - node.master=true
      - network.publish_host=172.20.5.193
      - node.name=hostname-master
      - http.compression=true
      - http.port=9200
      - transport.tcp.port=9300
      - transport.bind_host=_eth0_
      - xpack.security.enabled=false
      - xpack.watcher.enabled=false
      - discovery.zen.ping.unicast.hosts=172.20.5.193:9300
    volumes:
    - /app/data/elk/master/data:/usr/share/elasticsearch/data
    - /app/data/elk/master/logs:/usr/share/elasticsearch/logs
    ports:
      - "9200:9200"
      - "9300:9300"
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    cap_add:
      - IPC_LOCK

  tribe:
    image: docker.elastic.co/elasticsearch/elasticsearch:5.2.2
    environment:
      - cluster.name=logs.es
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
      - network.publish_host=172.20.5.193
      - node.name=hostname-tribe
      - http.compression=true
      - http.port=9201
      - transport.tcp.port=9301
      - transport.bind_host=_eth0_
      - tribe.cluster1.cluster.name=logs.es
      - tribe.cluster1.discovery.zen.ping.unicast.hosts=172.20.5.193:9300
      - tribe.cluster1.transport.tcp.port=9302
      - xpack.security.enabled=false
      - xpack.watcher.enabled=false
    volumes:
      - /app/data/elk/tribe/data:/usr/share/elasticsearch/data
      - /app/data/elk/tribe/logs:/usr/share/elasticsearch/logs
    ports:
      - "9201:9201"
      - "9301:9301"
      - "9302:9302"
    ulimits:
      memlock:
        soft: -1
        hard: -1
      nofile:
        soft: 65536
        hard: 65536
    cap_add:
      - IPC_LOCK

Provide logs (if relevant):
Logs from the master showing the issue:

[2017-03-09T01:54:52,941][INFO ][o.e.c.s.ClusterService   ] [hostname-master] removed {{hostname-tribe/cluster1}{3qO9HNDCT3mFj2dnnuuDsg}{JLFzoCPmS8KjrSi1Skxq0w}{172.20.5.193}{172.20.5.193:9302},}, reason: zen-disco-node-failed({hostname-tribe/cluster1}{3qO9HNDCT3mFj2dnnuuDsg}{JLFzoCPmS8KjrSi1Skxq0w}{172.20.5.193}{172.20.5.193:9302}), reason(failed to ping, tried [3] times, each with maximum [30s] timeout)[{hostname-tribe/cluster1}{3qO9HNDCT3mFj2dnnuuDsg}{JLFzoCPmS8KjrSi1Skxq0w}{172.20.5.193}{172.20.5.193:9302} failed to ping, tried [3] times, each with maximum [30s] timeout], zen-disco-node-failed({hostname-tribe/cluster1}{1ME8L_ejRwmC1ySEt6aZLw}{XJGFTz1uSk-hwHN9kxSvDg}{172.20.5.195}{172.20.5.193:9302}), reason(failed to ping, tried [3] times, each with maximum [30s] timeout)[{hostname-tribe/cluster1}{1ME8L_ejRwmC1ySEt6aZLw}{XJGFTz1uSk-hwHN9kxSvDg}{172.20.5.193}{172.20.5.193:9302} failed to ping, tried [3] times, each with maximum [30s] timeout]
[2017-03-09T01:54:52,946][WARN ][o.e.a.a.c.n.i.TransportNodesInfoAction] [hostname-master] not accumulating exceptions, excluding exception from response
org.elasticsearch.action.FailedNodeException: Failed node [3qO9HNDCT3mFj2dnnuuDsg]
        at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction.onFailure(TransportNodesAction.java:247) ~[elasticsearch-5.2.2.jar:5.2.2]
        at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction.access$300(TransportNodesAction.java:160) ~[elasticsearch-5.2.2.jar:5.2.2]
        at org.elasticsearch.action.support.nodes.TransportNodesAction$AsyncAction$1.handleException(TransportNodesAction.java:219) ~[elasticsearch-5.2.2.jar:5.2.2]
        at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1024) ~[elasticsearch-5.2.2.jar:5.2.2]
        at org.elasticsearch.transport.TcpTransport.lambda$handleException$17(TcpTransport.java:1411) ~[elasticsearch-5.2.2.jar:5.2.2]
        at org.elasticsearch.common.util.concurrent.EsExecutors$1.execute(EsExecutors.java:109) [elasticsearch-5.2.2.jar:5.2.2]
        at org.elasticsearch.transport.TcpTransport.handleException(TcpTransport.java:1409) ~[elasticsearch-5.2.2.jar:5.2.2]
        at org.elasticsearch.transport.TcpTransport.handlerResponseError(TcpTransport.java:1401) [elasticsearch-5.2.2.jar:5.2.2]
        at org.elasticsearch.transport.TcpTransport.messageReceived(TcpTransport.java:1345) [elasticsearch-5.2.2.jar:5.2.2]
        at org.elasticsearch.transport.netty4.Netty4MessageChannelHandler.channelRead(Netty4MessageChannelHandler.java:74) [transport-netty4-5.2.2.jar:5.2.2]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:293) [netty-codec-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:280) [netty-codec-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:396) [netty-codec-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:248) [netty-codec-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:341) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1334) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:363) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:349) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:926) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:129) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:642) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:527) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:481) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:441) [netty-transport-4.1.7.Final.jar:4.1.7.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:858) [netty-common-4.1.7.Final.jar:4.1.7.Final]
        at java.lang.Thread.run(Thread.java:745) [?:1.8.0_92-internal]
Caused by: org.elasticsearch.transport.RemoteTransportException: [hostname-tribe/cluster1][172.17.0.4:9302][cluster:monitor/nodes/info[n]]
Caused by: org.elasticsearch.ElasticsearchSecurityException: failed to authenticate user [elastic]
        at org.elasticsearch.xpack.security.support.Exceptions.authenticationError(Exceptions.java:39) ~[?:?]
        at org.elasticsearch.xpack.security.authc.esnative.ReservedRealm.lambda$null$2(ReservedRealm.java:95) ~[?:?]
        at org.elasticsearch.xpack.security.authc.esnative.ReservedRealm.lambda$doAuthenticate$3(ReservedRealm.java:99) ~[?:?]
        at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:56) ~[elasticsearch-5.2.2.jar:5.2.2]
        at org.elasticsearch.xpack.security.authc.esnative.ReservedRealm.getUserInfo(ReservedRealm.java:193) ~[?:?]
        at org.elasticsearch.xpack.security.authc.esnative.ReservedRealm.doAuthenticate(ReservedRealm.java:78) ~[?:?]
        at org.elasticsearch.xpack.security.authc.support.CachingUsernamePasswordRealm.doAuthenticateAndCache(CachingUsernamePasswordRealm.java:139) ~[?:?]
        at org.elasticsearch.xpack.security.authc.support.CachingUsernamePasswordRealm.authenticateWithCache(CachingUsernamePasswordRealm.java:106) ~[?:?]
        at org.elasticsearch.xpack.security.authc.support.CachingUsernamePasswordRealm.authenticate(CachingUsernamePasswordRealm.java:92) ~[?:?]
        at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$consumeToken$9(AuthenticationService.java:253) ~[?:?]
        at org.elasticsearch.xpack.common.IteratingActionListener.run(IteratingActionListener.java:58) ~[?:?]
        at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.consumeToken(AuthenticationService.java:272) ~[?:?]
        at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$extractToken$6(AuthenticationService.java:228) ~[?:?]
        at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.extractToken(AuthenticationService.java:236) ~[?:?]
        at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$authenticateAsync$0(AuthenticationService.java:184) ~[?:?]
        at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lambda$lookForExistingAuthentication$2(AuthenticationService.java:201) ~[?:?]
        at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.lookForExistingAuthentication(AuthenticationService.java:213) ~[?:?]
        at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.authenticateAsync(AuthenticationService.java:180) ~[?:?]
        at org.elasticsearch.xpack.security.authc.AuthenticationService$Authenticator.access$000(AuthenticationService.java:142) ~[?:?]
        at org.elasticsearch.xpack.security.authc.AuthenticationService.authenticate(AuthenticationService.java:114) ~[?:?]
        at org.elasticsearch.xpack.security.transport.ServerTransportFilter$NodeProfile.inbound(ServerTransportFilter.java:142) ~[?:?]
        at org.elasticsearch.xpack.security.transport.SecurityServerTransportInterceptor$ProfileSecuredRequestHandler.messageReceived(SecurityServerTransportInterceptor.java:296) ~[?:?]
        at org.elasticsearch.transport.RequestHandlerRegistry.processMessageReceived(RequestHandlerRegistry.java:69) ~[elasticsearch-5.2.2.jar:5.2.2]
        at org.elasticsearch.transport.TcpTransport$RequestHandler.doRun(TcpTransport.java:1488) ~[elasticsearch-5.2.2.jar:5.2.2]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:596) ~[elasticsearch-5.2.2.jar:5.2.2]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-5.2.2.jar:5.2.2]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) ~[?:1.8.0_92-internal]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) ~[?:1.8.0_92-internal]
        ... 1 more
@jaymode
Copy link
Member

jaymode commented Mar 9, 2017

Adding tribe.cluster1.xpack.security.enabled: false should resolve this

@dforste
Copy link
Author

dforste commented Mar 9, 2017

@jaymode Thanks I will try that.

@dforste
Copy link
Author

dforste commented Mar 9, 2017

@jaymode That fixed things up thanks a bunch.

@dforste dforste closed this as completed Mar 9, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@jaymode @dforste