Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

java.lang.ClassCastException when upgrading to 2.0.0 #14706

Closed
krisb78 opened this issue Nov 12, 2015 · 13 comments
Closed

java.lang.ClassCastException when upgrading to 2.0.0 #14706

krisb78 opened this issue Nov 12, 2015 · 13 comments
Assignees
@martijnvg
Labels
>bug :Distributed/Distributed A catch all label for anything in the Distributed Area. If you aren't sure, use this one.

Comments

@krisb78
Copy link

krisb78 commented Nov 12, 2015

Trying to upgrade from 1.7.2 to 2.0.0, following the instructions provided on:

https://www.elastic.co/guide/en/elasticsearch/reference/2.0/restart-upgrade.html

Shut the whole cluster down, removed old plugins, installed the new cloud-aws plugin.

When starting the first master node with 2.0.0 installed, I'm getting:

java.lang.ClassCastException: org.elasticsearch.cluster.metadata.AliasOrIndex$Index cannot be cast to org.elasticsearch.cluster.metadata.AliasOrIndex$Alias

Full log:

root@cubitsearch-master-1:~# sudo -u elasticsearch /usr/bin/java -Xms3g -Xmx3g -Xmn200m -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/usr/share/elasticsearch -cp /usr/share/elasticsearch/lib/elasticsearch-2.0.0.jar:/usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch start -Des.default.path.home=/usr/share/elasticsearch -Des.default.path.logs=/var/log/elasticsearch -Des.default.path.data=/var/lib/elasticsearch -Des.default.path.work=/tmp/elasticsearch -Des.default.path.conf=/etc/elasticsearch -Des.max-open-files=true
[2015-11-12 11:49:58,131][INFO ][bootstrap                ] max_open_files [4096]
[2015-11-12 11:49:59,130][INFO ][node                     ] [cubitsearch-master-1] version[2.0.0], pid[29560], build[de54438/2015-10-22T08:09:48Z]
[2015-11-12 11:49:59,130][INFO ][node                     ] [cubitsearch-master-1] initializing ...
[2015-11-12 11:50:00,367][INFO ][plugins                  ] [cubitsearch-master-1] loaded [cloud-aws], sites []
[2015-11-12 11:50:00,514][INFO ][env                      ] [cubitsearch-master-1] using [1] data paths, mounts [[/ (/dev/xvda1)]], net usable_space [5.8gb], net total_space [7.8gb], spins? [no], types [ext4]
[2015-11-12 11:50:10,624][ERROR][gateway                  ] [cubitsearch-master-1] failed to read local state, exiting...
java.lang.ClassCastException: org.elasticsearch.cluster.metadata.AliasOrIndex$Index cannot be cast to org.elasticsearch.cluster.metadata.AliasOrIndex$Alias
    at org.elasticsearch.cluster.metadata.MetaData$Builder.build(MetaData.java:1008)
    at org.elasticsearch.gateway.MetaStateService.loadFullState(MetaStateService.java:104)
    at org.elasticsearch.gateway.GatewayMetaState.loadMetaState(GatewayMetaState.java:97)
    at org.elasticsearch.gateway.GatewayMetaState.pre20Upgrade(GatewayMetaState.java:223)
    at org.elasticsearch.gateway.GatewayMetaState.<init>(GatewayMetaState.java:85)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.elasticsearch.common.inject.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:56)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:86)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:865)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
    at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
    at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
    at org.elasticsearch.common.inject.SingleParameterInjector.inject(SingleParameterInjector.java:42)
    at org.elasticsearch.common.inject.SingleParameterInjector.getAll(SingleParameterInjector.java:66)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:865)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
    at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
    at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
    at org.elasticsearch.common.inject.SingleParameterInjector.inject(SingleParameterInjector.java:42)
    at org.elasticsearch.common.inject.SingleParameterInjector.getAll(SingleParameterInjector.java:66)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:865)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
    at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
    at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
    at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:201)
    at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:193)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:858)
    at org.elasticsearch.common.inject.InjectorBuilder.loadEagerSingletons(InjectorBuilder.java:193)
    at org.elasticsearch.common.inject.InjectorBuilder.injectDynamically(InjectorBuilder.java:175)
    at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:110)
    at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:93)
    at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:70)
    at org.elasticsearch.common.inject.ModulesBuilder.createInjector(ModulesBuilder.java:47)
    at org.elasticsearch.node.Node.<init>(Node.java:198)
    at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170)
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
[2015-11-12 11:50:12,770][ERROR][gateway                  ] [cubitsearch-master-1] failed to read local state, exiting...
java.lang.ClassCastException: org.elasticsearch.cluster.metadata.AliasOrIndex$Index cannot be cast to org.elasticsearch.cluster.metadata.AliasOrIndex$Alias
    at org.elasticsearch.cluster.metadata.MetaData$Builder.build(MetaData.java:1008)
    at org.elasticsearch.gateway.MetaStateService.loadFullState(MetaStateService.java:104)
    at org.elasticsearch.gateway.GatewayMetaState.loadMetaState(GatewayMetaState.java:97)
    at org.elasticsearch.gateway.GatewayMetaState.pre20Upgrade(GatewayMetaState.java:223)
    at org.elasticsearch.gateway.GatewayMetaState.<init>(GatewayMetaState.java:85)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.elasticsearch.common.inject.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:56)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:86)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:865)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
    at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
    at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
    at org.elasticsearch.common.inject.SingleParameterInjector.inject(SingleParameterInjector.java:42)
    at org.elasticsearch.common.inject.SingleParameterInjector.getAll(SingleParameterInjector.java:66)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:865)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
    at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
    at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
    at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:201)
    at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:193)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:858)
    at org.elasticsearch.common.inject.InjectorBuilder.loadEagerSingletons(InjectorBuilder.java:193)
    at org.elasticsearch.common.inject.InjectorBuilder.injectDynamically(InjectorBuilder.java:175)
    at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:110)
    at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:93)
    at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:70)
    at org.elasticsearch.common.inject.ModulesBuilder.createInjector(ModulesBuilder.java:47)
    at org.elasticsearch.node.Node.<init>(Node.java:198)
    at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170)
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
[2015-11-12 11:50:14,300][ERROR][gateway                  ] [cubitsearch-master-1] failed to read local state, exiting...
java.lang.ClassCastException: org.elasticsearch.cluster.metadata.AliasOrIndex$Index cannot be cast to org.elasticsearch.cluster.metadata.AliasOrIndex$Alias
    at org.elasticsearch.cluster.metadata.MetaData$Builder.build(MetaData.java:1008)
    at org.elasticsearch.gateway.MetaStateService.loadFullState(MetaStateService.java:104)
    at org.elasticsearch.gateway.GatewayMetaState.loadMetaState(GatewayMetaState.java:97)
    at org.elasticsearch.gateway.GatewayMetaState.pre20Upgrade(GatewayMetaState.java:223)
    at org.elasticsearch.gateway.GatewayMetaState.<init>(GatewayMetaState.java:85)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.elasticsearch.common.inject.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:56)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:86)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:865)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
    at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
    at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
    at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:201)
    at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:193)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:858)
    at org.elasticsearch.common.inject.InjectorBuilder.loadEagerSingletons(InjectorBuilder.java:193)
    at org.elasticsearch.common.inject.InjectorBuilder.injectDynamically(InjectorBuilder.java:175)
    at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:110)
    at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:93)
    at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:70)
    at org.elasticsearch.common.inject.ModulesBuilder.createInjector(ModulesBuilder.java:47)
    at org.elasticsearch.node.Node.<init>(Node.java:198)
    at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170)
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Exception in thread "main" java.lang.ClassCastException: org.elasticsearch.cluster.metadata.AliasOrIndex$Index cannot be cast to org.elasticsearch.cluster.metadata.AliasOrIndex$Alias
    at org.elasticsearch.cluster.metadata.MetaData$Builder.build(MetaData.java:1008)
    at org.elasticsearch.gateway.MetaStateService.loadFullState(MetaStateService.java:104)
    at org.elasticsearch.gateway.GatewayMetaState.loadMetaState(GatewayMetaState.java:97)
    at org.elasticsearch.gateway.GatewayMetaState.pre20Upgrade(GatewayMetaState.java:223)
    at org.elasticsearch.gateway.GatewayMetaState.<init>(GatewayMetaState.java:85)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at <<<guice>>>
    at org.elasticsearch.node.Node.<init>(Node.java:198)
    at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170)
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
@clintongormley
Copy link

Hi @krisb78

Could you try it without the cloud aws plugin and see if you have the same problem? Also, could you upload the output of GET /_cluster/state (which you'll need to get from the 1.7.2 cluster)

thanks

@krisb78
Copy link
Author

krisb78 commented Nov 17, 2015

Hi @clintongormley ,

Here's my cluster state, I'll try to start the node without cloud-aws and let you know shortly.

cluster_state.txt

@dominicbartl
Copy link

I just ran into the same issue.
cluster-state.json.txt

@krisb78
Copy link
Author

krisb78 commented Nov 17, 2015

Tried without cloud-aws, but got the same error:

root@cubitsearch-master-1:/usr/share/elasticsearch# sudo -u elasticsearch /usr/bin/java -Xms3g -Xmx3g -Xmn200m -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/usr/share/elasticsearch -cp /usr/share/elasticsearch/lib/elasticsearch-2.0.0.jar:/usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch start -Des.default.path.home=/usr/share/elasticsearch -Des.default.path.logs=/var/log/elasticsearch -Des.default.path.data=/var/lib/elasticsearch -Des.default.path.work=/tmp/elasticsearch -Des.default.path.conf=/etc/elasticsearch -Des.max-open-files=true
[2015-11-17 14:22:36,284][INFO ][bootstrap                ] max_open_files [4096]
[2015-11-17 14:22:37,329][INFO ][node                     ] [cubitsearch-master-1] version[2.0.0], pid[7642], build[de54438/2015-10-22T08:09:48Z]
[2015-11-17 14:22:37,329][INFO ][node                     ] [cubitsearch-master-1] initializing ...
[2015-11-17 14:22:37,664][INFO ][plugins                  ] [cubitsearch-master-1] loaded [], sites []
[2015-11-17 14:22:38,048][INFO ][env                      ] [cubitsearch-master-1] using [1] data paths, mounts [[/ (/dev/xvda1)]], net usable_space [5.8gb], net total_space [7.8gb], spins? [no], types [ext4]
[2015-11-17 14:22:46,488][ERROR][gateway                  ] [cubitsearch-master-1] failed to read local state, exiting...
java.lang.ClassCastException: org.elasticsearch.cluster.metadata.AliasOrIndex$Index cannot be cast to org.elasticsearch.cluster.metadata.AliasOrIndex$Alias
    at org.elasticsearch.cluster.metadata.MetaData$Builder.build(MetaData.java:1008)
    at org.elasticsearch.gateway.MetaStateService.loadFullState(MetaStateService.java:104)
    at org.elasticsearch.gateway.GatewayMetaState.loadMetaState(GatewayMetaState.java:97)
    at org.elasticsearch.gateway.GatewayMetaState.pre20Upgrade(GatewayMetaState.java:223)
    at org.elasticsearch.gateway.GatewayMetaState.<init>(GatewayMetaState.java:85)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.elasticsearch.common.inject.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:56)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:86)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:865)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
    at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
    at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
    at org.elasticsearch.common.inject.SingleParameterInjector.inject(SingleParameterInjector.java:42)
    at org.elasticsearch.common.inject.SingleParameterInjector.getAll(SingleParameterInjector.java:66)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:865)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
    at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
    at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
    at org.elasticsearch.common.inject.SingleParameterInjector.inject(SingleParameterInjector.java:42)
    at org.elasticsearch.common.inject.SingleParameterInjector.getAll(SingleParameterInjector.java:66)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:865)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
    at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
    at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
    at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:201)
    at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:193)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:858)
    at org.elasticsearch.common.inject.InjectorBuilder.loadEagerSingletons(InjectorBuilder.java:193)
    at org.elasticsearch.common.inject.InjectorBuilder.injectDynamically(InjectorBuilder.java:175)
    at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:110)
    at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:93)
    at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:70)
    at org.elasticsearch.common.inject.ModulesBuilder.createInjector(ModulesBuilder.java:47)
    at org.elasticsearch.node.Node.<init>(Node.java:198)
    at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170)
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
[2015-11-17 14:22:48,683][ERROR][gateway                  ] [cubitsearch-master-1] failed to read local state, exiting...
java.lang.ClassCastException: org.elasticsearch.cluster.metadata.AliasOrIndex$Index cannot be cast to org.elasticsearch.cluster.metadata.AliasOrIndex$Alias
    at org.elasticsearch.cluster.metadata.MetaData$Builder.build(MetaData.java:1008)
    at org.elasticsearch.gateway.MetaStateService.loadFullState(MetaStateService.java:104)
    at org.elasticsearch.gateway.GatewayMetaState.loadMetaState(GatewayMetaState.java:97)
    at org.elasticsearch.gateway.GatewayMetaState.pre20Upgrade(GatewayMetaState.java:223)
    at org.elasticsearch.gateway.GatewayMetaState.<init>(GatewayMetaState.java:85)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.elasticsearch.common.inject.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:56)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:86)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:865)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
    at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
    at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
    at org.elasticsearch.common.inject.SingleParameterInjector.inject(SingleParameterInjector.java:42)
    at org.elasticsearch.common.inject.SingleParameterInjector.getAll(SingleParameterInjector.java:66)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:865)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
    at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
    at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
    at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:201)
    at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:193)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:858)
    at org.elasticsearch.common.inject.InjectorBuilder.loadEagerSingletons(InjectorBuilder.java:193)
    at org.elasticsearch.common.inject.InjectorBuilder.injectDynamically(InjectorBuilder.java:175)
    at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:110)
    at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:93)
    at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:70)
    at org.elasticsearch.common.inject.ModulesBuilder.createInjector(ModulesBuilder.java:47)
    at org.elasticsearch.node.Node.<init>(Node.java:198)
    at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170)
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
[2015-11-17 14:22:50,248][ERROR][gateway                  ] [cubitsearch-master-1] failed to read local state, exiting...
java.lang.ClassCastException: org.elasticsearch.cluster.metadata.AliasOrIndex$Index cannot be cast to org.elasticsearch.cluster.metadata.AliasOrIndex$Alias
    at org.elasticsearch.cluster.metadata.MetaData$Builder.build(MetaData.java:1008)
    at org.elasticsearch.gateway.MetaStateService.loadFullState(MetaStateService.java:104)
    at org.elasticsearch.gateway.GatewayMetaState.loadMetaState(GatewayMetaState.java:97)
    at org.elasticsearch.gateway.GatewayMetaState.pre20Upgrade(GatewayMetaState.java:223)
    at org.elasticsearch.gateway.GatewayMetaState.<init>(GatewayMetaState.java:85)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at org.elasticsearch.common.inject.DefaultConstructionProxyFactory$1.newInstance(DefaultConstructionProxyFactory.java:56)
    at org.elasticsearch.common.inject.ConstructorInjector.construct(ConstructorInjector.java:86)
    at org.elasticsearch.common.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:104)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter$1.call(ProviderToInternalFactoryAdapter.java:47)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:865)
    at org.elasticsearch.common.inject.ProviderToInternalFactoryAdapter.get(ProviderToInternalFactoryAdapter.java:43)
    at org.elasticsearch.common.inject.Scopes$1$1.get(Scopes.java:59)
    at org.elasticsearch.common.inject.InternalFactoryToProviderAdapter.get(InternalFactoryToProviderAdapter.java:46)
    at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:201)
    at org.elasticsearch.common.inject.InjectorBuilder$1.call(InjectorBuilder.java:193)
    at org.elasticsearch.common.inject.InjectorImpl.callInContext(InjectorImpl.java:858)
    at org.elasticsearch.common.inject.InjectorBuilder.loadEagerSingletons(InjectorBuilder.java:193)
    at org.elasticsearch.common.inject.InjectorBuilder.injectDynamically(InjectorBuilder.java:175)
    at org.elasticsearch.common.inject.InjectorBuilder.build(InjectorBuilder.java:110)
    at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:93)
    at org.elasticsearch.common.inject.Guice.createInjector(Guice.java:70)
    at org.elasticsearch.common.inject.ModulesBuilder.createInjector(ModulesBuilder.java:47)
    at org.elasticsearch.node.Node.<init>(Node.java:198)
    at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170)
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)
Exception in thread "main" java.lang.ClassCastException: org.elasticsearch.cluster.metadata.AliasOrIndex$Index cannot be cast to org.elasticsearch.cluster.metadata.AliasOrIndex$Alias
    at org.elasticsearch.cluster.metadata.MetaData$Builder.build(MetaData.java:1008)
    at org.elasticsearch.gateway.MetaStateService.loadFullState(MetaStateService.java:104)
    at org.elasticsearch.gateway.GatewayMetaState.loadMetaState(GatewayMetaState.java:97)
    at org.elasticsearch.gateway.GatewayMetaState.pre20Upgrade(GatewayMetaState.java:223)
    at org.elasticsearch.gateway.GatewayMetaState.<init>(GatewayMetaState.java:85)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
    at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
    at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
    at <<<guice>>>
    at org.elasticsearch.node.Node.<init>(Node.java:198)
    at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:145)
    at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:170)
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:270)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:35)

@martijnvg
Copy link
Member

@krisb78 Looking at your cluster stats I found that there is an index named hotcoffee_v1 and also an alias named hotcoffee_v1. Alias and index names need to be unique in a cluster, so this causes this unexpected failure. I wonder how this could have happened... did you by any chance do a restore with a rename?

@krisb78
Copy link
Author

krisb78 commented Nov 18, 2015

This indeed is the case - I first created a template with the pattern set to 'hotcoffee_v1*', then started indexing into the 'hotcoffee_v1' index - the cluster (1.7.2) didn't complain.

So I have:

[LIVE] krisba@cubitsearch-client-2:~$ curl -XGET localhost:9200/_template/hotcoffee_v1?pretty
{
  "hotcoffee_v1" : {
    "order" : 0,
    "template" : "hotcoffee_v1*",
    "settings" : {
      "index.merge.policy.merge_factor" : "30",
      "index.refresh_interval" : "-1"
    },
    "mappings" : {
      "hotcoffee" : {
        "dynamic_templates" : [ {
          "store_generic" : {
            "mapping" : {
              "index" : "not_analyzed",
              "store" : true,
              "doc_values" : true
            },
            "match" : "*"
          }
        } ],
        "_ttl" : {
          "enabled" : true,
          "default" : "90d"
        },
        "properties" : {
          "pf_timestamp" : {
            "store" : "true",
            "doc_values" : true,
            "type" : "date"
          },
          "pf_prev_timestamp" : {
            "store" : "true",
            "doc_values" : true,
            "type" : "date"
          }
        }
      }
    },
    "aliases" : {
      "hotcoffee_v1" : { }
    }
  }
}

and then

[LIVE] krisba@cubitsearch-client-2:~$ curl -XGET localhost:9200/hotcoffee_v1?pretty
{
  "hotcoffee_v1" : {
    "aliases" : {
      "hotcoffee_v1" : { }
    },
    "mappings" : {
      "hotcoffee" : {
        "dynamic_templates" : [ {
          "store_generic" : {
            "mapping" : {
              "index" : "not_analyzed",
              "store" : true,
              "doc_values" : true
            },
            "match" : "*"
          }
        } ],
        "_ttl" : {
          "enabled" : true,
          "default" : 7776000000
        },
        "properties" : {
          "ach_name" : {
            "type" : "string",
            "index" : "not_analyzed",
            "store" : true,
            "doc_values" : true
          },
          "app_name" : {
            "type" : "string",
            "index" : "not_analyzed",
            "store" : true,
            "doc_values" : true
          },
          "appid" : {
            "type" : "long",
            "store" : true,
            "doc_values" : true
          },
          "pf_network" : {
            "type" : "string",
            "index" : "not_analyzed",
            "store" : true,
            "doc_values" : true
          },
          "pf_playtime_delta" : {
            "type" : "long",
            "store" : true,
            "doc_values" : true
          },
          "pf_prev_timestamp" : {
            "type" : "date",
            "store" : true,
            "doc_values" : true,
            "format" : "dateOptionalTime"
          },
          "pf_steamid" : {
            "type" : "string",
            "index" : "not_analyzed",
            "store" : true,
            "doc_values" : true
          },
          "pf_timestamp" : {
            "type" : "date",
            "store" : true,
            "doc_values" : true,
            "format" : "dateOptionalTime"
          },
          "pf_type" : {
            "type" : "string",
            "index" : "not_analyzed",
            "store" : true,
            "doc_values" : true
          },
          "playtime_2weeks" : {
            "type" : "long",
            "store" : true,
            "doc_values" : true
          },
          "playtime_forever" : {
            "type" : "long",
            "store" : true,
            "doc_values" : true
          },
          "timestamp_delta_minutes" : {
            "type" : "long",
            "store" : true,
            "doc_values" : true
          }
        }
      }
    },
    "settings" : {
      "index" : {
        "creation_date" : "1433238565453",
        "uuid" : "tE5bQrzkT5-Hei6TMp7KVQ",
        "number_of_replicas" : "1",
        "number_of_shards" : "5",
        "refresh_interval" : "-1",
        "version" : {
          "created" : "1050299"
        },
        "merge" : {
          "policy" : {
            "merge_factor" : "30"
          }
        }
      }
    },
    "warmers" : { }
  }
}

I think I have one more index/template set up like that, it never caused me a problem in 1.7.2...

Has it changed in 2.0.0 ?

@martijnvg
Copy link
Member

No (except than the error that you're seeing that expects index and alias names to be unique). So this error needs to be fixed and be able to handle alias and index with the same name.

I do think that adding an alias with the same name as the index via index templates should not be allowed.

@krisb78
Copy link
Author

krisb78 commented Nov 18, 2015

Hm, I see your point...

I'll check if renaming these aliases fixes the issue.

@martijnvg martijnvg mentioned this issue Nov 18, 2015
Merged
@clintongormley
Copy link

I do think that adding an alias with the same name as the index via index templates should not be allowed.

agreed! We should

@clintongormley
Copy link

It looks like trying to create an index and alias with the same name in 2.0 throws the same exception, so this should become clearer with the better exception proposed in #14842

@clintongormley
Copy link

It looks like it is impossible to remove an alias with the same name as an index in 1.x:

InvalidAliasNameException[[foo] Invalid alias name [foo], an index exists with the same name as the alias]

So I think the only solution is to reindex into a new index, and to delete the old index

@krisb78
Copy link
Author

krisb78 commented Nov 19, 2015

Ok, I think I will be able to get out of this by making sure I don't create new indices with the same names as the aliases defined in my templates.

Thanks a lot for looking into this.

@martijnvg
Copy link
Member

Fixed via #14842

@clintongormley clintongormley added :Distributed/Distributed A catch all label for anything in the Distributed Area. If you aren't sure, use this one. and removed :Cluster labels Feb 13, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@martijnvg martijnvg

Labels
>bug :Distributed/Distributed A catch all label for anything in the Distributed Area. If you aren't sure, use this one.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@clintongormley @krisb78 @dominicbartl @martijnvg