Preconfigured keyspace support

[deejay1]

In our installation we have preconfigured keyspaces with access to operations only inside that keyspace.
Unfortunately it seems that the datastax backend doesn't support such case. It would be great if it could detect that the keyspace is present and create only the tables.

Unfortunately "CREATE KEYSPACE IF NOT EXISTS" doesn't work, as it seems that the create keyspace permission is more important.

Exception:

16:01:29.844 [cluster1-nio-worker-5] INFO  com.spotify.heroic.metric.datastax.schema.ng.NextGenSchema - Creating keyspace heroic
16:01:29.863 [main] ERROR com.spotify.heroic.HeroicService - Failed to start Heroic instance
java.util.concurrent.ExecutionException: eu.toolchain.async.TransformException: error in transform
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture$Sync.get(ConcurrentResolvableFuture.java:527)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.get(ConcurrentResolvableFuture.java:304)
    at com.spotify.heroic.HeroicService.main(HeroicService.java:106)
    at com.spotify.heroic.HeroicService.main(HeroicService.java:59)
Caused by: eu.toolchain.async.TransformException: error in transform
    at eu.toolchain.async.helper.ResolvedTransformHelper.resolved(ResolvedTransformHelper.java:26)
    at eu.toolchain.async.DirectAsyncCaller.resolve(DirectAsyncCaller.java:10)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture$2.run(ConcurrentResolvableFuture.java:221)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.run(ConcurrentResolvableFuture.java:439)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.resolve(ConcurrentResolvableFuture.java:97)
    at com.spotify.heroic.HeroicCore$Instance.start(HeroicCore.java:866)
    ... 2 more
Caused by: java.util.concurrent.ExecutionException: java.lang.Exception: 1 exception(s) caught: User heroic_adm has no CREATE permission on <all keyspaces> or any of its parents
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture$Sync.get(ConcurrentResolvableFuture.java:542)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.get(ConcurrentResolvableFuture.java:316)
    at com.spotify.heroic.HeroicCore.awaitLifeCycles(HeroicCore.java:512)
    at com.spotify.heroic.HeroicCore.startLifeCycles(HeroicCore.java:461)
    at com.spotify.heroic.HeroicCore$Instance.lambda$new$128(HeroicCore.java:817)
    at com.spotify.heroic.HeroicCore$Instance$$Lambda$112/1234586997.transform(Unknown Source)
    at eu.toolchain.async.helper.ResolvedTransformHelper.resolved(ResolvedTransformHelper.java:24)
    ... 7 more
Caused by: java.lang.Exception: 1 exception(s) caught: User heroic_adm has no CREATE permission on <all keyspaces> or any of its parents
    at eu.toolchain.async.TinyThrowableUtils.buildCollectedException(TinyThrowableUtils.java:23)
    at eu.toolchain.async.helper.CollectHelper.done(CollectHelper.java:142)
    at eu.toolchain.async.helper.CollectHelper.add(CollectHelper.java:128)
    at eu.toolchain.async.helper.CollectHelper.cancelled(CollectHelper.java:85)
    at eu.toolchain.async.DirectAsyncCaller.cancel(DirectAsyncCaller.java:28)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture$2.run(ConcurrentResolvableFuture.java:217)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.run(ConcurrentResolvableFuture.java:439)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.cancel(ConcurrentResolvableFuture.java:115)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.cancel(ConcurrentResolvableFuture.java:121)
    at eu.toolchain.async.helper.CollectHelper.checkFailed(CollectHelper.java:94)
    at eu.toolchain.async.helper.CollectHelper.failed(CollectHelper.java:80)
    at eu.toolchain.async.DirectAsyncCaller.fail(DirectAsyncCaller.java:19)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture$2.run(ConcurrentResolvableFuture.java:212)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.run(ConcurrentResolvableFuture.java:439)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.fail(ConcurrentResolvableFuture.java:106)
    at eu.toolchain.async.helper.ResolvedTransformHelper.failed(ResolvedTransformHelper.java:16)
    at eu.toolchain.async.DirectAsyncCaller.fail(DirectAsyncCaller.java:19)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture$2.run(ConcurrentResolvableFuture.java:212)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.run(ConcurrentResolvableFuture.java:439)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.fail(ConcurrentResolvableFuture.java:106)
    at eu.toolchain.async.helper.ResolvedLazyTransformHelper.failed(ResolvedLazyTransformHelper.java:16)
    at eu.toolchain.async.DirectAsyncCaller.fail(DirectAsyncCaller.java:19)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture$2.run(ConcurrentResolvableFuture.java:212)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.run(ConcurrentResolvableFuture.java:439)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.fail(ConcurrentResolvableFuture.java:106)
    at eu.toolchain.async.helper.ResolvedLazyTransformHelper$1.failed(ResolvedLazyTransformHelper.java:33)
    at eu.toolchain.async.DirectAsyncCaller.fail(DirectAsyncCaller.java:19)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture$2.run(ConcurrentResolvableFuture.java:212)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.run(ConcurrentResolvableFuture.java:439)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.fail(ConcurrentResolvableFuture.java:106)
    at eu.toolchain.async.helper.ResolvedTransformHelper.failed(ResolvedTransformHelper.java:16)
    at eu.toolchain.async.DirectAsyncCaller.fail(DirectAsyncCaller.java:19)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture$2.run(ConcurrentResolvableFuture.java:212)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.run(ConcurrentResolvableFuture.java:439)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.fail(ConcurrentResolvableFuture.java:106)
    at eu.toolchain.async.helper.ResolvedLazyTransformHelper$1.failed(ResolvedLazyTransformHelper.java:33)
    at eu.toolchain.async.DirectAsyncCaller.fail(DirectAsyncCaller.java:19)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture$2.run(ConcurrentResolvableFuture.java:212)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.run(ConcurrentResolvableFuture.java:439)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.fail(ConcurrentResolvableFuture.java:106)
    at eu.toolchain.async.helper.ResolvedLazyTransformHelper.failed(ResolvedLazyTransformHelper.java:16)
    at eu.toolchain.async.DirectAsyncCaller.fail(DirectAsyncCaller.java:19)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture$2.run(ConcurrentResolvableFuture.java:212)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.run(ConcurrentResolvableFuture.java:439)
    at eu.toolchain.async.concurrent.ConcurrentResolvableFuture.fail(ConcurrentResolvableFuture.java:106)
    at com.spotify.heroic.metric.datastax.Async$1.onFailure(Async.java:46)
    at com.google.common.util.concurrent.Futures$6.run(Futures.java:1764)
    at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:456)
    at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:817)
    at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:753)
    at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:634)
    at com.datastax.driver.core.DefaultResultSetFuture.onSet(DefaultResultSetFuture.java:149)
    at com.datastax.driver.core.RequestHandler.setFinalResult(RequestHandler.java:183)
    at com.datastax.driver.core.RequestHandler.access$2300(RequestHandler.java:44)
    at com.datastax.driver.core.RequestHandler$SpeculativeExecution.setFinalResult(RequestHandler.java:751)
    at com.datastax.driver.core.RequestHandler$SpeculativeExecution.onSet(RequestHandler.java:573)
    at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:1009)
    at com.datastax.driver.core.Connection$Dispatcher.channelRead0(Connection.java:932)
    at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
    at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:254)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
    at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
    at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
    at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
    at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
    at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
    at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:618)
    at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:329)
    at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:250)
    at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
    at java.lang.Thread.run(Thread.java:745)
    Suppressed: com.datastax.driver.core.exceptions.UnauthorizedException: User heroic_adm has no CREATE permission on <all keyspaces> or any of its parents
        at com.datastax.driver.core.Responses$Error.asException(Responses.java:101)
    ... 25 more

[udoprog]

For now you can take the tables CQL and run it manually:
https://github.com/spotify/heroic/blob/master/metric/datastax/src/main/resources/com.spotify.heroic.metric.datastax.schema.ng/tables.cql

I'll add a task that specifies either to skip the creation of a keyspace, or somehow detect if it exists.

[deejay1]

@udoprog yes, I did that :) I just reported that to point out the potential issue especially when the cassandra clusters are managed by an outside entity/team.

[udoprog]

All right. Thanks for the report and good luck!