Table creation throws io.confluent.ksql.exception.KafkaResponseGetFailedException: Failed to guarantee existence of topic

[alexhafner]

On creating tables in the CLI, we ran into

io.confluent.ksql.exception.KafkaResponseGetFailedException: Failed to guarantee existence of topic PAGEVIEWS_AVG_1

when creating a table like so:

CREATE TABLE pageviews_avg_1b \
    AS SELECT \
        userid, \
      SUM(viewtime) AS sum_viewtime, \
      MIN(viewtime) AS min_viewtime, \
      MAX(viewtime) AS max_viewtime, \
      COUNT(*) AS view_count \
    FROM pageviews \
    WINDOW SESSION (60 SECONDS) GROUP BY userid;

If we re-issued that statement, the table would create fine and SELECT worked.

This could be reproduced within the session (CREATE TABLE pageviews_avg_1c etc) , and also happened on another CLI session by another user. Exiting the CLI and starting it again healed the issue, and the tables then created straight away.

After a while, the issue started happening again in the session where it just worked

The CLI log showed

[2017-09-22 14:35:34,795] ERROR io.confluent.ksql.exception.KafkaResponseGetFailedException: Failed to guarantee existence of topic PAGEVIEWS_AVG_1A
	at io.confluent.ksql.util.KafkaTopicClientImpl.createTopic(KafkaTopicClientImpl.java:63)
	at io.confluent.ksql.structured.SchemaKStream.createSinkTopic(SchemaKStream.java:336)
	at io.confluent.ksql.structured.SchemaKTable.into(SchemaKTable.java:70)
	at io.confluent.ksql.structured.SchemaKTable.into(SchemaKTable.java:51)
	at io.confluent.ksql.physical.PhysicalPlanBuilder.buildOutput(PhysicalPlanBuilder.java:208)
	at io.confluent.ksql.physical.PhysicalPlanBuilder.kafkaStreamsDsl(PhysicalPlanBuilder.java:127)
	at io.confluent.ksql.physical.PhysicalPlanBuilder.kafkaStreamsDsl(PhysicalPlanBuilder.java:104)
	at io.confluent.ksql.physical.PhysicalPlanBuilder.buildPhysicalPlan(PhysicalPlanBuilder.java:100)
	at io.confluent.ksql.QueryEngine.buildQueryPhysicalPlan(QueryEngine.java:221)
	at io.confluent.ksql.QueryEngine.buildPhysicalPlans(QueryEngine.java:199)
	at io.confluent.ksql.KsqlEngine.planQueries(KsqlEngine.java:139)
	at io.confluent.ksql.KsqlEngine.buildMultipleQueries(KsqlEngine.java:125)
	at io.confluent.ksql.rest.server.computation.StatementExecutor.startQuery(StatementExecutor.java:331)
	at io.confluent.ksql.rest.server.computation.StatementExecutor.executeStatement(StatementExecutor.java:274)
	at io.confluent.ksql.rest.server.computation.StatementExecutor.handleStatementWithTerminatedQueries(StatementExecutor.java:215)
	at io.confluent.ksql.rest.server.computation.StatementExecutor.handleStatement(StatementExecutor.java:114)
	at io.confluent.ksql.rest.server.computation.CommandRunner.executeStatement(CommandRunner.java:109)
	at io.confluent.ksql.rest.server.computation.CommandRunner.run(CommandRunner.java:75)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TopicExistsException: Topic 'PAGEVIEWS_AVG_1A' already exists.
	at org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
	at org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
	at org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
	at org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:213)
	at io.confluent.ksql.util.KafkaTopicClientImpl.createTopic(KafkaTopicClientImpl.java:61)
	... 18 more
Caused by: org.apache.kafka.common.errors.TopicExistsException: Topic 'PAGEVIEWS_AVG_1A' already exists.
 (io.confluent.ksql.rest.server.computation.StatementExecutor:220)

[rajnishgarg]

@alexhafner did you able to figure out this?

[apurvam]

This is fixed by #779 and #788

[rajnishgarg]

Thanks @apurvam, When there will be ksql release with these commits?

[apurvam]

Yes, the KSQL GA release targeted for later this month will have these changes.

[rajnishgarg]

@apurvam Could you please provide, when you guys have plan for the release target (Last one is 1 feb) and looks like after that there are many fixes ?

[apurvam]

Hi @rajnishgarg . Sorry for the delay. The target for the Confluent 4.1 release was April 2, but was pushed back to April 16.

Right now, it is likely that the ksql binaries with the fixes will be released on that day.