You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When I use trino(389 version) kafka connector, I find some problems.
For example:
SELECT _timestamp FROM kafka_table WHERE _timestamp <timestamp'2022-07-07 07:14:43.437'
Stack Trace:
'io.trino.spi.TrinoException: Cannot list splits for table ' ' reading topic ' '
at io.trino.plugin.kafka.KafkaSplitManager.getSplits(KafkaSplitManager.java:110)
at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorSplitManager.getSplits(ClassLoaderSafeConnectorSplitManager.java:51)
at io.trino.split.SplitManager.getSplits(SplitManager.java:58)
at io.trino.sql.planner.SplitSourceFactory$Visitor.visitScanAndFilter(SplitSourceFactory.java:177)
at io.trino.sql.planner.SplitSourceFactory$Visitor.visitFilter(SplitSourceFactory.java:246)
at io.trino.sql.planner.SplitSourceFactory$Visitor.visitFilter(SplitSourceFactory.java:125)
at io.trino.sql.planner.plan.FilterNode.accept(FilterNode.java:72)
at io.trino.sql.planner.SplitSourceFactory$Visitor.visitLimit(SplitSourceFactory.java:353)
at io.trino.sql.planner.SplitSourceFactory$Visitor.visitLimit(SplitSourceFactory.java:125)
at io.trino.sql.planner.plan.LimitNode.accept(LimitNode.java:124)
at io.trino.sql.planner.SplitSourceFactory.createSplitSources(SplitSourceFactory.java:105)
at io.trino.execution.scheduler.SqlQueryScheduler$PipelinedDistributedStagesScheduler.createStageScheduler(SqlQueryScheduler.java:1364)
at io.trino.execution.scheduler.SqlQueryScheduler$PipelinedDistributedStagesScheduler.create(SqlQueryScheduler.java:1244)
at io.trino.execution.scheduler.SqlQueryScheduler.createDistributedStagesScheduler(SqlQueryScheduler.java:371)
at io.trino.execution.scheduler.SqlQueryScheduler.start(SqlQueryScheduler.java:332)
at io.trino.execution.SqlQueryExecution.start(SqlQueryExecution.java:425)
at io.trino.execution.SqlQueryManager.createQuery(SqlQueryManager.java:243)
at io.trino.dispatcher.LocalDispatchQuery.lambda$startExecution$7(LocalDispatchQuery.java:143)
at io.trino.$gen.Trino_388_21_g844f91b_dirty____20220713_142511_2.run(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
I guess the problem is that when we use timestamp push down, the range may be (-1, timestamp).
And kafkaConsumer.offsetsForTimes cannot be negative.
So I think when we meet '-1' , we do not push down timestamp.
The text was updated successfully, but these errors were encountered:
harris233
changed the title
Fix small bug when we use trino to select kafka with timestamp
Fix bug when we use trino to select kafka with timestamp
Jul 15, 2022
harris233
changed the title
Fix bug when we use trino to select kafka with timestamp
Fix predicate failure for negative offset value on _timestamp column
Jul 26, 2022
When I use trino(389 version) kafka connector, I find some problems.
For example:
I guess the problem is that when we use timestamp push down, the range may be (-1, timestamp).
And kafkaConsumer.offsetsForTimes cannot be negative.
So I think when we meet '-1' , we do not push down timestamp.
The text was updated successfully, but these errors were encountered: