Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression: Queries fail with INTERNAL ERROR since 424 #18865

Closed
guyco33 opened this issue Aug 30, 2023 · 1 comment · Fixed by #18887
Closed

Regression: Queries fail with INTERNAL ERROR since 424 #18865

guyco33 opened this issue Aug 30, 2023 · 1 comment · Fixed by #18887
Labels
bug Something isn't working RELEASE-BLOCKER

Comments

@guyco33
Copy link
Member

guyco33 commented Aug 30, 2023

Since 424 the following query on hive partitioned table fails:

CREATE TABLE hive.temp.test_part
WITH (partitioned_by = ARRAY['part_col'])
AS SELECT * FROM (VALUES ('some text', 'part1')) AS t(comment, part_col);

SELECT part_col
FROM hive.temp.test_part
WHERE part_col = 'part1'
AND regexp_replace(comment, '([a]+)([b]+)', x ->x[1]) LIKE '%'

stack trace:

java.lang.IllegalArgumentException: Missing column assignment for expr
	at com.google.common.base.Preconditions.checkArgument(Preconditions.java:218)
	at io.trino.sql.planner.LookupSymbolResolver.getValue(LookupSymbolResolver.java:44)
	at io.trino.sql.planner.ExpressionInterpreter$Visitor.visitSymbolReference(ExpressionInterpreter.java:397)
	at io.trino.sql.tree.SymbolReference.accept(SymbolReference.java:41)
	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
	at io.trino.sql.planner.ExpressionInterpreter$Visitor.processWithExceptionHandling(ExpressionInterpreter.java:316)
	at io.trino.sql.planner.ExpressionInterpreter$Visitor.visitSubscriptExpression(ExpressionInterpreter.java:1509)
	at io.trino.sql.tree.SubscriptExpression.accept(SubscriptExpression.java:50)
	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
	at io.trino.sql.planner.ExpressionInterpreter$Visitor.processWithExceptionHandling(ExpressionInterpreter.java:316)
	at io.trino.sql.planner.ExpressionInterpreter$Visitor.visitLambdaExpression(ExpressionInterpreter.java:1069)
	at io.trino.sql.tree.LambdaExpression.accept(LambdaExpression.java:60)
	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
	at io.trino.sql.planner.ExpressionInterpreter$Visitor.processWithExceptionHandling(ExpressionInterpreter.java:316)
	at io.trino.sql.planner.ExpressionInterpreter$Visitor.visitFunctionCall(ExpressionInterpreter.java:1031)
	at io.trino.sql.tree.FunctionCall.accept(FunctionCall.java:121)
	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
	at io.trino.sql.planner.ExpressionInterpreter$Visitor.processWithExceptionHandling(ExpressionInterpreter.java:316)
	at io.trino.sql.planner.ExpressionInterpreter$Visitor.visitFunctionCall(ExpressionInterpreter.java:1031)
	at io.trino.sql.tree.FunctionCall.accept(FunctionCall.java:121)
	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
	at io.trino.sql.planner.ExpressionInterpreter$Visitor.processWithExceptionHandling(ExpressionInterpreter.java:316)
	at io.trino.sql.planner.ExpressionInterpreter$Visitor.visitLogicalExpression(ExpressionInterpreter.java:973)
	at io.trino.sql.tree.LogicalExpression.accept(LogicalExpression.java:80)
	at io.trino.sql.tree.AstVisitor.process(AstVisitor.java:27)
	at io.trino.sql.planner.ExpressionInterpreter$Visitor.processWithExceptionHandling(ExpressionInterpreter.java:316)
	at io.trino.sql.planner.ExpressionInterpreter.optimize(ExpressionInterpreter.java:296)
	at io.trino.sql.planner.LayoutConstraintEvaluator.lambda$isCandidate$0(LayoutConstraintEvaluator.java:61)
	at io.trino.operator.scalar.TryFunction.evaluate(TryFunction.java:132)
	at io.trino.sql.planner.LayoutConstraintEvaluator.isCandidate(LayoutConstraintEvaluator.java:61)
	at io.trino.plugin.hive.HivePartitionManager.partitionMatches(HivePartitionManager.java:221)
	at io.trino.plugin.hive.HivePartitionManager.lambda$getPartitions$1(HivePartitionManager.java:105)
	at java.base/java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:178)
	at java.base/java.util.Collections$2.tryAdvance(Collections.java:4853)
	at java.base/java.util.Collections$2.forEachRemaining(Collections.java:4861)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:921)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
	at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:682)
	at io.trino.plugin.hive.HivePartitionManager.getPartitions(HivePartitionManager.java:106)
	at io.trino.plugin.hive.HiveMetadata.applyFilter(HiveMetadata.java:2941)
	at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.applyFilter(ClassLoaderSafeConnectorMetadata.java:913)
	at io.trino.tracing.TracingConnectorMetadata.applyFilter(TracingConnectorMetadata.java:1073)
	at io.trino.metadata.MetadataManager.applyFilter(MetadataManager.java:1908)
	at io.trino.tracing.TracingMetadata.applyFilter(TracingMetadata.java:905)
	at io.trino.sql.planner.iterative.rule.PushPredicateIntoTableScan.pushFilterIntoTableScan(PushPredicateIntoTableScan.java:247)
	at io.trino.sql.planner.iterative.rule.PushPredicateIntoTableScan.apply(PushPredicateIntoTableScan.java:120)
	at io.trino.sql.planner.iterative.rule.PushPredicateIntoTableScan.apply(PushPredicateIntoTableScan.java:83)
	at io.trino.sql.planner.iterative.IterativeOptimizer.transform(IterativeOptimizer.java:209)
	at io.trino.sql.planner.iterative.IterativeOptimizer.exploreNode(IterativeOptimizer.java:176)
	at io.trino.sql.planner.iterative.IterativeOptimizer.exploreGroup(IterativeOptimizer.java:139)
	at io.trino.sql.planner.iterative.IterativeOptimizer.exploreChildren(IterativeOptimizer.java:259)
	at io.trino.sql.planner.iterative.IterativeOptimizer.exploreGroup(IterativeOptimizer.java:141)
	at io.trino.sql.planner.iterative.IterativeOptimizer.exploreChildren(IterativeOptimizer.java:259)
	at io.trino.sql.planner.iterative.IterativeOptimizer.exploreGroup(IterativeOptimizer.java:141)
	at io.trino.sql.planner.iterative.IterativeOptimizer.optimize(IterativeOptimizer.java:123)
	at io.trino.sql.planner.LogicalPlanner.runOptimizer(LogicalPlanner.java:299)
	at io.trino.sql.planner.LogicalPlanner.plan(LogicalPlanner.java:269)
	at io.trino.sql.planner.LogicalPlanner.plan(LogicalPlanner.java:238)
	at io.trino.sql.planner.LogicalPlanner.plan(LogicalPlanner.java:233)
	at io.trino.execution.SqlQueryExecution.doPlanQuery(SqlQueryExecution.java:482)
	at io.trino.execution.SqlQueryExecution.planQuery(SqlQueryExecution.java:462)
	at io.trino.execution.SqlQueryExecution.start(SqlQueryExecution.java:400)
	at io.trino.execution.SqlQueryManager.createQuery(SqlQueryManager.java:256)
	at io.trino.dispatcher.LocalDispatchQuery.startExecution(LocalDispatchQuery.java:145)
	at io.trino.dispatcher.LocalDispatchQuery.lambda$waitForMinimumWorkers$2(LocalDispatchQuery.java:129)
	at io.airlift.concurrent.MoreFutures.lambda$addSuccessCallback$12(MoreFutures.java:568)
	at io.airlift.concurrent.MoreFutures$3.onSuccess(MoreFutures.java:543)
	at com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1133)
	at io.trino.$gen.Trino_424____20230830_094727_2.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

No failure when part_col is not selected

@guyco33 guyco33 added bug Something isn't working RELEASE-BLOCKER labels Aug 30, 2023
@martint
Copy link
Member

martint commented Aug 31, 2023

It's caused by 0f4b813, which surfaced a pre-existing bug when pushing down predicates into connectors under the following conditions:

  • There's a composite expression involving terms that can be converted into TupleDomain and term which can't be
  • The residual expression contains a lambda expression

In that case, the LayoutConstraintEvaluator incorrectly attempts to resolve references in the lambda body as columns, and fails if one hasn't been bound.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working RELEASE-BLOCKER
Development

Successfully merging a pull request may close this issue.

2 participants