Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix NPE in pushdown to NULL expression #3408

Merged
merged 1 commit into from
Apr 10, 2020

Conversation

alexey-fin
Copy link
Contributor

toImmutableList does not accept nulls, so pushdown to a null expression fails:

Caused by: java.lang.NullPointerException
at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:877)
at com.google.common.collect.ImmutableList$Builder.add(ImmutableList.java:788)
at java.base/java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169)
at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195)
at java.base/java.util.Collections$2.tryAdvance(Collections.java:4747)
at java.base/java.util.Collections$2.forEachRemaining(Collections.java:4755)
at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484)
at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474)
at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913)
at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.base/java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578)
at io.prestosql.sql.planner.ExpressionInterpreter$Visitor.visitBindExpression(ExpressionInterpreter.java:974)
....

@cla-bot cla-bot bot added the cla-signed label Apr 10, 2020
@alexey-fin alexey-fin requested a review from martint April 10, 2020 11:32
@tooptoop4
Copy link
Contributor

@alexey-fin what is a sample query affected?

Copy link
Member

@martint martint left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Can you add a test that reproduces the issue? It should go in TestExpressionInterpreter.

@@ -971,7 +971,7 @@ protected Object visitBindExpression(BindExpression node, Object context)
{
List<Object> values = node.getValues().stream()
.map(value -> process(value, context))
.collect(toImmutableList());
.collect(Collectors.toList());
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

static import + comment

Suggested change
.collect(Collectors.toList());
.collect(toList()); // values are nullable

@@ -57,6 +57,17 @@
super(ImmutableMap.of(ENABLE_DYNAMIC_FILTERING, "true"));
}

@Test
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@martint
i think the test should be in the planner,
is it a good enough place?

@alexey-fin
Copy link
Contributor Author

@tooptoop4 this fails
WITH
u(k, v) AS (SELECT null, CAST(name AS varchar(4)) FROM nation)
SELECT 1
FROM u
WHERE try(cast(u.k as varchar)) = 'x'

@@ -57,6 +57,17 @@
super(ImmutableMap.of(ENABLE_DYNAMIC_FILTERING, "true"));
}

@Test
public void testPushdownBindToNull() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This failure is unrelated to predicate pushdown. It's due to the null being inlined in the argument to the internal bind function that's used to model try when the InlineProjections optimizer rule executes. The query can be simplified to the following and still reproduce the issue:

SELECT try(k)
FROM (SELECT null) t(k)

Let's add a test using the simplified query in io.prestosql.sql.query.TestExpressions instead. Also add a comment in the test linking to the github issue so that we can trace back the reason for the test in the future.

@martint
Copy link
Member

martint commented Apr 10, 2020

There's a checkstyle error:

[ERROR] src/test/java/io/prestosql/sql/query/TestExpressions.java:[53,38] (blocks) LeftCurly: '{' at column 38 should be on a new line.

Also, can you squash the commits? Once the build passes, I'll merge this. Thanks for finding and fixing this issue!

@alexey-fin alexey-fin reopened this Apr 10, 2020
@alexey-fin
Copy link
Contributor Author

@martint
sorry for the mess

@martint martint merged commit ece91e5 into trinodb:master Apr 10, 2020
@martint
Copy link
Member

martint commented Apr 10, 2020

Merged, thanks!

@martint martint added this to the 333 milestone Apr 10, 2020
@martint martint mentioned this pull request Apr 21, 2020
8 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

4 participants