Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Filtering on a boolean field with just the field name results in NullPointerException #8444

Closed
jadami10 opened this issue Mar 30, 2022 · 8 comments
Closed

Filtering on a boolean field with just the field name results in NullPointerException #8444

jadami10 opened this issue Mar 30, 2022 · 8 comments
Labels
bug

Comments

@jadami10
Copy link
Contributor

Running a query with just select <fields> from <table> where <boolean_field> results in a null point exception. This should be a supported query.

Stack trace from pinot

[2022-03-30 20:40:05.343615] java.lang.NullPointerException: null
[2022-03-30 20:40:05.343660] 	at org.apache.pinot.core.query.optimizer.filter.FlattenAndOrFilterOptimizer.optimize(FlattenAndOrFilterOptimizer.java:69) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.343694] 	at org.apache.pinot.core.query.optimizer.filter.FlattenAndOrFilterOptimizer.optimize(FlattenAndOrFilterOptimizer.java:77) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.343736] 	at org.apache.pinot.core.query.optimizer.filter.FlattenAndOrFilterOptimizer.optimize(FlattenAndOrFilterOptimizer.java:64) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.343766] 	at org.apache.pinot.core.query.optimizer.QueryOptimizer.optimize(QueryOptimizer.java:81) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.343796] 	at org.apache.pinot.broker.requesthandler.BaseBrokerRequestHandler.handleSQLRequest(BaseBrokerRequestHandler.java:404) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.343830] 	at org.apache.pinot.broker.requesthandler.BaseBrokerRequestHandler.handleRequest(BaseBrokerRequestHandler.java:196) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.343863] 	at org.apache.pinot.broker.requesthandler.BaseBrokerRequestHandler.handleRequest(BaseBrokerRequestHandler.java:101) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.343892] 	at org.apache.pinot.broker.api.resources.PinotClientRequest.processSqlQueryPost(PinotClientRequest.java:191) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.343915] 	at jdk.internal.reflect.GeneratedMethodAccessor64.invoke(Unknown Source) ~[?:?]
[2022-03-30 20:40:05.343938] 	at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
[2022-03-30 20:40:05.343963] 	at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
[2022-03-30 20:40:05.344004] 	at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344039] 	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344075] 	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344105] 	at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$VoidOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:159) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344140] 	at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344177] 	at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:469) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344222] 	at org.glassfish.jersey.server.model.ResourceMethodInvoker.lambda$apply$0(ResourceMethodInvoker.java:381) ~[pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344257] 	at org.glassfish.jersey.server.ServerRuntime$AsyncResponder$2$1.run(ServerRuntime.java:819) [pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344289] 	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) [pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344312] 	at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) [pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344346] 	at org.glassfish.jersey.internal.Errors.process(Errors.java:292) [pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344372] 	at org.glassfish.jersey.internal.Errors.process(Errors.java:274) [pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344398] 	at org.glassfish.jersey.internal.Errors.process(Errors.java:244) [pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344429] 	at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) [pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344456] 	at org.glassfish.jersey.server.ServerRuntime$AsyncResponder$2.run(ServerRuntime.java:814) [pinot-all-0.10.0-2022-03-30-4a8f605171-SNAPSHOT-jar-with-dependencies.jar:0.10.0-2022-03-30-4a8f605171-SNAPSHOT-4a8f6051719a3cbd04abff1b454998ac352248d4]
[2022-03-30 20:40:05.344480] 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) [?:?]
[2022-03-30 20:40:05.344524] 	at java.util.concurrent.FutureTask.run(FutureTask.java:264) [?:?]
[2022-03-30 20:40:05.344545] 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
[2022-03-30 20:40:05.344566] 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
[2022-03-30 20:40:05.344582] 	at java.lang.Thread.run(Thread.java:829) [?:?]

user facing error

ProcessingException(errorCode:450, message:InternalError:
java.io.IOException: Failed : HTTP error code : 500
	at org.apache.pinot.controller.api.resources.PinotQueryResource.sendPostRaw(PinotQueryResource.java:305)
	at org.apache.pinot.controller.api.resources.PinotQueryResource.sendRequestRaw(PinotQueryResource.java:343)
	at org.apache.pinot.controller.api.resources.PinotQueryResource.getQueryResponse(PinotQueryResource.java:225)
	at org.apache.pinot.controller.api.resources.PinotQueryResource.handlePostSql(PinotQueryResource.java:138))
@KKcorps
Copy link
Contributor

KKcorps commented Apr 4, 2022

@Jackie-Jiang Should we go with re-writing the query WHERE A to WHERE A IS TRUE ?

@Jackie-Jiang
Copy link
Contributor

@KKcorps Good solution! We can re-write the filter expression if the underlying predicate does not have FilterKind as the function. Pinot supports WHERE A = true

@siddharthteotia
Copy link
Contributor

@Jackie-Jiang / @KKcorps / @jadami10 are you planning to work on this ?

If not, @vvivekiyer can help pick it up as he is familiarizing himself through the code

@KKcorps
Copy link
Contributor

KKcorps commented Apr 5, 2022

@siddharthteotia I am working on BrokerResponse so @vvivekiyer can pick it up

@siddharthteotia
Copy link
Contributor

@vvivekiyer is working on the fix

@Jackie-Jiang Jackie-Jiang mentioned this issue Apr 8, 2022
Closed
@vvivekiyer vvivekiyer mentioned this issue Apr 12, 2022
Merged
vvivekiyer pushed a commit to vvivekiyer/pinot that referenced this issue May 2, 2022
Support filtering on bool/scalar fields without evaluator
a229fc7 
Fixes the bugs in apache#8444 and apache#8487.
Added unit tests.
siddharthteotia pushed a commit that referenced this issue May 2, 2022
@vvivekiyer
Support filtering on bool/scalar fields without evaluator (#8518)
d49d117 
* Support filtering on bool/scalar fields without evaluator

Fixes the bugs in #8444 and #8487.
Added unit tests.

* Address review comments and refactor code

* Address review comments 2

* RequestContextUtil changes

* Review comments in RequestContextUtils

* Refactor RequestContextUtils

* Refactor ComparisonPredicateRewriter

Co-authored-by: Vivek Iyer Vaidyanathan <vvaidyan@vvaidyan-mn1.linkedin.biz>
@siddharthteotia
Copy link
Contributor

Fixed with #8518

@jadami10
Copy link
Contributor Author

jadami10 commented May 3, 2022

thanks for working on this! I didn't see a test example, but does WHERE NOT <boolean_field> work correctly?

@vvivekiyer
Copy link
Contributor

Hi Johan, yes it should. I just tested it out locally for a use-case. I didn't add a test-case example because AND, OR, NOT are processed similarly.

@vvivekiyer vvivekiyer mentioned this issue May 3, 2022
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@siddharthteotia @jadami10 @KKcorps @Jackie-Jiang @vvivekiyer