Support FetchOrientation.FetchFirst #27
Comments
yaooqinn
pushed a commit
that referenced
this issue
Nov 15, 2023
### _Why are the changes needed?_ To close #5677 Typeof expression miss column information, in this pr we add a place holder to support this. For sql ``` SELECT typeof(id), typeof(age) FROM default.t1 LIMIT 1 ``` Without this pr, typeof after optimizer passed to PrivilegeBuild is just an attribute, miss the origin child expression info. ``` GlobalLimit 21 +- LocalLimit 21 +- Project [int AS typeof(id)#27, int AS typeof(age)#28] +- Relation default.t1[id#18,age#19,part#20] parquet ``` When handle this plan's project list, it's an attribute and miss expression when expression result is a constant value, then we can't extract the `TypeOf`'s child expression.  This is caused by `TypeOf` expression is foldable <img width="778" alt="image" src="https://github.com/apache/kyuubi/assets/46485123/770a36fc-235f-4f26-bca7-a5058e120919"> It will be convert to a constant value after spark optimizer, then cause we miss the child expression, then can't extract columns by `collectLeaves` In this pr we wrap the `TypeOf` by a non-foldable expression placeholder then we can get the expression contains column of table when `mergeProjection` After this pr, the plan is like below ``` GlobalLimit 21 +- LocalLimit 21 +- Project [typeofplaceholder(id#21) AS typeof(id)#30, typeofplaceholder(day#23) AS typeof(day)#31] +- HiveTableRelation [`default`.`table1`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, Data Cols: [id#21, scope#22, day#23], Partition Cols: []] ``` ### _How was this patch tested?_ - [x] Add some test cases that check the changes thoroughly including negative and positive cases if possible - [ ] Add screenshots for manual tests if appropriate - [ ] [Run test](https://kyuubi.readthedocs.io/en/master/contributing/code/testing.html#running-tests) locally before make a pull request ### _Was this patch authored or co-authored using generative AI tooling?_ No Closes #5678 from AngersZhuuuu/KYUUBO-5677. Closes #5677 273cd61 [Angerszhuuuu] Update TypeOfPlaceHolder.scala 8b4383d [Angerszhuuuu] done af5dae5 [Angerszhuuuu] Update RangerSparkExtensionSuite.scala 37edcdf [Angerszhuuuu] fix for spark-3.1 df15599 [Angerszhuuuu] update a7dd3c4 [Angerszhuuuu] Update RangerSparkExtension.scala 6b76dac [Angerszhuuuu] [KYUUBI #5677][AUTHZ] Typeof expression miss column information f8a94c7 [Angerszhuuuu] [KYUUBI #5677][AUTHZ] Typeof expression miss column information Authored-by: Angerszhuuuu <angers.zhu@gmail.com> Signed-off-by: Kent Yao <yao@apache.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Expected behavior
Actual behavior.
Steps to reproduce the problem.
Specifications like the version of the project, operating system, or hardware.
The text was updated successfully, but these errors were encountered: