Skip to content

Feat: support bit_get function #1713

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 20 commits into from
Jun 26, 2025
Merged

Conversation

kazantsev-maksim
Copy link
Contributor

@kazantsev-maksim kazantsev-maksim commented May 3, 2025

Which issue does this PR close?

Related to Epic: #240
bit_get: SELECT bit_get(0) => 0
DataFusionComet bit_get has same behavior with Spark 's bit_get function
Spark: https://spark.apache.org/docs/latest/api/sql/index.html#bit_get

Closes #.

Rationale for this change

Defined under Epic: #240

What changes are included in this PR?

bitwise_get.rs: impl for bit_get function
QueryPlanSerde.scala: bit_get pattern matching case has been added,
bitwise.scala: The bit_get function is the last function of the bitwise expressions type from the epic: #240. I moved them into a separate file, following the same approach as with the array functions.
CometBitwiseExpressionSuite.scala: A new UT has been added for bit_get function.

How are these changes tested?

A new UT has been added.

@kazantsev-maksim kazantsev-maksim changed the title Bitwise get Feat: support bit_get function May 3, 2025
@kazantsev-maksim kazantsev-maksim marked this pull request as draft May 3, 2025 09:50
@kazantsev-maksim kazantsev-maksim marked this pull request as ready for review May 3, 2025 10:28
Copy link
Contributor

@comphead comphead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this PR supposed to replace the #1602 PR ?

@codecov-commenter
Copy link

codecov-commenter commented May 5, 2025

Codecov Report

Attention: Patch coverage is 16.04938% with 68 lines in your changes missing coverage. Please review.

Project coverage is 58.47%. Comparing base (f09f8af) to head (ca7b32c).
Report is 286 commits behind head on main.

Files with missing lines Patch % Lines
...rc/main/scala/org/apache/comet/serde/bitwise.scala 16.41% 55 Missing and 1 partial ⚠️
.../scala/org/apache/comet/serde/QueryPlanSerde.scala 14.28% 6 Missing and 6 partials ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #1713      +/-   ##
============================================
+ Coverage     56.12%   58.47%   +2.34%     
- Complexity      976     1141     +165     
============================================
  Files           119      131      +12     
  Lines         11743    12897    +1154     
  Branches       2251     2396     +145     
============================================
+ Hits           6591     7541     +950     
- Misses         4012     4134     +122     
- Partials       1140     1222      +82     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kazantsev-maksim
Copy link
Contributor Author

@comphead I'd like to merge #1602 first, and then update this PR again, but the reverse order is also fine.

@parthchandra
Copy link
Contributor

Same comment as in #1602 (#1602 (comment))

Kazantsev Maksim added 7 commits May 9, 2025 14:08
# Conflicts:
#	native/spark-expr/src/bitwise_funcs/mod.rs
#	native/spark-expr/src/comet_scalar_funcs.rs
#	spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala
#	spark/src/test/scala/org/apache/comet/CometExpressionSuite.scala
@kazantsev-maksim kazantsev-maksim requested a review from comphead May 31, 2025 07:10
Kazantsev Maksim and others added 6 commits June 2, 2025 21:55
# Conflicts:
#	native/spark-expr/src/bitwise_funcs/mod.rs
#	spark/src/main/scala/org/apache/comet/serde/QueryPlanSerde.scala
# Conflicts:
#	native/spark-expr/src/bitwise_funcs/mod.rs
#	native/spark-expr/src/comet_scalar_funcs.rs
@andygrove
Copy link
Member

andygrove commented Jun 26, 2025

@kazantsev-maksim I took the liberty of upmerging this PR

@parthchandra @comphead could you take another look when you have a chance?

Copy link
Contributor

@parthchandra parthchandra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

val bitwiseNotExpr = expr.asInstanceOf[BitwiseNot]
val childProto = exprToProto(bitwiseNotExpr.child, inputs, binding)
val bitNotScalarExpr =
scalarFunctionExprToProto("bit_not", childProto)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can't find such function 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This function based on: native/spark-expr/src/bitwise_funcs/bitwise_not.rs

Copy link
Contributor

@comphead comphead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm thanks @andygrove I cannot find bit_not function but it is not related to this specific PR

@andygrove andygrove merged commit c51f977 into apache:main Jun 26, 2025
102 of 103 checks passed
@andygrove
Copy link
Member

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants