Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA] Run Spark UT with spark-rapids #10744

Open
8 tasks
GaryShen2008 opened this issue Apr 26, 2024 · 0 comments
Open
8 tasks

[FEA] Run Spark UT with spark-rapids #10744

GaryShen2008 opened this issue Apr 26, 2024 · 0 comments
Assignees
Labels
epic Issue that encompasses a significant feature or body of work feature request New feature or request test Only impacts tests

Comments

@GaryShen2008
Copy link
Collaborator

GaryShen2008 commented Apr 26, 2024

Is your feature request related to a problem? Please describe.
Run the Spark UT with enabling spark-rapids to verify the correctness of features as many as possible.
The test cases are in https://github.com/apache/spark/tree/master/sql/.

There may be different Suite code in different Spark versions. It means we need to push different test code to shims.
And we can focus on Spark 3.3.0 as the first step.

Here're some Suite we may begin from

  • Initialize a test framework to run Spark UT in Spark 3.3.0 shims #10745
  • ./catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/MathExpressionsSuite.scala
  • ./catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/RegexpExpressionsSuite.scala
  • ./core/src/test/scala/org/apache/spark/sql/StringFunctionsSuite.scala
  • ./core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala
  • ./catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/JsonExpressionsSuite.scala
  • ./core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala
  • To add more
@GaryShen2008 GaryShen2008 added feature request New feature or request ? - Needs Triage Need team to review and classify epic Issue that encompasses a significant feature or body of work labels Apr 26, 2024
@GaryShen2008 GaryShen2008 added the test Only impacts tests label Apr 26, 2024
@mattahrens mattahrens removed the ? - Needs Triage Need team to review and classify label Apr 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
epic Issue that encompasses a significant feature or body of work feature request New feature or request test Only impacts tests
Projects
None yet
Development

No branches or pull requests

3 participants