Skip to content

chore: Add hive windows workflow#4850

Open
comphead wants to merge 1 commit into
apache:mainfrom
comphead:hive_windows
Open

chore: Add hive windows workflow#4850
comphead wants to merge 1 commit into
apache:mainfrom
comphead:hive_windows

Conversation

@comphead

@comphead comphead commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Closes #.

Rationale for this change

What changes are included in this PR?

How are these changes tested?

@kazuyukitanimura kazuyukitanimura left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just one comment
pending ci

# not include it, so `hive/testOnly *` above never sees HiveWindowFunctionQuerySuite
# et al. Prepend an SBT `set` command to graft that source directory into the hive
# project's test sources, then target the three compatibility suites explicitly.
- {name: "sql_hive_compat-1", args1: "'set LocalProject(\"hive\") / Test / unmanagedSourceDirectories += file(\"sql/hive/compatibility/src/test/scala\")'", args2: "hive/testOnly org.apache.spark.sql.hive.execution.HiveWindowFunctionQuerySuite org.apache.spark.sql.hive.execution.HiveWindowFunctionQueryFileSuite org.apache.spark.sql.hive.execution.HiveCompatibilitySuite -- -l org.apache.spark.tags.ExtendedHiveTest -l org.apache.spark.tags.SlowHiveTest"}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we use hive/test instead of hive/testOnly and exclude already tested tags...

@comphead comphead changed the title Add hive windows workflow chore: Add hive windows workflow Jul 7, 2026
@andygrove

Copy link
Copy Markdown
Member

Nice, I see it reproduced the issue

@comphead

comphead commented Jul 8, 2026

Copy link
Copy Markdown
Contributor Author

The Spark checks Spark result against HiveQL, compares the string by string, no way we can insert something like checkSparkAnswerWithTol.

the golden file looks like that src/test/resources/golden/windowing.q -- 5. testCountWithWindowingUDAF-0-3bde93728761b780a745c2ce0398aa0f
The comparison is line by line
org/apache/spark/sql/hive/execution/HiveComparisonTest.scala:371

So I'm not sure if this hive WF actually needed

@parthchandra

Copy link
Copy Markdown
Contributor

the golden file looks like that src/test/resources/golden/windowing.q -- 5. testCountWithWindowingUDAF-0-3bde93728761b780a745c2ce0398aa0f The comparison is line by line org/apache/spark/sql/hive/execution/HiveComparisonTest.scala:371

If we decide not to skip these tests then it should be possible to change Spark's comparison to be smarter in the diff file.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants