Skip to content

[regression] split hive compress-type heavy scan into p2#61073

Open
xylaaaaa wants to merge 1 commit intoapache:masterfrom
xylaaaaa:split-hive-compress-type-p0-p2
Open

[regression] split hive compress-type heavy scan into p2#61073
xylaaaaa wants to merge 1 commit intoapache:masterfrom
xylaaaaa:split-hive-compress-type-p0-p2

Conversation

@xylaaaaa
Copy link
Contributor

@xylaaaaa xylaaaaa commented Mar 5, 2026

Summary

  • slim down test_hive_compress_type in external_table_p0 by removing heavy test_compress_partitioned scans
  • add new test_hive_compress_type_large_data in external_table_p2 to cover the moved large-data checks

Why

  • reduce externregression p0 runtime for this case
  • keep heavy-data/file-split behavior validation in p2 instead of dropping coverage

Details

  • p0 keeps lightweight parquet LZ4/LZO compression query checks
  • p2 keeps large-table checks with file_split_size=0 and file_split_size=8388608 and validates row counts/scan split expectations

Test

  • not run locally; CI pipeline validation expected

Copilot AI review requested due to automatic review settings March 5, 2026 08:27
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR moves the heavy Hive compression/scan-splitting regression coverage out of the external_table_p0 suite into a new external_table_p2 suite to reduce p0 runtime while keeping large-data validation.

Changes:

  • Removed large test_compress_partitioned scan/split assertions from external_table_p0/hive/test_hive_compress_type.groovy.
  • Added external_table_p2/hive/test_hive_compress_type_large_data.groovy to cover the large-data file_split_size behaviors and row-count checks.
  • Kept lightweight parquet LZ4/LZO checks in the p0 suite.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File Description
regression-test/suites/external_table_p2/hive/test_hive_compress_type_large_data.groovy Adds a new p2 suite for large-data test_compress_partitioned split-count expectations and count validations (note: currently contains malformed Groovy triple-quote SQL strings on the dt="gzip" or dt="mix" queries, which will fail to parse).
regression-test/suites/external_table_p0/hive/test_hive_compress_type.groovy Removes the heavy scan/split assertions so p0 retains only lightweight parquet compression checks.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

suite("test_hive_compress_type_large_data", "p2,external") {
String enabled = context.config.otherConfigs.get("enableHiveTest")
if (enabled == null || !enabled.equalsIgnoreCase("true")) {
logger.info("diable Hive test.")
Copy link

Copilot AI Mar 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Log message has a typo: "diable Hive test." should be "disable Hive test." (or "disable Hive tests").

Suggested change
logger.info("diable Hive test.")
logger.info("disable Hive tests.")

Copilot uses AI. Check for mistakes.
@xylaaaaa
Copy link
Contributor Author

xylaaaaa commented Mar 5, 2026

run buildall

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants