Skip to content

[Fix](Regression) reduce duplicate hive2/hive3 regression runs#61671

Open
xylaaaaa wants to merge 1 commit intoapache:masterfrom
xylaaaaa:codex/extern-hive2-only-regression-20260324
Open

[Fix](Regression) reduce duplicate hive2/hive3 regression runs#61671
xylaaaaa wants to merge 1 commit intoapache:masterfrom
xylaaaaa:codex/extern-hive2-only-regression-20260324

Conversation

@xylaaaaa
Copy link
Contributor

@xylaaaaa xylaaaaa commented Mar 24, 2026

Summary

  • reduce a set of general external Hive regression suites from hive2+hive3 dual-run to hive3-only
  • keep the more version-sensitive Hive3-specific coverage separate
  • shrink duplicate extern regression workload without removing dedicated Hive3 feature cases

Testing

  • git diff --check -- regression-test/suites/external_table_p0 regression-test/suites/mtmv_p0

Copilot AI review requested due to automatic review settings March 24, 2026 09:13
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR reduces CI workload for external Hive regression testing by removing the duplicated Hive3 run from a set of “general” suites, keeping Hive3-only coverage to dedicated suites.

Changes:

  • Update multiple external Hive regression suites to iterate over ["hive2"] instead of ["hive2", "hive3"].
  • Apply the same hive2-only change to selected MTMV Hive suites.

Reviewed changes

Copilot reviewed 55 out of 55 changed files in this pull request and generated no comments.

Show a summary per file
File Description
regression-test/suites/mtmv_p0/test_hive_rewrite_mtmv.groovy Run MTMV rewrite coverage against Hive2 only
regression-test/suites/mtmv_p0/test_hive_olap_mtmv.groovy Run MTMV Hive↔Olap coverage against Hive2 only
regression-test/suites/mtmv_p0/test_hive_mtmv.groovy Run MTMV basic coverage against Hive2 only
regression-test/suites/external_table_p0/hive/test_wide_table.groovy Hive wide-table suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_utf8_check.groovy UTF-8 validation suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_upper_case_column_name.groovy Upper-case column name suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_truncate_char_or_varchar_columns.groovy Truncate CHAR/VARCHAR suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_text_skip_header.groovy Text skip-header suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_text_garbled_file.groovy Garbled text file suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_string_dict_filter.groovy String dict filter suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_special_orc_formats.groovy Special ORC formats suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_parquet_nested_types.groovy Parquet nested types suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_parquet_bloom_filter.groovy Parquet bloom filter suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_orc_nested_types.groovy ORC nested types suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_open_csv_serde.groovy OpenCSV serde suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_multi_delimit_serde.groovy Multi-delimiter serde suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_mixed_par_locations.groovy Mixed partition locations suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_varbinary_type.groovy VARBINARY type suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_to_date.groovy to_date compatibility suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_to_array.groovy to_array compatibility suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_text_complex_type.groovy Text complex type suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_tablesample_p0.groovy TABLESAMPLE suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_statistics_p0.groovy Hive statistics suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_statistics_all_type_p0.groovy Hive statistics (all types) suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_statistic_timeout.groovy Hive statistic timeout suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_statistic_clean.groovy Hive statistic clean suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_statistic_auto.groovy Hive statistic auto suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_statistic.groovy Hive statistic suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_schema_evolution.groovy Hive schema evolution suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_same_db_table_name.groovy Same DB/table name suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_runtime_filter_partition_pruning.groovy Runtime filter partition pruning suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_remove_partition.groovy Remove partition suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_query_cache.groovy Query cache suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_partition_location.groovy Partition location suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_partition_column_analyze.groovy Partition-column analyze suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_parquet_skip_page.groovy Parquet skip-page suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_parquet_alter_column.groovy Parquet alter-column suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_parquet.groovy Parquet suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_other.groovy Misc Hive suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_orc.groovy ORC suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_get_schema_from_table.groovy Get-schema-from-table suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_default_partition.groovy Default partition suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_broker_scan.groovy Broker scan suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_basic_type.groovy Basic types suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_hive_analyze_db.groovy Analyze database suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_external_credit_data.groovy External credit data suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_external_catalog_hive_partition.groovy External catalog partition suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_external_catalog_hive.groovy External catalog suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_drop_expired_table_stats.groovy Drop expired stats suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_different_parquet_types.groovy Different parquet types suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_different_column_orders.groovy Different column orders suite: Hive2 only
regression-test/suites/external_table_p0/hive/test_complex_types.groovy Complex types suite: Hive2 only
regression-test/suites/external_table_p0/hive/hive_tpch_sf1_parquet.groovy TPC-H SF1 parquet suite: Hive2 only
regression-test/suites/external_table_p0/hive/hive_tpch_sf1_orc.groovy TPC-H SF1 ORC suite: Hive2 only
regression-test/suites/external_table_p0/hive/hive_json_basic_test.groovy Hive JSON basic suite: Hive2 only
Comments suppressed due to low confidence (2)

regression-test/suites/external_table_p0/hive/test_hive_analyze_db.groovy:47

  • This suite is test_hive_analyze_db, but catalog_name is ${hivePrefix}_test_hive_partition_column_analyze (same as test_hive_partition_column_analyze). Reusing another suite’s catalog name risks collisions/flaky behavior if suites run concurrently and makes test output confusing. Rename the catalog to something unique to test_hive_analyze_db.
         String extHiveHmsHost = context.config.otherConfigs.get("externalEnvIp")
         String extHiveHmsPort = context.config.otherConfigs.get(hivePrefix + "HmsPort")
         String catalog_name = "${hivePrefix}_test_hive_partition_column_analyze"
         sql """drop catalog if exists ${catalog_name};"""

regression-test/suites/external_table_p0/hive/test_text_skip_header.groovy:30

  • catalog_name is set to ${hivePrefix}_test_utf8_check, which is the same catalog name used by test_utf8_check and does not match this suite (test_hive_text_skip_header). This can cause cross-suite interference (dropping/recreating another suite’s catalog) and makes failures harder to diagnose. Use a suite-specific catalog name (e.g., based on test_hive_text_skip_header).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@xylaaaaa
Copy link
Contributor Author

run buildall

1 similar comment
@xylaaaaa
Copy link
Contributor Author

run buildall

@xylaaaaa xylaaaaa changed the title [test](external) reduce duplicate hive2/hive3 regression runs [Fix](Regression) reduce duplicate hive2/hive3 regression runs Mar 25, 2026
Keep general external Hive coverage on hive3-only and preserve the more version-sensitive suites separately.
@xylaaaaa xylaaaaa force-pushed the codex/extern-hive2-only-regression-20260324 branch from 04a5b62 to d229063 Compare March 25, 2026 05:43
@xylaaaaa
Copy link
Contributor Author

run buildall

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants