Skip to content

[#12736] fix(doris): validate ADD INDEX fields - #12737

Merged
yuqi1129 merged 1 commit into
apache:mainfrom
jiangxt2:fix/doris-alter-add-index-field-validation
Aug 31, 2026
Merged

[#12736] fix(doris): validate ADD INDEX fields#12737
yuqi1129 merged 1 commit into
apache:mainfrom
jiangxt2:fix/doris-alter-add-index-field-validation

Conversation

@jiangxt2

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

This pull request adds a Doris-specific field-shape validator shared by CREATE TABLE index generation and ALTER TABLE ADD INDEX. The validator requires exactly one non-blank top-level field and returns that validated field for SQL generation.

It also adds focused tests for valid index SQL, malformed field shapes, consistent CREATE and ALTER validation, the no-JDBC-statement failure path, and the real Doris 4.x ADD/DROP INDEX lifecycle.

Why are the changes needed?

The Doris ALTER ADD INDEX path currently reads only getFieldNames()[0][0]. When a request contains multiple fields or a nested field path, Gravitino silently discards the remaining components and can create a valid single-column Doris index that does not match the request.

Rejecting unsupported field shapes before DDL generation prevents this semantic truncation and makes CREATE and ALTER enforce the same Doris connector contract. Valid single-field index SQL and existing key-model behavior remain unchanged.

Fix: #12736

Does this PR introduce any user-facing change?

Yes. Doris CREATE TABLE and ALTER TABLE ADD INDEX requests containing multiple fields, nested field paths, empty field shapes, or blank field names now fail with an explicit IllegalArgumentException instead of being truncated or producing invalid SQL. Valid single-field index operations are unchanged, and no public API or property key is added or removed.

How was this patch tested?

  • ./gradlew :catalogs:catalog-jdbc-doris:spotlessCheck
  • ./gradlew rat
  • ./gradlew :catalogs:catalog-jdbc-doris:test -PskipITs
  • ./gradlew :catalogs:catalog-jdbc-doris:test --tests 'org.apache.gravitino.catalog.doris.integration.test.CatalogDoris4xIT.testAddAndDropInvertedIndex' -PskipDockerTests=false -PdorisMultiVersionTest
  • ./gradlew :catalogs:catalog-jdbc-doris:build -x test

Validate Doris index field shapes consistently for CREATE and ALTER before generating DDL, preventing silent truncation of multiple or nested fields.

Signed-off-by: jiangxt2 <jiangxt2@vip.qq.com>
@github-actions

Copy link
Copy Markdown

Code Coverage Report

Overall Project 68.54% +0.09% 🟢
Files changed 82.85% 🟢

Module Coverage
aliyun 19.74% 🔴
api 52.09% 🟢
authorization-common 85.96% 🟢
authorization-ranger 4.38% 🔴
aws 53.54% 🟢
azure 32.1% 🔴
catalog-common 9.9% 🔴
catalog-fileset 80.3% 🟢
catalog-glue 69.24% 🟢
catalog-hive 82.96% 🟢
catalog-jdbc-common 45.69% 🟢
catalog-jdbc-doris 82.69% +2.28% 🟢
catalog-jdbc-mysql 79.33% 🟢
catalog-jdbc-postgresql 83.39% 🟢
catalog-jdbc-starrocks 79.16% 🟢
catalog-kafka 76.99% 🟢
catalog-lakehouse-generic 60.55% 🟢
catalog-lakehouse-hudi 79.1% 🟢
catalog-lakehouse-iceberg 85.93% 🟢
catalog-lakehouse-paimon 84.26% 🟢
catalog-model 77.99% 🟢
cli 44.48% 🟢
client-java 77.44% 🟢
common 55.5% 🟢
core 83.59% -0.04% 🟢
filesystem-hadoop3 76.45% 🟢
flink 0.0% 🔴
flink-common 50.29% 🟢
flink-runtime 0.0% 🔴
gcp 32.2% 🔴
hadoop-auth 68.0% 🟢
hadoop-common 17.84% 🔴
hive-metastore-common 53.4% 🟢
iceberg-aliyun-bundle 0.0% 🔴
iceberg-common 64.75% 🟢
iceberg-rest-server 75.96% 🟢
idp-basic 85.98% 🟢
integration-test-common 0.0% 🔴
jobs 62.92% 🟢
lance-common 32.63% 🔴
lance-rest-server 65.46% 🟢
lineage 53.02% 🟢
optimizer 83.24% 🟢
optimizer-api 21.95% 🔴
server 88.11% 🟢
server-common 80.5% 🟢
spark 28.57% 🔴
spark-common 48.92% 🟢
tencent 69.84% 🟢
trino-connector 47.44% 🟢
Files
Module File Coverage
catalog-jdbc-doris DorisTableOperations.java 84.91% 🟢
core AsyncQueueListener.java 66.18% 🟢

@yuqi1129 yuqi1129 added the branch-1.3 Automatically cherry-pick commit to branch-1.3 label Aug 31, 2026
@yuqi1129
yuqi1129 merged commit 3704b7a into apache:main Aug 31, 2026
38 checks passed
yuqi1129 pushed a commit that referenced this pull request Sep 3, 2026
…ields (#12737) (#12762)

**Cherry-pick Information:**
- Original commit: `3704b7a5bfe978421e43d53d5472f6c52bc15073`
- Original PR: #12737
- Target branch: `branch-1.3`
- Status: ✅ Conflict resolved manually in the `CatalogDoris4xIT`
assertion imports
- Replaces: #12753

### What changes were proposed in this pull request?

This pull request manually backports #12737 to `branch-1.3`. It
preserves the shared Doris CREATE/ALTER index field-shape validation and
its focused unit and Doris 4.x integration coverage.

### Why are the changes needed?

The automated cherry-pick PR #12753 contains unresolved conflict markers
in `CatalogDoris4xIT` and cannot be updated from the contributor's fork.
This PR resolves that conflict while preserving the original fix.

Fix: #12736

### Does this PR introduce _any_ user-facing change?

Yes. Doris CREATE TABLE and ALTER TABLE ADD INDEX requests containing
multiple fields, nested field paths, empty field shapes, or blank field
names now fail with an explicit `IllegalArgumentException` instead of
being truncated or producing invalid SQL. Valid single-field index
operations are unchanged, and no public API or property key is added or
removed.

### How was this patch tested?

- `./gradlew :catalogs:catalog-jdbc-doris:spotlessCheck`
- `./gradlew rat`
- `./gradlew :catalogs:catalog-jdbc-doris:test --tests
'org.apache.gravitino.catalog.doris.operation.TestDorisTableOperationsSqlGeneration'
-PskipITs`
- `./gradlew :catalogs:catalog-jdbc-doris:test --tests
'org.apache.gravitino.catalog.doris.integration.test.CatalogDoris4xIT.testAddAndDropInvertedIndex'
-PskipDockerTests=false -PdorisMultiVersionTest` (1 test, 0 skipped, 0
failures, 0 errors)
- `./gradlew :catalogs:catalog-jdbc-doris:build -x test`

Signed-off-by: jiangxt2 <jiangxt2@vip.qq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

branch-1.3 Automatically cherry-pick commit to branch-1.3

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug report] Doris ALTER ADD INDEX silently truncates multiple or nested fields

2 participants