Skip to content

cherry-pick(1.12.7): fix(csv): correct entityType in recursive import extension validation + row-count accounting (#27593)#27669

Merged
yan-3005 merged 1 commit into1.12.7from
cherry-pick/27593-to-1.12.7
Apr 23, 2026
Merged

cherry-pick(1.12.7): fix(csv): correct entityType in recursive import extension validation + row-count accounting (#27593)#27669
yan-3005 merged 1 commit into1.12.7from
cherry-pick/27593-to-1.12.7

Conversation

@yan-3005
Copy link
Copy Markdown
Contributor

@yan-3005 yan-3005 commented Apr 23, 2026

Summary

Cherry-pick of #27593 (commit 4930b105) to 1.12.7.
image

  • Wrong entityType in extension validation: All rows in a recursive import were validated against the top-level entityType (e.g. database), so custom properties registered on table triggered false "Unknown custom field" errors. Added rowEntityType field on EntityCsv set per-row in DatabaseServiceRepository, DatabaseRepository, and DatabaseSchemaRepository.
  • Row-count accounting bugs: Header row was counted as a processed/passed row; getRecordNumber() is 1-indexed including the header, so subtract 1 from all processed counts. Multiple field failures on one row incremented numberOfRowsFailed once per field — added countedFailureRecords Set to deduplicate.
  • Empty extension token on export: CsvUtil.addExtension now filters entries whose formatted value is blank so null/empty-list custom properties are never written as bare key: tokens in exported CSVs.
  • UI grid regression for all-fail imports: BulkEntityImportPage no longer routes FAILURE back to the upload step when rows were processed — only short-circuits for aborted or failure with processed=0 (malformed CSV).

Conflict resolution notes

  • EntityCsvTest.java: took 1.12.7 base (6 tests) + applied only the 3 count-change lines from 4930b105; new tests from main require infrastructure not present in 1.12.7.
  • DatabaseServiceResourceIT.java: took 1.12.7 base + added only test_recursiveImportCustomPropertyExtension; removed getVersionHistoryPaginated/getVersionHistoryWithFieldChanged overrides that reference SDK overloads not present in 1.12.7.
  • GlossaryImportExport.spec.ts: kept 1.12.7 assertion style, applied corrected row counts.
  • EntityCsv.java / other Java: resolved by merging 1.12.7 throws IOException declarations with main's new logic.

Test plan

  • mvn -pl openmetadata-service test -Dtest=CsvUtilTest,EntityCsvTest -DskipSwagger=true — CsvUtilTest all pass; EntityCsvTest 3 count-change assertions pass (4 pre-existing RuleEngine init failures on 1.12.7 unrelated to this PR)
  • mvn test-compile -pl openmetadata-integration-tests -DskipSwagger=true — clean compile
  • Playwright GlossaryImportExport.spec.ts — validated locally, "Import validation - missing required fields" passes with rebuilt backend

… extension validation + row-count accounting (#27593)

Cherry-pick of 4930b10 to 1.12.7.

Key changes:
- CsvUtil.addExtension: filter blank/empty extension tokens on export
- EntityCsv: rowEntityType field for per-row entity type override in extension validation
- EntityCsv: header row excluded from numberOfRowsProcessed/Passed counts
- EntityCsv: countedFailureRecords dedup to count per-row failures once
- EntityCsv: skip empty-value extension tokens instead of failing
- DatabaseServiceRepository/DatabaseRepository/DatabaseSchemaRepository: set rowEntityType per row
- BulkEntityImportPage: only short-circuit to upload on aborted or failure+processed=0
- GlossaryImportExport.spec.ts: fix version-history row counts (3->2) for header exclusion
- BulkEditEntity/BulkImport/TestCaseImportExport E2E: update row counts for header exclusion
- GlossaryResourceIT/TestCaseResourceIT: update expected counts for header exclusion
- DatabaseServiceResourceIT: add recursive import custom property extension IT test
- EntityCsvTest/CsvUtilTest: update assertSummary counts and blank extension filter assertion
@yan-3005 yan-3005 requested a review from a team as a code owner April 23, 2026 12:25
@github-actions
Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@gitar-bot
Copy link
Copy Markdown

gitar-bot Bot commented Apr 23, 2026

Code Review ✅ Approved

Backports the fix for CSV recursive import validation and row-count accounting to the 1.12.7 release branch. No issues found.

Options

Display: compact → Showing less information.

Comment with these commands to change:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@yan-3005 yan-3005 merged commit 355eca3 into 1.12.7 Apr 23, 2026
22 of 50 checks passed
@yan-3005 yan-3005 deleted the cherry-pick/27593-to-1.12.7 branch April 23, 2026 12:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant