fix: cherry-pick 413 iterative bisection and TaggedOperation refactor into 1.12.5

[manerow]

Cherry-pick of #26347 (da400385d4) from main into 1.12.5.

Changes

• Adds iterative bisection on 413 (Request Entity Too Large) responses in ElasticSearchIndexSink and OpenSearchIndexSink — instead of failing the whole batch, the sink splits it in half recursively until the oversized document is isolated and skipped with a clear error
• Introduces TaggedOperation wrapper so entity references are tracked through the bulk pipeline, ensuring errors report the actual entity instead of a positional index
• Conflict resolution: kept all 1.12.5-specific test dependencies in openmetadata-service/pom.xml and dropped junit-jupiter-params which was removed by the original commit

Motivation

The Data Insights Application on customer environments was failing with 413 errors when bulk indexing large entity batches. This fix was already merged to main but was missing from the 1.12.5 release branch.

Fixes #26344

[gitar-bot]

💡 Quality: Raw-type TaggedOperation loses compile-time type safety

In DataAssetsWorkflow.processSource, the processor returns List<?> and each element is wrapped as TaggedOperation<>(op, ref) where op is Object. The sink field is declared as raw Sink searchIndexSink, so write(batch) compiles without type checking. If the processor ever returns a non-BulkOperation object, the error would only surface at runtime deep inside the sink.

This is a pre-existing raw-type pattern on the Sink field, but the new code adds another layer of type erasure through TaggedOperation<?>. Consider narrowing the types to catch mismatches at compile time.

Suggested fix:

// In DataAssetsWorkflow, type the sink and processor fields properly:
private Sink<List<TaggedOperation<BulkOperation>>, BulkResponse> searchIndexSink;
private Processor<List<BulkOperation>, EntityInterface> entityProcessor;

// Then the forEach becomes type-safe:
List<BulkOperation> bulkOps = entityProcessor.process(enriched, contextData);
bulkOps.forEach(op -> opsQueue.add(new TaggedOperation<>(op, ref)));

_{Was this helpful? React with 👍 / 👎 | Reply gitar fix to apply this suggestion}

[gitar-bot]

Code Review 👍 Approved with suggestions 0 resolved / 1 findings

Cherry-picks iterative bisection and TaggedOperation refactor into 1.12.5 to address issue #26344. Consider adding type parameters to TaggedOperation usage in DataAssetsWorkflow to maintain compile-time type safety.

💡 Quality: Raw-type TaggedOperation loses compile-time type safety

📄 openmetadata-service/src/main/java/org/openmetadata/service/apps/bundles/insights/workflows/dataAssets/DataAssetsWorkflow.java:289-292 📄 openmetadata-service/src/main/java/org/openmetadata/service/apps/bundles/insights/workflows/dataAssets/DataAssetsWorkflow.java:347-356

In DataAssetsWorkflow.processSource, the processor returns List<?> and each element is wrapped as TaggedOperation<>(op, ref) where op is Object. The sink field is declared as raw Sink searchIndexSink, so write(batch) compiles without type checking. If the processor ever returns a non-BulkOperation object, the error would only surface at runtime deep inside the sink.

This is a pre-existing raw-type pattern on the Sink field, but the new code adds another layer of type erasure through TaggedOperation<?>. Consider narrowing the types to catch mismatches at compile time.

Suggested fix

// In DataAssetsWorkflow, type the sink and processor fields properly:
private Sink<List<TaggedOperation<BulkOperation>>, BulkResponse> searchIndexSink;
private Processor<List<BulkOperation>, EntityInterface> entityProcessor;

// Then the forEach becomes type-safe:
List<BulkOperation> bulkOps = entityProcessor.process(enriched, contextData);
bulkOps.forEach(op -> opsQueue.add(new TaggedOperation<>(op, ref)));

🤖 Prompt for agents

Code Review: Cherry-picks iterative bisection and TaggedOperation refactor into 1.12.5 to address issue #26344. Consider adding type parameters to TaggedOperation usage in DataAssetsWorkflow to maintain compile-time type safety.

1. 💡 Quality: Raw-type TaggedOperation loses compile-time type safety
   Files: openmetadata-service/src/main/java/org/openmetadata/service/apps/bundles/insights/workflows/dataAssets/DataAssetsWorkflow.java:289-292, openmetadata-service/src/main/java/org/openmetadata/service/apps/bundles/insights/workflows/dataAssets/DataAssetsWorkflow.java:347-356

   In `DataAssetsWorkflow.processSource`, the processor returns `List<?>` and each element is wrapped as `TaggedOperation<>(op, ref)` where `op` is `Object`. The sink field is declared as raw `Sink searchIndexSink`, so `write(batch)` compiles without type checking. If the processor ever returns a non-BulkOperation object, the error would only surface at runtime deep inside the sink.
   
   This is a pre-existing raw-type pattern on the `Sink` field, but the new code adds another layer of type erasure through `TaggedOperation<?>`. Consider narrowing the types to catch mismatches at compile time.

   Suggested fix:
   // In DataAssetsWorkflow, type the sink and processor fields properly:
   private Sink<List<TaggedOperation<BulkOperation>>, BulkResponse> searchIndexSink;
   private Processor<List<BulkOperation>, EntityInterface> entityProcessor;
   
   // Then the forEach becomes type-safe:
   List<BulkOperation> bulkOps = entityProcessor.process(enriched, contextData);
   bulkOps.forEach(op -> opsQueue.add(new TaggedOperation<>(op, ref)));

Options

Display: compact → Showing less information.

Comment with these commands to change:

Compact

gitar display:verbose

_{Was this helpful? React with 👍 / 👎 | Gitar}

[manerow]

Dropping this for now — will cherry-pick into 1.12.6 once the branch is up.