Skip to content

feat(flink): Support writing VECTOR columns for flink writer#18877

Draft
cshuo wants to merge 2 commits into
apache:masterfrom
cshuo:support_vector_write
Draft

feat(flink): Support writing VECTOR columns for flink writer#18877
cshuo wants to merge 2 commits into
apache:masterfrom
cshuo:support_vector_write

Conversation

@cshuo
Copy link
Copy Markdown
Collaborator

@cshuo cshuo commented May 29, 2026

Describe the issue this Pull Request addresses

Summary and Changelog

Impact

Risk Level

Documentation Update

Contributor's checklist

  • Read through contributor's guide
  • Enough context is provided in the sections above
  • Adequate tests were added if applicable

@cshuo cshuo marked this pull request as draft May 29, 2026 03:05
@github-actions github-actions Bot added the size:XL PR with lines of changes > 1000 label May 29, 2026
@cshuo cshuo force-pushed the support_vector_write branch from 86fd838 to f88222b Compare May 29, 2026 09:30
@cshuo cshuo force-pushed the support_vector_write branch from f88222b to d9f9492 Compare May 29, 2026 12:27
@codecov-commenter
Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 73.48703% with 92 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.81%. Comparing base (7af8cdf) to head (d9f9492).
⚠️ Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
...va/org/apache/hudi/util/VectorConversionUtils.java 39.17% 51 Missing and 8 partials ⚠️
...va/org/apache/hudi/util/HoodieSchemaConverter.java 76.66% 7 Missing and 7 partials ⚠️
...i/io/storage/row/parquet/ParquetRowDataWriter.java 81.48% 2 Missing and 3 partials ⚠️
...org/apache/hudi/common/util/HoodieVectorUtils.java 84.61% 2 Missing and 2 partials ⚠️
.../java/org/apache/hudi/table/HoodieTableSource.java 50.00% 3 Missing ⚠️
.../hudi/table/format/HoodieRowDataParquetReader.java 57.14% 2 Missing and 1 partial ⚠️
...io/storage/row/HoodieRowDataFileWriterFactory.java 0.00% 1 Missing ⚠️
...io/storage/row/parquet/ParquetSchemaConverter.java 96.42% 0 Missing and 1 partial ⚠️
...hudi/SparkFileFormatInternalRowReaderContext.scala 0.00% 0 Missing and 1 partial ⚠️
.../hudi/table/format/cow/CopyOnWriteInputFormat.java 80.00% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #18877      +/-   ##
============================================
- Coverage     68.82%   68.81%   -0.01%     
- Complexity    29134    29221      +87     
============================================
  Files          2515     2517       +2     
  Lines        139940   140204     +264     
  Branches      17190    17242      +52     
============================================
+ Hits          96307    96487     +180     
- Misses        35857    35922      +65     
- Partials       7776     7795      +19     
Flag Coverage Δ
common-and-other-modules 44.39% <71.46%> (+0.06%) ⬆️
hadoop-mr-java-client 44.89% <0.00%> (-0.03%) ⬇️
spark-client-hadoop-common 48.22% <0.00%> (-0.01%) ⬇️
spark-java-tests 49.37% <78.12%> (-0.02%) ⬇️
spark-scala-tests 45.27% <43.75%> (-0.01%) ⬇️
utilities 37.43% <18.75%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...hudi/io/storage/row/HoodieRowDataCreateHandle.java 87.30% <100.00%> (+0.30%) ⬆️
.../storage/row/HoodieRowDataParquetWriteSupport.java 100.00% <100.00%> (ø)
...udi/io/storage/row/RowDataParquetWriteSupport.java 88.88% <100.00%> (+0.65%) ⬆️
.../org/apache/hudi/util/AvroToRowDataConverters.java 53.84% <100.00%> (+5.36%) ⬆️
...ava/org/apache/hudi/util/RowDataQueryContexts.java 78.04% <100.00%> (ø)
.../org/apache/hudi/util/RowDataToAvroConverters.java 73.52% <100.00%> (+2.85%) ⬆️
...ache/hudi/io/storage/HoodieSparkParquetReader.java 78.49% <100.00%> (ø)
.../apache/hudi/io/storage/VectorConversionUtils.java 81.48% <100.00%> (+1.08%) ⬆️
...va/org/apache/hudi/configuration/FlinkOptions.java 99.78% <100.00%> (+<0.01%) ⬆️
...pache/hudi/sink/clustering/ClusteringOperator.java 48.83% <100.00%> (+0.39%) ⬆️
... and 18 more

... and 12 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@hudi-bot
Copy link
Copy Markdown
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL PR with lines of changes > 1000

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants