Skip to content

[GLUTEN-11630][VL] Enable iceberg tests#11641

Closed
jinchengchenghh wants to merge 5 commits intoapache:mainfrom
jinchengchenghh:fix-iceberg
Closed

[GLUTEN-11630][VL] Enable iceberg tests#11641
jinchengchenghh wants to merge 5 commits intoapache:mainfrom
jinchengchenghh:fix-iceberg

Conversation

@jinchengchenghh
Copy link
Contributor

@jinchengchenghh jinchengchenghh commented Feb 23, 2026

This PR adds IcebergConnector to adapt to Velox refactor.
Temporarily drop iceberg test in enhanced features because of the native code bug, waiting for fix.
Related issue: #11630

@jinchengchenghh
Copy link
Contributor Author

Could you help find what should I update for the enhanced feature, I have not update the write @PingLiuPing https://github.com/apache/incubator-gluten/blob/main/cpp/velox/compute/iceberg/IcebergWriter.cc

@PingLiuPing
Copy link

Could you help find what should I update for the enhanced feature, I have not update the write @PingLiuPing https://github.com/apache/incubator-gluten/blob/main/cpp/velox/compute/iceberg/IcebergWriter.cc

To use IBM/velox, you don't need to change anything, most of the PRs from upstream velox have been reverted.

@jinchengchenghh
Copy link
Contributor Author

Some revert may have issues, the test with Enhanced feature failed, looks like read delete does not make effect

Error:    Run 2: TestGlutenMergeOnReadUpdate>TestUpdate.testUpdateWithoutCondition:434->SparkTestHelperBase.assertEquals:61 [Should have expected rows: number of results should match] 
Actual and expected should have same size but actual size is:
  6
while expected size is:
  3
Actual was:
  [[-1, "hardware"],
    [2, "hardware"],
    [-1, "hr"],
    [-1, "hr"],
    [null, "hr"],
    [1, "hr"]]
Expected was:
  [[-1, "hardware"], [-1, "hr"], [-1, "hr"]]

@jinchengchenghh jinchengchenghh marked this pull request as ready for review February 24, 2026 12:48
@jinchengchenghh
Copy link
Contributor Author

Hi, @zhouyuan , can we merge this PR first and wait @PingLiuPing to resolve the enhanced feature IBM branch?

std::make_shared<velox::connector::hive::HiveConnector>(kHiveConnectorId, hiveConf, ioExecutor_.get()));

IcebergConnectorFactory icebergFactory;
auto icebergConnector = icebergFactory.newConnector(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the IcebergConnectorFactory is reverted, this PR cannot compile successfully CC @PingLiuPing

@zhouyuan
Copy link
Member

Cc: @infvg

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants