Snowflake on Azure Open Catalog does not work

[ryoma-nagata]

Search before asking

• I had searched in the issues and found no similar issues.

Please describe the bug 🐞

I ran into an issue while using Snowflake's (Azure) polaris catalog. Documenting here.

java -jar xtable-utilities/target/xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar --datasetConfig ./demo/config.yml -i ./demo/catalog.yaml 

Error

2025-03-17 16:41:17 INFO  org.apache.iceberg.metrics.LoggingMetricsReporter:38 - Received metrics report: ScanReport{tableName=azure_ext.ICEBIRG_DEMO.PUBLIC.TEST_TABLE_MANAGED_ST, snapshotId=8300558301514482769, filter=true, schemaId=0, projectedFieldIds=[1], projectedFieldNames=[COL1], scanMetrics=ScanMetricsResult{totalPlanningDuration=TimerResult{timeUnit=NANOSECONDS, totalDuration=PT0.547000254S, count=1}, resultDataFiles=CounterResult{unit=COUNT, value=1}, resultDeleteFiles=CounterResult{unit=COUNT, value=0}, totalDataManifests=CounterResult{unit=COUNT, value=1}, totalDeleteManifests=CounterResult{unit=COUNT, value=0}, scannedDataManifests=CounterResult{unit=COUNT, value=1}, skippedDataManifests=CounterResult{unit=COUNT, value=0}, totalFileSizeInBytes=CounterResult{unit=BYTES, value=512}, totalDeleteFileSizeInBytes=CounterResult{unit=BYTES, value=0}, skippedDataFiles=CounterResult{unit=COUNT, value=0}, skippedDeleteFiles=CounterResult{unit=COUNT, value=0}, scannedDeleteManifests=CounterResult{unit=COUNT, value=0}, skippedDeleteManifests=CounterResult{unit=COUNT, value=0}, indexedDeleteFiles=CounterResult{unit=COUNT, value=0}, equalityDeleteFiles=CounterResult{unit=COUNT, value=0}, positionalDeleteFiles=CounterResult{unit=COUNT, value=0}}, metadata={iceberg-version=Apache Iceberg 1.4.2 (commit f6bb9173b13424d77e7ad8439b5ef9627e530cb2)}}
2025-03-17 16:41:17 ERROR org.apache.xtable.utilities.RunSync:239 - Error running sync for abfss:///my-container@myaccount.dfs.core.windows.net/hub/public/test_table_managed_st
java.lang.NullPointerException: null
        at org.apache.xtable.iceberg.IcebergColumnStatsConverter.lambda$fromIceberg$2(IcebergColumnStatsConverter.java:106) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT]
        at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) ~[?:?]
        at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:177) ~[?:?]
        at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) ~[?:?]
        at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) ~[?:?]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) ~[?:?]
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:913) ~[?:?]
        at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:?]
        at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:578) ~[?:?]
        at org.apache.xtable.iceberg.IcebergColumnStatsConverter.fromIceberg(IcebergColumnStatsConverter.java:119) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT]
        at org.apache.xtable.iceberg.IcebergDataFileExtractor.fromIceberg(IcebergDataFileExtractor.java:64) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT]
        at org.apache.xtable.iceberg.IcebergDataFileExtractor.fromIceberg(IcebergDataFileExtractor.java:53) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT]
        at org.apache.xtable.iceberg.IcebergConversionSource.fromIceberg(IcebergConversionSource.java:176) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT]
        at org.apache.xtable.iceberg.IcebergConversionSource.getCurrentSnapshot(IcebergConversionSource.java:156) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT]
        at org.apache.xtable.spi.extractor.ExtractFromSource.extractSnapshot(ExtractFromSource.java:38) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT]
        at org.apache.xtable.conversion.ConversionController.syncSnapshot(ConversionController.java:282) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT]
        at org.apache.xtable.conversion.ConversionController.syncTableFormats(ConversionController.java:202) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT]
        at org.apache.xtable.conversion.ConversionController.sync(ConversionController.java:105) ~[xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT]
        at org.apache.xtable.utilities.RunSync.runSync(RunSync.java:237) [xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT]
        at org.apache.xtable.utilities.RunSync.main(RunSync.java:160) [xtable-utilities_2.12-0.2.0-SNAPSHOT-bundled.jar:0.2.0-SNAPSHOT]

config.yaml

targetFormats:
  - DELTA
datasets:
  -
    tableBasePath: abfss://my-container@myaccount.dfs.core.windows.net/hub/public/test_table_managed_st
    tableDataPath: abfss://my-container@myaccount.dfs.core.windows.net/hub/public/test_table_managed_st/data # replace this with gs://path/to/delta-dataset/people if your data is in GCS. 
    tableName: TEST_TABLE_MANAGED_ST
    namespace: ICEBIRG_DEMO.PUBLIC

catalog.yaml

catalogImpl: org.apache.iceberg.rest.RESTCatalog
catalogName: azure_ext
catalogOptions:

  warehouse: azure_ext
  uri: https://<id>.snowflakecomputing.com/polaris/api/catalog
  credential: <>:<>
  header.X-Iceberg-Access-Delegation: vended-credentials
  scope: PRINCIPAL_ROLE:ALL
  client.region: japan-east.azure

​This error occurs with public access to the blob enabled, so I don't believe it's related to credential vending.

Are you willing to submit PR?

• I am willing to submit a PR!
• I am willing to submit a PR but need help getting started!

Code of Conduct

• I agree to follow this project's Code of Conduct