Workflow execution fails on Windows: Iceberg local storage requires winutils.exe

[aglinxinyuan]

What happened?

On a Windows development machine, every workflow execution fails immediately at execution start, before any operator runs. Expected behavior: workflows run on a stock Windows checkout.

Root cause: Iceberg's HadoopFileIO (used for the local warehouse by the postgres and hadoop catalog types) writes through Hadoop's LocalFileSystem. On Windows, hadoop-common applies POSIX permissions on every file/directory creation by shelling out to %HADOOP_HOME%\bin\winutils.exe, and there is no configuration flag to disable this:

ExecutionStatsService                    (create runtime-stats Iceberg table)
 └─ IcebergUtil.createTable
     └─ JdbcTableOperations.doCommit → HadoopFileIO
         └─ RawLocalFileSystem.mkdirs / create
             └─ RawLocalFileSystem.setPermission     ← on every file/dir creation
                 └─ Shell.getSetPermissionCommand    ← requires winutils.exe chmod
                     └─ FileNotFoundException: Hadoop bin directory does not exist

Hadoop only skips the winutils path when its native library (hadoop.dll) is loadable, which also requires a native Hadoop installation. Neither ships with Texera, so a stock checkout cannot run workflows on Windows with these catalog types. POSIX permission bits carry no meaning on NTFS, so the chmod-on-create can be safely skipped there instead of failing the execution.

Affected storage paths	workflow results, runtime statistics, console messages (all Iceberg local-warehouse writes)
Catalog types affected	postgres, hadoop
Not affected	rest catalog (stock default since #6049, writes through S3FileIO), Linux/macOS hosts
Hadoop version	hadoop-common 3.5.0 (RawLocalFileSystem.setPermission)
Regression window	worked on hadoop-common 3.3.1; broken since the #6201 (3.4.3) / #6227 (3.5.0) bumps — 3.4+ applies POSIX permissions on every local file/directory creation

How to reproduce?

1. On Windows, without HADOOP_HOME pointing to a native Hadoop installation (the default for a fresh checkout), configure the postgres (or hadoop) Iceberg catalog type in storage.conf.
2. Start Texera and run any workflow.
3. Execution fails immediately with the log output below.

Version/Branch

1.3.0-incubating-SNAPSHOT (main)

Commit Hash (Optional)

c882625

What browsers are you seeing the problem on?

No response

Relevant log output

[ERROR] [org.apache.texera.web.service.WorkflowService] [dw-100] - error during execution
java.lang.RuntimeException: java.io.FileNotFoundException: Hadoop bin directory does not exist: C:\Users\...\Coursier\cache\v1\https\repo1.maven.org\maven2\org\apache\hadoop\bin -see https://wiki.apache.org/hadoop/WindowsProblems
	at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:735)
	at org.apache.hadoop.util.Shell.getSetPermissionCommand(Shell.java:270)
	at org.apache.hadoop.util.Shell.getSetPermissionCommand(Shell.java:286)
	at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:978)
	at org.apache.hadoop.fs.RawLocalFileSystem.mkOneDirWithMode(RawLocalFileSystem.java:660)
	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirsWithOptionalPermission(RawLocalFileSystem.java:700)
	at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:672)
	at org.apache.hadoop.fs.ChecksumFileSystem.mkdirs(ChecksumFileSystem.java:788)
	at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:513)
	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1195)
	at org.apache.iceberg.hadoop.HadoopOutputFile.createOrOverwrite(HadoopOutputFile.java:85)
	at org.apache.iceberg.TableMetadataParser.internalWrite(TableMetadataParser.java:125)
	at org.apache.iceberg.TableMetadataParser.overwrite(TableMetadataParser.java:115)
	at org.apache.iceberg.BaseMetastoreTableOperations.writeNewMetadata(BaseMetastoreTableOperations.java:160)
	at org.apache.iceberg.jdbc.JdbcTableOperations.doCommit(JdbcTableOperations.java:106)
	at org.apache.iceberg.BaseMetastoreTableOperations.commit(BaseMetastoreTableOperations.java:125)
	at org.apache.iceberg.BaseMetastoreCatalog$BaseMetastoreCatalogTableBuilder.create(BaseMetastoreCatalog.java:201)
	at org.apache.iceberg.catalog.Catalog.createTable(Catalog.java:75)
	at org.apache.texera.amber.util.IcebergUtil$.createTable(IcebergUtil.scala:202)
	at org.apache.texera.amber.core.storage.DocumentFactory$.createDocument(DocumentFactory.scala:86)
	at org.apache.texera.web.service.ExecutionStatsService.<init>(ExecutionStatsService.scala:84)
	at org.apache.texera.web.service.WorkflowExecutionService.executeWorkflow(WorkflowExecutionService.scala:160)
	at org.apache.texera.web.service.WorkflowService.initExecutionService(WorkflowService.scala:286)
	at org.apache.texera.web.resource.WorkflowWebsocketResource.myOnMsg(WorkflowWebsocketResource.scala:118)