[lake/iceberg] Cannot rebind an existing Iceberg table across Fluss clusters with different replication factors

[litiliu]

Search before asking

• I searched in the issues and found nothing similar.

Fluss version

main (development)

Please describe the bug 🐞

Problem

When a Fluss table is created with lake tiering enabled, an existing Iceberg table cannot be rebound from another Fluss cluster if the clusters have different cluster-level replication factors.

The replication factor is a Fluss storage/replication setting. It should not make the Iceberg table incompatible, because it does not change the Iceberg schema, partition spec, or Iceberg data layout.

Reproduction

1. Start Fluss cluster A with:

   default.replication.factor=1
   

2. Create a primary-key Fluss table with lake tiering enabled, for example:

   CREATE TABLE IF NOT EXISTS db.table_name (
       id BIGINT,
       lastmodifiedtime TIMESTAMP_LTZ(3),
       PRIMARY KEY (id) NOT ENFORCED
   ) WITH (
       'bucket.num' = '2',
       'table.datalake.enabled' = 'true',
       'table.datalake.freshness' = '20m',
       'table.datalake.auto-compaction' = 'true',
       'iceberg.format-version' = '3'
   );

   Do not write any data.
3. The Iceberg table is created during the Fluss table creation flow. Its properties include:

   fluss.table.replication.factor=1
   

4. Start a second Fluss cluster B that uses the same Iceberg catalog and table, but has:

   default.replication.factor=2
   

   Cluster B does not have the corresponding Fluss metadata, for example because it is a new cluster or the Fluss metadata was removed while the Iceberg table remained.
5. Execute the same CREATE TABLE IF NOT EXISTS statement on cluster B.

Actual behavior

The operation fails with an error similar to:

The table db.table_name already exists in Iceberg catalog, but the table properties are not compatible.
Existing properties: {fluss.table.replication.factor=1, ...}
New properties: {fluss.table.replication.factor=2, ...}
Please first drop the table in Iceberg catalog or use a new table name.

No data write or tiering job execution is required to reproduce the problem.

Also, IF NOT EXISTS only checks whether the Fluss metadata exists. It does not bypass compatibility checks for an existing Iceberg table when the Fluss metadata is absent.

Expected behavior

The same Fluss table definition should be able to bind to the existing Iceberg table from another Fluss cluster when the Iceberg schema, partition spec, and Iceberg-specific properties are compatible. A different Fluss cluster-level replication factor should not cause the Iceberg binding to fail.

Solution

Possible approaches:

• Do not persist cluster-local Fluss runtime properties, such as table.replication.factor, as Iceberg table properties; or
• Exclude such properties from the Iceberg compatibility check when an existing Iceberg table is rebound.

The same consideration may apply to other Fluss-only properties such as KV format version and standby-replica settings. Iceberg compatibility should focus on properties that affect the Iceberg table's schema or data layout.

Are you willing to submit a PR?

• I'm willing to submit a PR!