You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Once we get support for merge-append #363 operations, and also compaction in the future, it would be good to have rolling-manifest writers that open a new file once the manifest gets to a certain size (both bytes and number of entries). This makes sure that Iceberg tables have decent parallelization when doing query planning.
The text was updated successfully, but these errors were encountered:
Hi, I would like to give this a shot if no one else is currently working on this:)
I see the java implementation has a RollingManifestWriter so I'd start there.
Feature Request / Improvement
Once we get support for merge-append #363 operations, and also compaction in the future, it would be good to have rolling-manifest writers that open a new file once the manifest gets to a certain size (both bytes and number of entries). This makes sure that Iceberg tables have decent parallelization when doing query planning.
The text was updated successfully, but these errors were encountered: