Manifest List is a LinkedList Indexed during Write

[johnclara]

ManifestLists are stored as linked lists:
https://github.com/apache/iceberg/blob/master/core/src/main/java/org/apache/iceberg/ManifestLists.java#L46

But with Manifest V1/V2 split, they're getting wrapped and indexed by individual tasks:
https://github.com/apache/iceberg/blob/master/core/src/main/java/org/apache/iceberg/SnapshotProducer.java#L180

Any reason why the manifest list should stay as a linked list?

@rdblue

[rdblue]

Those two code sections aren't very related. The write is using an array because the manifests are processed in parallel. That happens to ensure that each manifest has metadata. If the ManifestFile that will be written doesn't already have metadata about its contents, then it will be scanned to generate that metadata.

On the read side, all of the ManifestFile instances in the manifest list are read into memory for later use in scan planning and other operations. I don't see a strong need for those to be in a linked list, but I don't think it would be a problem either.

[johnclara]

Not worth changing

[johnclara]

manifests.get(index) this section of line 180 in the snapshot producer was the indexing part in the linked list, not the filling of the array.

[rdblue]

Oh, I see. We could change that to an ArrayList. Maybe that would be faster?