Core: find first existent previous metadata need FIFO or LIFO of ReachableFileUtil class

[felixYyu]

in BaseDeleteOrphanFilesSparkAction to buildValidMetadataFileDF->BaseSparkAction.buildOtherMetadataFileDF->ReachableFileUtil.metadataFileLocations, the first existent previous metadata need FIFO or LIFO?

public static Set<String> metadataFileLocations(Table table, boolean recursive) {
    Set<String> metadataFileLocations = Sets.newHashSet();
    TableOperations ops = ((HasTableOperations) table).operations();
    TableMetadata tableMetadata = ops.current();
    metadataFileLocations.add(tableMetadata.metadataFileLocation());
    metadataFileLocations(tableMetadata, metadataFileLocations, ops.io(), recursive);
    return metadataFileLocations;
  }

  private static void metadataFileLocations(TableMetadata metadata, Set<String> metadataFileLocations,
                                            FileIO io, boolean recursive) {
    List<MetadataLogEntry> metadataLogEntries = metadata.previousFiles();
    if (metadataLogEntries.size() > 0) {
      for (MetadataLogEntry metadataLogEntry : metadataLogEntries) {
        metadataFileLocations.add(metadataLogEntry.file());
      }
      if (recursive) {
        TableMetadata previousMetadata = findFirstExistentPreviousMetadata(metadataLogEntries, io);
        if (previousMetadata != null) {
          metadataFileLocations(previousMetadata, metadataFileLocations, io, recursive);
        }
      }
    }
  }

  private static TableMetadata findFirstExistentPreviousMetadata(List<MetadataLogEntry> metadataLogEntries, FileIO io) {
    TableMetadata metadata = null;
    for (MetadataLogEntry metadataLogEntry : metadataLogEntries) {
      try {
        metadata = TableMetadataParser.read(io, metadataLogEntry.file());
        break;
      } catch (Exception e) {
        LOG.error("Failed to load {}", metadataLogEntry, e);
      }
    }
    return metadata;
  }

[felixYyu]

cc @rdblue @RussellSpitzer

[rdblue]

I don't think I understand the question. Could you clarify what you mean?

[RussellSpitzer]

I'm also a bit confused about the question here? I believe the code in question is just trying to find as many possible file locations as can be determined from the history. From most recent to oldest, it reads the metadata files listed in the log, it picks the first one (the most recent) which is readable.

[felixYyu]

because of list is LIFO, the method of findFirstExistentPreviousMetadata's first metadata means of oldest I think，but from most recent to oldest@RussellSpitzer，I want to ask find first metadata is the oldest or the recent to oldest?

thanks @rdblue @RussellSpitzer

[RussellSpitzer]

I could have gotten it wrong, it's whatever the first element of the list is in the metadata.json.

[felixYyu]

I got it, thanks and close this issue.