Skip to content

Commit

Permalink
Only read the second header if we can't determine translator from pri…
Browse files Browse the repository at this point in the history
…mary
  • Loading branch information
timj committed Mar 22, 2021
1 parent 4dde162 commit c9ecbb4
Showing 1 changed file with 19 additions and 11 deletions.
30 changes: 19 additions & 11 deletions python/lsst/obs/base/ingest.py
Original file line number Diff line number Diff line change
Expand Up @@ -357,22 +357,30 @@ def extractMetadata(self, filename: ButlerURI) -> RawFileData:
sidecar_fail_msg = " (via sidecar)"
else:
# Read the metadata from the data file itself.
# Manually merge the primary and "first data" headers here
# because we do not know in general if an input file has
# set INHERIT=T.

# For remote files download the entire file to get the
# header. This is very inefficient and it would be better
# to have some way of knowing where in the file the headers
# are and to only download those parts of the file.
with filename.as_local() as local_file:
phdu = readMetadata(local_file.ospath, 0)
header = merge_headers([phdu, readMetadata(local_file.ospath)], mode="overwrite")

# Try to work out a translator class.
translator_class = MetadataTranslator.determine_translator(header, filename=filename)

# Obtain additional headers if needed
headers = translator_class.read_all_headers(filename.ospath, header)
# Read the primary. This might be sufficient.
header = readMetadata(local_file.ospath, 0)

try:
# Try to work out a translator class early.
translator_class = MetadataTranslator.determine_translator(header, filename=filename)
except ValueError:
# Primary header was not sufficient (maybe this file
# has been compressed or is a MEF with minimal
# primary). Read second header and merge with primary.
header = merge_headers([header, readMetadata(local_file.ospath, 1)], mode="overwrite")

# Try again to work out a translator class, letting this
# fail.
translator_class = MetadataTranslator.determine_translator(header, filename=filename)

# Obtain additional headers if needed
headers = translator_class.read_all_headers(filename.ospath, header)

# Add each header to the dataset list
datasets = [self._calculate_dataset_info(h, filename) for h in headers]
Expand Down

0 comments on commit c9ecbb4

Please sign in to comment.