Skip to content

Commit

Permalink
Check for duplicate dataId/datasetType ingest requests
Browse files Browse the repository at this point in the history
This can happen if by some chance someone asks the same file
to be ingested from two different locations.
  • Loading branch information
timj committed Mar 16, 2021
1 parent 0fe3f94 commit 7f3a3d0
Showing 1 changed file with 4 additions and 0 deletions.
4 changes: 4 additions & 0 deletions python/lsst/daf/butler/_butler.py
Original file line number Diff line number Diff line change
Expand Up @@ -1433,6 +1433,10 @@ def ingest(self, *datasets: FileDataset, transfer: Optional[str] = "auto", run:
# associated with `dataset`.
resolvedRefs: List[DatasetRef] = []
for ref in dataset.refs:
if ref.dataId in groupedData[ref.datasetType]:
raise RuntimeError(f"Ingest conflict. Dataset {dataset.path} has same DataId as "
f"other ingest dataset"
f" {groupedData[ref.datasetType][ref.dataId][0].path} ({ref.dataId})")
groupedData[ref.datasetType][ref.dataId] = (dataset, resolvedRefs)

# Now we can bulk-insert into Registry for each DatasetType.
Expand Down

0 comments on commit 7f3a3d0

Please sign in to comment.