Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DM-27767: Add special raw file template #445

Merged
merged 7 commits into from
Jan 8, 2021
Merged

DM-27767: Add special raw file template #445

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
0dbaf3a
Add visit and exposure day_obs to file hierarchy
timj Jan 7, 2021
2fc6ede
Remove label from file template since it is not a dimension
timj Jan 7, 2021
082c205
Change detector and exposure defaults in template
timj Jan 7, 2021
0a3740c
Move day_obs earlier in hierarchy above filter
timj Jan 7, 2021
ac1457b
Switch to obs_id in directory path
timj Jan 7, 2021
6090220
Add special raw-like file template
timj Jan 7, 2021
c235503
Move instrument to the front of default file template
timj Jan 8, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
5 changes: 4 additions & 1 deletion python/lsst/daf/butler/configs/datastores/fileDatastore.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,9 @@ datastore:
table: file_datastore_records
create: true
templates:
default: "{run:/}/{datasetType}.{component:?}/{tract:?}/{patch:?}/{label:?}/{band:?}/{subfilter:?}/{physical_filter:?}/{visit:?}/{datasetType}_{component:?}_{tract:?}_{patch:?}_{label:?}_{band:?}_{physical_filter:?}_{visit:?}_{exposure:?}_{detector:?}_{instrument:?}_{skymap:?}_{skypix:?}_{run}"
default: "{run:/}/{datasetType}.{component:?}/{tract:?}/{patch:?}/{visit.day_obs:?}/{exposure.day_obs:?}/{band:?}/{subfilter:?}/{physical_filter:?}/{visit:?}/{exposure.obs_id:?}/{datasetType}_{component:?}_{instrument:?}_{tract:?}_{patch:?}_{band:?}_{physical_filter:?}_{visit:?}_{exposure.obs_id:?}_{detector.full_name:?}_{skymap:?}_{skypix:?}_{run}"
# For raw-type files do not include band or filter in hierarchy
# and remove band from file name
physical_filter+detector+exposure: "{run:/}/{datasetType}.{component:?}/{exposure.day_obs}/{exposure.obs_id}/{datasetType}_{component:?}_{instrument:?}_{physical_filter}_{exposure.obs_id}_{detector.full_name}_{run}"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems a little strange that instrument moves around. You don't want it before band in the default template, for example?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point. I've moved instrument to the front in the default to match.

formatters: !include formatters.yaml
composites: !include composites.yaml