Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DM-17025: Template handling enhancements #134

Merged
merged 17 commits into from Feb 28, 2019
Merged

DM-17025: Template handling enhancements #134

merged 17 commits into from Feb 28, 2019

Conversation

timj
Copy link
Member

@timj timj commented Feb 20, 2019

  • Add support for instrument overrides in formatter and template selection.

@timj
Add some dunder methods to FileTemplate
c5e1545 
Makes it easier to write unit tests comparing to templates.
@timj
Support instrument overrides in formatter and template selection
990c80d 
In config files "instrument<name>" can be used to specify
an override for the named instrument to use.
@timj
Refactor name lookup for templates/formatters/composites
ad6d676 
Now wherever a config file has keys that can be StorageClass
names, DatasetType names or "instrument<xxx>" overrides,
the parsing of this is done in one place and a new LookupKey
object is created and used as the key in the mappings.

This provides the ground work for supporting Dimensions
in config files.
@timj timj marked this pull request as ready for review February 25, 2019 18:14
@timj timj requested a review from TallJimbo February 25, 2019 18:14
TallJimbo
TallJimbo approved these changes Feb 27, 2019
View reviewed changes
Copy link
Member

@TallJimbo TallJimbo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments. Overall looks great!

python/lsst/daf/butler/core/fileTemplates.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/core/formatter.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/core/configSupport.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/core/datasets.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/core/configSupport.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/core/formatter.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/core/formatter.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/core/mappingFactory.py Show resolved Hide resolved
python/lsst/daf/butler/core/mappingFactory.py Outdated Show resolved Hide resolved
python/lsst/daf/butler/core/fileTemplates.py Outdated Show resolved Hide resolved
@timj
Check that dimensions match when constructing a DatasetRef
1c6a0ef
@timj
Add dimensions support to template/formatter lookups
eb939a7 
* Dimensions are normalized after reading.
* Dimensions are added to lookup names in DatasetType
@timj
Fix variable names to be camel case
d4ee015
@timj
Add unnecessary usage of lsst.utils from tests
8664d4b 
Retains the usage in code that is using afw
@timj
Use lsst.geom rather than lsst.afw.geom
328dbec 
afw.geom has been renamed geom
@timj
Base package is required for lsstimport
93c861f
@timj
Fix test skipping when afw is missing
e60f1d9
@timj
Add ability for FileTemplate to report the field names in the template
4343fc7
@timj
Add ability to validate a template against a DatasetType
a431a4d
@timj
Do basic template validation in constructor
f421487 
* Check that fields have been provided
* Check that run/collection are included
@timj
Use single quote for string to simplify double quote escaping
8fee770
@timj
Docstring updates
ee44e32
@timj
Use modern approach to ignoring sphinx outputs
9079034
@timj
Fix some template validation exceptions
5cabb8f
@timj timj merged commit fa6888d into master Feb 28, 2019
@timj timj deleted the tickets/DM-17025 branch February 28, 2019 23:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@TallJimbo TallJimbo TallJimbo approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@timj @TallJimbo