DM-18577: Support butler relocation #141
Conversation
There was a problem hiding this comment.
A few minor (mostly clarification) comments. My only big-picture concern (which is also a minor one) is that - if I've understood correctly - sometimes the root in a Config instance is expanded, and sometimes it isn't, and that means knowing what to expect in one depends on some state that's hard to track. If it's not too hard, I'd prefer for there to be a different attribute (maybe in the Config, maybe in the things that use one) that's always expanded so the one in the Config can always be unexpanded. But if there's no good place to put that attribute, I could certainly believe that suggestion isn't worth the cost.
Mind updating the commit messages for the two YAML commits to make it clear that the unsafe loading is in gen2convert? Right now, looking at just the messages, one would get the impression that one commit reverts the other.
|
@TallJimbo I don't understand your main comment about being uncertain about whether something will be expanded or not. If you don't use |
We have no control over the formatter yaml output and to read gen2 repository configuration YAML we also need to use an unsafe loader.
Use "<root>" in a configuration file to specify that you are relative to the butler root directory.
Some of the tests were loading the butler.yaml file directly which triggered POSIXDatastore instance to create a new directory in tests/configs/basic (where the butler.yaml file was located). Previously these had gone into the daf_butler root directory. Now modify the tests to always use a temp directory.
This allows the butler configuration to be in one place but the datastore and registry to be in a different directory tree.
…erConfig Seeding one from another means they do have the same original configuration directory.
This ensures that the butler resources are freed and without causing difficulties with registry disconnection.
with Butler() as butler:
# Do something
Whilst it seemed like a good idea to put all the datastore files in a standalone subdirectory, this does confuse ci_hsc a bit so for now leave it out. Can revisit in the future.
Decide against using '{root}' because the curly brace is
special syntax in YAML and would require that the root be
quoted every time.
This change cleans up a test so that the root tag is
retrieved from code and cleans up the documentation
so that the root tag is documented in sphinx without
hard coding the value of the special tag into a
docstring.
Use `noqa: Nxxx` syntax to hide specific codes.
butler.yamlfile.