DM-31698: Move ConvertReferenceCatalog classes out of ingestIndex file #297
Conversation
The useful methods in "ingestIndex" now live in convertReferenceCatalog.py, which is where they are used. "ingest" in gen3 is an entirely separate process. As part of this, I removed the convert base class, because it was only there for gen2/gen3 compatibility. I refactored the interface as part of that, to remove methods that only were split to modularize the gen2/gen3 differences.
This was ticketed for DM-31817, but it makes sense to do it here. Have to add that field back to the `bad_config.py` test, as it is still a necessary field (so that the name of the refcat is tracked in something other than the path it is written to).
This should make it more clear why these tests both matter: a naive coverage comparison might suggest one could be removed.
Having only one file for convert tests, and one file for load tests (plus the nopytest file for the parallel convert tests) should help reduce confusion about what the various tests are for.
| ) | ||
| ref_dataset_name = pexConfig.Field( | ||
| dtype=str, | ||
| doc="Name of this reference catalog; this should match the name used during butler ingest.", |
There was a problem hiding this comment.
Is this an optional or required field? Do the default config settings make sure this is required with a default of None so it must be set?
There was a problem hiding this comment.
This is required: we may want to revamp this at some point in the future, but other than this field, the only thing tracking the refcat name is the directory it's written to (and then the butler datasetType it's registered as, once it's added to a butler). I'd much rather have the name the author expected written to a config file.
ConvertReferenceCatalogTask is a Task, so it will validate the config on init (thanks, DM-971!), including when this is run on the commandline.
There was a problem hiding this comment.
Though your comment made me check on, and clean up, a couple validate()-related things.
Post-DM-971, Tasks always validate their configs on init, so we don't need to validate when running from the commandline via main().
No description provided.