DM-23231: Change everywhere to use expId rather than visit+snap #163
Conversation
| @@ -162,7 +162,8 @@ def testDetectorName(self): | |||
| name = self.mapper._extractDetectorName({'detector': 29}) | |||
| self.assertEqual(name, "R10_S02") | |||
|
|
|||
| name = self.mapper._extractDetectorName({'visit': 3019031900001}) | |||
| name = self.mapper._extractDetectorName({'obsid': "MC_C_20190319_000001", | |||
| 'lsstSerial': "ITL-3800C-041"}) | |||
There was a problem hiding this comment.
Maybe replace one of the above with expId?
config/ingest.py
Outdated
| } | ||
| config.register.unique = ["visit", "detector"] | ||
| config.register.visit = ['visit', 'filter', 'dateObs', 'expTime'] | ||
| config.register.unique = ["obsid", "detector"] |
There was a problem hiding this comment.
In all of these, this should likely be expId, not obsid, as the former is more likely to be looked up by a user and hence would benefit from the index.
config/ingest.py
Outdated
| config.register.unique = ["visit", "detector"] | ||
| config.register.visit = ['visit', 'filter', 'dateObs', 'expTime'] | ||
| config.register.unique = ["obsid", "detector"] | ||
| config.register.visit = ['visit', 'filter', 'obsid', 'dateObs', 'expTime'] |
There was a problem hiding this comment.
Since this table is one-row-per-visit, it should not in general contain obsid or expId, which are only appropriate when visits are always one exposure.
There was a problem hiding this comment.
ingest requires that there is a key in unique that is also in visit otherwise it fails. I think I remember now that's why I added obsid to visit.
There was a problem hiding this comment.
I think this means I need to keep visit in unique along with expId.
There was a problem hiding this comment.
Adding visit to unique is better than adding obsid to visit, especially with obsid in unique (which I think will lead to failures). The most frequent lookup should be first in the unique list, so expid, detector, visit is preferable.
examples/reingest.sh
Outdated
| export DYLD_LIBRARY_PATH=${LSST_LIBRARY_PATH} | ||
|
|
||
| set -e | ||
| DATADIR=${OBS_LSST_DIR}/data/input |
There was a problem hiding this comment.
Could possibly be made into a more generic tool for use by others (including production Butler repos, which would need more safety than this git repo), and I'm not sure examples is the best place for it, but OK.
timj
force-pushed
the
tickets/DM-23231
branch
2 times, most recently
from
1a77150
to
004901b
Compare
expId is unique on its own in LSST cameras. visit is now derived from the groupId and we do not currently have access to any snap information.
timj
force-pushed
the
tickets/DM-23231
branch
from
004901b
to
06ba5cb
Compare
expId is unique on its own in LSST cameras. visit is now
derived from the groupId and we do not currently have access
to any snap information.