DM-19470: Create jointcal PipelineTask (version 0: tests) #165
Conversation
Replace inspect.stack() with self.id() for name of active test
parejkoj
force-pushed
the
tickets/DM-19470
branch
2 times, most recently
from
b331d96
to
8afdd55
Compare
| class JointcalTask(pipeBase.CmdLineTask): | ||
| """Jointly astrometrically and photometrically calibrate a group of images.""" | ||
| @dataclasses.dataclass | ||
| class JointcalInputData: |
There was a problem hiding this comment.
I see where you're going here with making the common dataclass for gen2 and gen3 to normalize things. However, I'm not sure if this will actually work (which is maybe part of the point of the new ticket?) because (a) the gen3 version (at least!) should be using the parquet files; (b) by reading all the catalogs first and not cutting them down as you read them this leads to significant memory overhead. But having a common dataclass in general does make sense.
There was a problem hiding this comment.
a) We'll see how this looks when I write the Parquet version. This class may not be as helpful then.
b) The gen2 code is still only reading one catalog/image at a time, so no memory pressure there. The dataclass is just to help separate the "read input" code (which is different in gen2/gen3) from the "create internal jointcal representation" code (which is middleware agnostic).
|
While looking through your comments, I noticed that I'd not written docstrings for the new test methods. I've added those and some cleanups on one commit. @erykoff : mind taking a quick look at the new commit? You might want to borrow some of it for your own fgcmcal test helpers. |
| transfer='symlink', | ||
| skip_dimensions={'instrument', 'detector', 'physical_filter'}) | ||
|
|
||
| def _runPipeline(self, repo, queryString=None, |
There was a problem hiding this comment.
I'm not sure why you don't want to use pipeline files. It's how the system is supposed to be run (though I suppose in a test it can be simulated, that's swimming upstream).
There was a problem hiding this comment.
Ok, I'll add it back in on DM-27869, where I can actually try it out.
This test is skipped for now, as the gen3 jointcal code is not finished.
Refactor gen2 data reading
parejkoj
force-pushed
the
tickets/DM-19470
branch
from
c9c15aa
to
e9555bc
Compare
No description provided.