DM-26140: Centralize Gen 3 pipeline configuration info for ap_verify datasets

[kfindeisen]

This PR adds instrument specializations of the ApVerify and ApVerifyWithFakes pipelines, and moves many of the previous pipeline overrides (in particular, the assumption of deep coadds) into individual datasets. It also modifies the behavior of the ap_verify.py --pipeline argument to more naturally support dataset-specific pipelines.

[mrawls]

As best I can tell these pipelines work how I'd expect, but I'm a bit lost in some of the details of the instrument-specific fakes pipelines. Please clarify, and make everything two-spaced indents while you're at it (sorry).

[mrawls]

I do think this needs to be here, but it is odd to me that it's not in the prepareFakes subset. Maybe a comment explaining?

[kfindeisen]

Just to make sure, is "this" just the visitFakes line? That probably is in the wrong place, based on me misunderstanding what that task is for.

[natelust]

I dont really know what each of these tasks is used for, but as a note, it is possible to include a label in more than one subset, they do not have to be mutually exclusive. In DRP there is processCcd, but also Single frame that includes even more. These are just helpful organizers for people to help run or refer to collections of tasks.

[kfindeisen]

I actually did mean to create a partition, because that's the organization that makes the most sense to me.

And on third(?) thought, I think visitFakes belongs in apPipeWithFakes (because it runs characterizeImage and calibrate on the data) but not in prepareFakes (because it requires partially processed raws whereas the other tasks can be done without them). Not ready to push yet, but here's what I have so far:

  apPipe:
    subset:
      - isr
      - characterizeImage
      - calibrate
      - imageDifferenceNoFakes
    description: >
      The AP pipeline without fakes. Only includes processing through
      image differencing.
  prepareFakes:
    subset:
      - createFakes
      - coaddFakes
    description: >
      Creation of fake sources.
  apPipeWithFakes:
    subset:
      - visitFakes    # characterizeImage and calibrate with fakes
      - imageDifference
      - transformDiaSrcCat
      - diaPipe
      - matchFakes
    description: >
      The AP pipeline with fakes. Requires apPipe and prepareFakes subsets.