DM-16333: Add Job metadata to the ap job.

[SimonKrughoff]

@kfindeisen and @jhoblitt I think we could do something like this to get the appropriate metadata in the job. Does this look reasonable, and how should I test?

[kfindeisen]

When I first wrote DM-11321 I'd intended for job-level metadata to be the responsibility of the ap.verify.metrics.AutoJob class. However, that entire module is being deprecated in DM-16017, so a temporary hack in pipeline_driver is fine.

However, I'm a bit concerned about how to translate this code into the DM-16017 framework. The camera should be easy enough, but we already have one metric (ap_association.totalUnassociatedDiaObjects) that applies to the entire dataset, and will therefore have an empty data ID. Should it have filter_name = None?

[SimonKrughoff]

I think in general, we will want to associate the data ids involved in measuring a metric with the metric. That's why I haven't considered including the filter name in the metadata as onerous when it makes sense.

[SimonKrughoff]

OK. I'll make the updates and try to test it out. @jhoblitt should we set up a test env to test the branch?

[SimonKrughoff]

@kfindeisen Have a look at this updated PR. I've taken into account your comments. Note that we no longer select specific metadata from the dataId, but ship the whole thing assuming it will be useful. Also note that I also had to fix some linting errors. I rebased yesterday, but some of these may still have been fixed. I'll make sure to rebase on master before merging.