DM-43472: Prompt Processing version 2 not compatible with Butler dimensions-config 6

[kfindeisen]

This PR adds a day_obs dimension to the fix merged on #143; for some reason the tests with that PR passed without it.

[kfindeisen]

I suspect the day_obs bug only appears once per day and that the table rows can come from a source other than PP, so I will confirm tomorrow morning before merging.

[timj]

It should be possible to do a more general fix that automatically finds dependent records and includes them so you would only need to list the dimensions you really want and then other dimensions would be found. We sort of do something similar in the Butler.transfer_dimension_records_from internals.

[kfindeisen]

I thought that was what populated_by was for? But neither group nor day_obs use it.

(And we specifically don't want to push all dependent records, because of deadlock problems; otherwise we could just use transfer_from(transfer_dimensions=True).)

[timj]

day_obs is not populated by anyone as such (whereas visit_detector_region is) but you can ask exposure which dimensions it uses itself and so if you have exposure in the list you can automatically know that group/day_obs/physical_filter/instrument will be needed. There is also a sorting API that lets you know which order you need to insert dimension records such that the dependencies are met.

[kfindeisen]

Right, instrument was one of the ones that deadlocked before we implemented it this way.

[timj]

Yes, but you can add an exclusion list for physical_filter/instrument/detector that have to be present via instrument registration.