Produce associations for orphaned exposures

[stscieisenhamer]

Any exposure that does not get associated will be associated into an "orphaned" association.

[stscieisenhamer]

Implementation

• Give option to save orphaned exposures as a table.

[stscieisenhamer]

Issues

@hbushouse

• One association per exposure, or all exposures in a single association?
  • Resolution: Will give the option to save the orphaned table.
• Type identifier of association. Proposed: orphaned
  • Should not be an association, but will leave as a table.

[hbushouse]

One orphaned exposure per association.

I think the association type could still use entries from the existing list: image, spec, etc., depending on what type of exposure it is.

[stscieisenhamer]

@hbushouse About the type: If an exposure matched one of the types, it would be in an association already. The only time there will be "orphaned" exposures are when an exposure does not match any of the types.

[hbushouse]

Do the "types" basically correspond to a given filter rule that's used to create groups of exposures to begin with? What I was referring to is if an orphaned exposures has exp_type 'NRC_IMAGE', then it gets an association type of 'image', or if its exp_type is 'MIR_MRS' or 'NRC_GRISM' (etc.) then it gets an association type of 'spec'.

[stscieisenhamer]

RIght, types generally match the rules, but also types are to be used to determine the type of processing that should occur. So, if an exposure doesn't match a rule, it should probably not have a type that will trigger a Level3 process. Such orphaned exposures, presumably, do not get Level3 processes, so we need separate types to indicate them as such.

If an "orphaned" exposure should get some type of Level3 processing, then it should really have been in either an existing rule, or it should have a matching rule so that it does get processed.

[hbushouse]

Orphans will receive level-3 processing. They won't get combined with any other data, but they will at least go through drizzling as a singleton to produce a resampled/rectified product that has a level-3 source-based product name. The level-3 pipelines are/will be setup to correctly handle a single input exposure in this way.

[stscieisenhamer]

Currently, there is nothing in any of the rules the force an association to contain more than one exposure. So, if there is an exposure that matches a rule and it is the only exposure that matches, the generator will still generate the association. If this covers what the "every exposure gets an association" requirement, then we're done. The situations you have been describing fall into this category.

However, if there are observing modes that produces singletons, single exposures, that should never associate with other exposures, then we will need to create rules for such singleton observing modes.

So, a final question is what to do with exposures that do not match anything, nor should match anything. Currently nothing is done with these, truly orphaned exposures. Should there be an association that contains these, or should they simply be ignored?

[stscieisenhamer]

After discussion with @hbushouse, it has been determined that the situation is as described: basically allow associations with a single member to occur, which currently does.
There is not, as yet, any observing modes which require only a single exposure; that will be dealt with when the case arises.
On the issue of what to do with actual orphaned exposures: An option will be added to save the orphaned exposure table. This will not be an association, but a table, by default in ecsv format.