Things to save from protopipe before the full-refactoring

[HealthyPear]

We are near to a complete refactoring of the pipeline based on ctapipe, but if we did it now some things would disappear and that is not good (both from the point-of-view of comparisons with historical pipelines and for any future study).

These features can and should be generalized to account for more use-cases than the ones taken into account currently by protopipe.

This issue is a summary list: I will add new things as I stumble upon them and each related PR should be linked to the respective feature.

• Image extraction status (Extractor status flag #1810)

This is the equivalent of any other QualityQuery-based class we use (in ImageProcessor and ShowerProcessor) but for image extractors.
It is needed by extractors like TwoPassWindowSum (CTAMARS-based analysis), for which it is fundamental to label some events (in this case images, not showers) which do not survive a step during the total image extraction process (in this case a preliminary image cleaning cut in between the two passes).

This will require to modify the ImageExtractor class to output a quality query together with charges and peak times.
Of course, for single-pass image extractors, this condition will should be always positive by default and thus it would have no visible effect.

• Custom trigger conditions

In the specific case of protopipe I have implemented the LST-stereo trigger condition, required for Prod3b simulations in which this requirement was not simulated

• calibscale (Add support for calibration scale and shift in R1 #1749)

This should be a simple scale factor for the reconstructed charge extracted from each pixel.
In particular, as reported here,

the factor to transform the integrated charges (in ADC counts) into number of photoelectrons is the pixel-wise one calculated by simtelarray (0.92*LasCalData->calib[][][]).

• Shower core position in TiltedFrame (Compute and record shower's core position also in the TiltedFrame when using HillasReconstructor #1745)

This allows the pipeline to read from DL2 files this quantity directly in the right frame were telescope-wise impact distances should be computed

• Wavelet cleaning

It has not been used in a long time and it should be added as an ImageCleaner, perhaps from a module itself.

• DL2 event estimation

This is something that should be added after PR #1726.

There should a function that takes as input

• what in protopipe I call TRAINING data (aka DL1b + shower geometry)

• information from the configuration of the model building process, in particular which features have been used

• the possibility to eval at runtime features that do not come directly from the TRAINING file, but that are functions of those (e.g. np.log10(hillas_intensity))

• some configuration options to compute the final estimated quantity (weighted average and if yes with what weight, e.g. standard deviation of the predictions, some parametrized function of TRAINING variables, etc...)

• Shower Impact Parameter(s)

Related issues:

• Store telescope impact distance in simulation container #1892
  Related Pull Requests:
• compute impact distance to shower axis #1899

#1745 will allow to compute it directly in the right frame, but for now it is something that is not done by ctapipe's ShowerProcessor.
From my side, I think wrapping the first step of the pipeline around ctapipe-process just to open the files - calculate impact_dist - close file - use ctapipe ML module (or protopipe.mva) is a bit uncomfortable...
This is of course because we need to have impact parameter in our files for benchmarking, otherwise it would be sufficient to compute it directly in the ML module.
Alternatively, the benchmarking code would compute it, but it seems a bit ugly to me.
Should we do allow this operation from ctapipe-process? in the end we already have 1 DL2 telescope-wise variable written to file (core_psi)....

[maxnoe]

Trigger conditions in #2136

[maxnoe]

I'll close this, the only thing remaining from the list is the wavelet cleaning and this is I think not strictly required and also works very differently to the currently implemented cleaning methods, which all do not modify the image but select pixels.