Adding the Ecal Endcaps to the ML based online ECAL DQM [13_0_X]

[abhih1]

PR description:

This PR introduces the Ecal Endcaps into the autoencoder-based online ECAL DQM feature, which was implemented for EB in #35990.
Separate Autoencoder (AE) models with ResNet architecture are trained for EE+ and EE-, apart from the model for EB, on certified good data (digi occupancy) from 2018 runs.

On giving an input occupancy map to the AE, the encoder part of the AE encodes and learns the features and the decoder reconstructs the data from the encoded latent space to match the input as closely as possible. The reconstruction loss is then calculated, which is a mean squared error (MSE) between the input and output images at a tower level. Thus given an anomalous tower, the AE which has learnt the features of the good data will have a hard time reconstructing it and give a higher loss on the anomaly than on the good towers. A quality threshold is then applied on this loss map which marks it as Good or Bad, which is then stored as an ML quality summary plot.
New correction factors are derived from 2022 collisions data to use in the pre-processing and inference, which follows the same steps as used for EB.

This PR thus introduces ML Quality summary plots for EE- and EE+, along with Loss Map and reconstructed occupancy maps from the AE.
It also introduces a trend plot to monitor the no. of bad towers flagged by the AE per lumisection in a run, as well as the map of these bad towers in an occupancy-like plot. This would be very helpful in monitoring per lumisection behaviour of bad towers/channels.

Please note that this PR should be tested along with the files added to cms-data/DQM-EcalMonitorClient#3

PR validation:

The code was validated by running the online Ecal DQM configuration and the resultant plots were examined by uploading the output file to a DQM test gui.
The new plots are confirmed and look reasonable.

If this PR is a backport please specify the original PR and why you need to backport that PR. If this PR will be backported please specify to which release cycle the backport is meant for:

This is a backport to 13_0_X currently used in production.
The master PR is: #41175

[cmsbuild]

A new Pull Request was created by @abhih1 (Abhirami Harilal) for CMSSW_13_0_X.

It involves the following packages:

• DQM/EcalMonitorClient (dqm)
• DQM/EcalMonitorTasks (dqm)

@emanueleusai, @cmsbuild, @syuvivida, @rvenditti, @micsucmed, @pmandrik can you please review it and eventually sign? Thanks.
@rchatter, @simonepigazzini, @thomreis, @argiro this is something you requested to watch as well.
@perrotta, @dpiparo, @rappoccio you are the release manager for this.

cms-bot commands are listed here