Deconvolution and OphitFinder #27

maridelg · 2023-04-05T07:33:27Z

The PR for the new Deconvolution module and the new version of OpHitFinder where the recob::OpWaveform object has been included.

duneopdet/OpticalDetector/Deconvolution/DecoAnalysis_module.cc

lpaulucc

Hello, I have included a few comments throughout the files.

duneopdet/OpticalDetector/Deconvolution/Deconvolution.fcl

duneopdet/OpticalDetector/Deconvolution/Deconvolution_module.cc

lpaulucc · 2023-04-10T11:59:15Z

duneopdet/OpticalDetector/Deconvolution/Deconvolution_module.cc

+        out_recob_float.resize(OriginalWaveformSize);
+      }
+
+


Can you add a brief explanation of what the letters mean?

lpaulucc · 2023-04-10T12:04:16Z

duneopdet/OpticalDetector/Deconvolution/Deconvolution_module.cc

+      double  fLineNoiseRMS;                    //!< Pedestal RMS in ADC counts
+      size_t fPreTrigger;                       //!< In ticks
+      std::vector<double> fSinglePEWaveform;    //!< Template for a single PE in ADC
+      double fSinglePEAmplitude;                //!< single PE amplitude


This will only take the fhicl parameter if no template file is provided, right? If so, maybe include this in the comment.

fSinglePEWaveform is a vector that stores the single PE template, this is needed to calculate the spe response spectral density. SinglePEAmplitude is a vector that stores the maximum amplitude of the single PE, we use this for found the maximum amplitude of the waveform in terms of PE.
I've relocated the definition of the vector in the module to avoid confusion and included this description.

duneopdet/OpticalDetector/OpHitFinder/OpHitAlg_deco.cxx

duneopdet/OpticalDetector/OpHitFinder/OpHitFinderDeco.fcl

jroto

Thank you, Maritza, this is fantastic work! I would say that some of these modules would be worth having in the general LArSoft repositories. Since this new OpHitFinder can run also with old raw::OpDetWaveforms, I would update the old OpHitFinder with this new one. I have added a few comments mainly to better describe the modules and how to use them.

duneopdet/OpticalDetector/fbk_deco.txt

duneopdet/OpticalDetector/OpHitFinder/OpHitFinderDeco.fcl

duneopdet/OpticalDetector/OpHitFinder/OpHitFindAna.fcl

duneopdet/OpticalDetector/OpHitFinder/OpHitFinder_deco_run.fcl

jroto · 2023-04-11T16:15:56Z

duneopdet/OpticalDetector/OpHitFinder/OpHitFindAna_module.cc

Why is this module needed? cannot OpFlashAna_module be used instead?

This analyzer module is based on opflashana but does not include an analysis of opflash, so far we have analyzed ophits.

In case the user is interested in the analysis of ophits, OphitAna or OpFlashAna could be adapted. Should we consider removing OpHitFindAna_module and its fchl from duneopdet?

We talked offline, and @maridelg saw that OpHitAna depends on OpDigiProperties, which we deleted, and OpFlashAna requires the existence of flashes. However, I think that just moving the flash-related code inside an if statement in OpFlashAna would do the work:
if(fMakeFlashHitMatchTree || fMakeFlashBreakdownTree || fMakePerFlashTree || fMakePerEventFlashTree || fMakePerFlashHists || fMakeFlashPosHist || fMakeFlashTimeHist)
And we would avoid be maintaining replicated code in the future.
Do you see any problem with that, @maridelg?

jroto · 2023-04-11T16:39:22Z

duneopdet/OpticalDetector/Deconvolution/DecoAnalysis_module.cc

+      int channel = decowaveform.Channel();
+
+      // Implement different end time for waveforms of variable length
+      double startTime = waveform.TimeStamp() - firstWaveformTime;


I see you are reading the raw::Waveforms just to take the TimeStamp... I see in the class definition that "The time is expected to be the same as for the raw::OpDetWaveform that originates it". However, in Raw::OpDetWaveform, the time is TimeStamp_t, while in recob::OpWaveform, the time is raw::RDTimeStamp. Is this the reason of the problem? Maybe we can ping Tingjun to ask why the timestamps have a different format.

Exactly, this is the problem.. The OpWaveform recob is designed based on recob::Wire (same structure and parameters), it was created for the TPC. According to Tom Junk, the RDTimeStamp was created as a plugin to raw::RawDigit to have an absolute timestamp channel-by-channel. I think we will adapt recob OpWaveform more to the needs of the optical detector.

At the end, what happened with this? Is recob::OpWaveform being adapted? I see the code stills reads raw::OpDetWaveform just to take the TimeStamp doubling the memory consumption in DecoAnalysis but also in the HitFinder.

duneopdet/OpticalDetector/Deconvolution/Deconvolution.fcl

duneopdet/OpticalDetector/OpHitFinder/OpHitAlg_deco.h

duneopdet/OpticalDetector/OpHitFinder/OpHitFindAna_module.cc

mjdelgadog · 2023-04-14T12:28:32Z

Hi Laura and Jose,
Thanks a lot for your timely comments. We are already working on this. I answer some of your questions inline and in the next few days I will upload the modules with the corrections.
Thank you

lpaulucc · 2023-05-02T13:42:37Z

duneopdet/OpticalDetector/OpHitFinder/OpHitFinderDeco.fcl

@@ -24,4 +24,12 @@ standard_ophit_finder_spe:

 }

+# If set to "raw", use the below configuration:
+


Where is the standard_ophit inheriting its configuration from?

standard_ophit comes from opticaldetectormodules.fcl which lives in Larana, and I have kept the values with which I get better results with raw waveforms. Do you agree or keep the default values?

The standard ophit configuration actually lives here: opticaldetectormodules_dune.fcl. If you need to change those values, please do it there. But keep in mind that we may want to keep for a while the simple PE config too so you may want to create a separate configuration if those numbers are to be good for something else.

maridelg

Thanks Laura for your suggestions.I have added two proposals.

maridelg · 2023-05-03T08:34:47Z

duneopdet/OpticalDetector/OpHitFinder/OpHitFinderDeco.fcl

@@ -24,4 +24,12 @@ standard_ophit_finder_spe:

 }

+# If set to "raw", use the below configuration:
+


standard_ophit comes from opticaldetectormodules.fcl which lives in Larana, and I have kept the values with which I get better results with raw waveforms. Do you agree or keep the default values?

duneopdet/OpticalDetector/Deconvolution/Deconvolution.fcl

lpaulucc · 2023-05-03T15:06:25Z

duneopdet/OpticalDetector/Deconvolution/Deconvolution.fcl

+  Samples:           1000     # Timewindow (ReadoutWindow) in [ticks]
+  Scale:             0.001    # Scaling of resulting deconvolution signal.
+  DigiDataColumn:      0      # SPE template source file column.                     
+  DigiDataFile:      "fbk_decon_digi_2023.txt"  # The SPE template with undershoot and without pretrigger (in ADC*us), 


I also think here we could include both configurations (FBK and HPK) by doing something similar to what I proposed for the filters:
dune_deconvolution_fbk:
{
(...)
DigiDataFile: "fbk_decon_digi_2023.txt" # The SPE template with undershoot and without pretrigger (in ADC*us),
# was obtained from DAPHNE V2 and the cold amplifier with 48 SiPM FBK.
(...)
}

dune_deconvolution_hpk: @Local::dune_deconvolution_fbk
dune_deconvolution_hpk.DigiDataFile: "hpk_decon_digi_2023.txt" # The SPE template with undershoot and without pretrigger (in ADC*us),
# was obtained from DAPHNE V2 and the cold amplifier with 48 SiPM HPK.

dune_deconvolution: @Local::dune_deconvolution_fbk

Would that make sense (assuming all the other parameters are the same for HPK and FBK)?

I like, we can do it !!

maridelg · 2023-05-17T08:47:19Z

Hello Laura and Jose,

Thanks a lot for your comments and suggestions. I have made a new commit with the suggested changes. I've checked all the files in the latest version of LarSoft (v09_72_01) and everything works fine. I have fixing three modules that had the error related to_registerAndSeedEngine_ reported by Laura and Dom (2 in OpticalDetector, 1 in Photonpropagation ).

Thanks and I stay tuned to your comments.

lpaulucc

Hi Maritza, thank you for your new commit. I have added a few comments. Please reach out in case of doubts.

lpaulucc · 2023-05-18T16:12:49Z

duneopdet/OpticalDetector/Deconvolution/Deconvolution.fcl

+
+}
+
+ //DataFile hpk


You can leave the lines that define the hpk deconvolution without the hash or were you getting problems here?

lpaulucc · 2023-05-18T16:26:51Z

duneopdet/OpticalDetector/OpHitFinder/OpHitFindAna_module.cc

+    evt.getByLabel(fInputModule, OpHitHandle);
+
+    // Access ART's TFileService, which will handle creating and writing
+    art::ServiceHandle<art::TFileService const> tfs;


Also in line 107

lpaulucc · 2023-05-18T16:27:05Z

duneopdet/OpticalDetector/OpHitFinder/OpHitFindAna_module.cc

+    fInputModule  =   pset.get< std::string >("InputModule");
+    fInstanceName =   pset.get< std::string >("InstanceName");
+
+    art::ServiceHandle<art::TFileService const> tfs;


Also in line 153

lpaulucc · 2023-05-18T16:37:42Z

duneopdet/OpticalDetector/OpHitFinder/OpHitFinderDeco.fcl

@@ -24,4 +24,12 @@ standard_ophit_finder_spe:

 }

+# If set to "raw", use the below configuration:
+


The standard ophit configuration actually lives here: opticaldetectormodules_dune.fcl. If you need to change those values, please do it there. But keep in mind that we may want to keep for a while the simple PE config too so you may want to create a separate configuration if those numbers are to be good for something else.

knoepfel · 2023-06-29T15:36:27Z

@maridelg, why was this PR closed? The SciSoft team is trying to analyze LArSoft/lardataobj#37, and it appears to be related to this PR.

jroto · 2023-06-29T17:24:50Z

@knoepfel, we merged it to the main branch, since we needed this code even it was not optimized.

lpaulucc · 2023-06-29T20:54:19Z

This PR was actually replaced by this other one #30.

knoepfel · 2023-06-29T20:55:34Z

Thanks @jroto and @lpaulucc for the explanations.

lpaulucc requested review from jroto and lpaulucc April 6, 2023 12:35

lpaulucc reviewed Apr 10, 2023

View reviewed changes

duneopdet/OpticalDetector/Deconvolution/DecoAnalysis_module.cc Outdated Show resolved Hide resolved

lpaulucc reviewed Apr 10, 2023

View reviewed changes

jroto reviewed Apr 14, 2023

View reviewed changes

lpaulucc reviewed May 2, 2023

View reviewed changes

maridelg commented May 3, 2023

View reviewed changes

lpaulucc reviewed May 3, 2023

View reviewed changes

lpaulucc reviewed May 18, 2023

View reviewed changes

jroto mentioned this pull request May 19, 2023

Wrong TimeStamp type in recob::OpWaveform LArSoft/lardataobj#37

Open

maridelg closed this May 29, 2023

maridelg force-pushed the Deconvolution-and-OpHiFinder branch from 76ae296 to edc5058 Compare May 29, 2023 10:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Deconvolution and OphitFinder #27

Deconvolution and OphitFinder #27

maridelg commented Apr 5, 2023

lpaulucc left a comment

lpaulucc Apr 10, 2023

lpaulucc Apr 10, 2023

mjdelgadog Apr 14, 2023

jroto left a comment

jroto Apr 11, 2023

maridelg Apr 26, 2023

maridelg May 5, 2023

jroto May 19, 2023

jroto Apr 11, 2023

maridelg Apr 26, 2023

jroto May 19, 2023

mjdelgadog commented Apr 14, 2023

lpaulucc May 2, 2023

maridelg May 3, 2023

lpaulucc May 18, 2023

maridelg left a comment

maridelg May 3, 2023

lpaulucc May 3, 2023

maridelg May 5, 2023

maridelg commented May 17, 2023 •

edited

Loading

lpaulucc left a comment

lpaulucc May 18, 2023

lpaulucc May 18, 2023 •

edited

Loading

lpaulucc May 18, 2023

lpaulucc May 18, 2023

knoepfel commented Jun 29, 2023

jroto commented Jun 29, 2023 •

edited

Loading

lpaulucc commented Jun 29, 2023

knoepfel commented Jun 29, 2023

		@@ -24,4 +24,12 @@ standard_ophit_finder_spe:

		}

		# If set to "raw", use the below configuration:

Deconvolution and OphitFinder #27

Deconvolution and OphitFinder #27

Conversation

maridelg commented Apr 5, 2023

lpaulucc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jroto left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mjdelgadog commented Apr 14, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maridelg left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maridelg commented May 17, 2023 • edited Loading

lpaulucc left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lpaulucc May 18, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

knoepfel commented Jun 29, 2023

jroto commented Jun 29, 2023 • edited Loading

lpaulucc commented Jun 29, 2023

knoepfel commented Jun 29, 2023

maridelg commented May 17, 2023 •

edited

Loading

lpaulucc May 18, 2023 •

edited

Loading

jroto commented Jun 29, 2023 •

edited

Loading