Skip to content

Debugging Response Functions

Thomas Ham edited this page Jul 28, 2021 · 53 revisions

Some things to consider if the results from Produce and validate event rate spectra with single systematic variations and/or Produce error envelopes for event rate spectra don't look good:

Generating the response functions 'correctly' (for nue)

  • For nue, we need to generate independent sets of response functions for each sample.

    • Should do this for cosmic and dirt too? The cosmic sample exists in name only atm (after all the cuts we end up with no events despite having initially generated millions..). Dirt events seem to get mapped to some numu reac modes and not the dirt mode - something to look at in the future? For now have lumped cosmic and dirt samples with numu.
    • We'd expect the response functions from the different samples but for the same reac mode and bin to look different (e.g. nueCCQE, numuCCQE and numutonueCCQE would not be identical).
  • When generating the response functions from the oscillated sample (for nue), we need to make sure we're actually reading some events. If the PhysHypothesis is set to kHypVoid we have no oscillations so no events to generate the splines.

    • Ideally want to take all events in the osc sample (with weight = 1, without applying oscillation weighting), but not straight forward to do with the existing apps. Easiest work around is just to set some osc params so we get some events (pick sinsq = 1 and dmsq = 100 to avoid dealing with very small or vanishing numbers in some Etrue bins). Oscillations and systematic effects should “commute”, so I think this approach should be okay.

Choices things to tweak change and be aware of

  • In the spline construction app, there is a line similar to

    double event_ratio = (nevents0 > 0) ? nevents/double(nevents0) : 1; // Ratio of the variation rate to the nominal rate

  • This requires more than N events (0 in this case) in a MCT bin before producing a non-flat spline. In previous iterations (for the numu analysis in particular), the check was set to nevents0 > 4. Not a 100% sure on the reasoning for choosing a non-zero number. Picking 0 should generate a spline which is a 'perfect' representation, but perhaps if splines look strange it might be caused by dividing by a tiny number.

  • Generating the splines is done by comparing the nominal to nsigma variations (see line above). Previously (again for the numu analysis), the nominal was produced by using the universe with a variation closest to zero - have now changed to using the true nominal (i.e. no variations at all).

  • Need to be careful as we can have a 'true' nominal spectrum (labelled 'nom') and a 'pseudo' nominal spectrum which is produced by using the universe variation closest to 0 (labelled 'p0'). For example the tweak_pdf_nsigma_var.cxx app generates both and when using the spline_validation.C macro to compare the spectra from the universes and from VALOR, we need to make sure we're using the same nominal spectrum.

  • The universe_variation_tweaks.cxx app counts events differently to the other apps/macros. It doesn't care about any PhysHypothesis and does what is mentioned above in the part about generating response functions for the osc sample I believe. This means that the spectrum from the oscillated sample will be different and the nominal ratio isn't comparable to the others. The single and double ratios are of course still comparable. The intrinsic and numu samples are unaffected by this.

Deep dive into event rates

  • Probably best to focus on an individual systematic and a single sample (if looking at the nue analysis) i.e. look at the plots from Produce and validate event rate spectra with single systematic variations.

  • First check the nominal spectra agree. If not confirm that the standard and universe dsts have the same number of events and then check we're looking at the same nominal spectra (see above)

  • If error coverage looks bad, check individual mct bins.

  • For nue, can look at individual samples.

  • For given reco bin, sum of all associated mct bin should give event rate in that reco bin seen in the 'standard' spectra.

  • If splines exist and are 'correct', but valor and universes still don't agree, the first thing to check is the reac_type in the definition of the systematic. Probably what's happening is that some modes are having an impact, but we've said that they shouldn't so the corresponding splines are being ignored. Can temporarily set reac_type to 'all' to check. This should use all the splines available, but i guess is bad practice.

Clone this wiki locally