Run a complete mock data analysis

Two-step analysis

Run SBND-only mock data fit

See https://github.com/VALOR-fit/VALOR-SBN/wiki/Run-a-single-fit for more details

Under the sterile hypothesis:

valor-run_fit \
--tag nd-mdc-1-sterile \
--first-octant \
--run 1 \
--ntoymc 1 \
--param-values-option  prefit-errors \
--fitter minuit+with_simplex_fallback+maxcalls=1000000 \
--config analysis/sbn_numu_disapp_2020a1_03c1_08a2020_nd_1.xml \
--mesg-thresholds verbosity.xml \
--without-stat-fluct \
--normal \
--load-prior-from-file - \
--pot-override -999 \
--store-spectra \
--physics-hypothesis eff_sterile_osc \
--osc-prior-file priors/sterile_priors.xml \
--read-data

Note: configuration must be setup with both <mc> and <data> tags pointing to the relevant configurations for the analysis.

Under the no-oscillation hypothesis:

valor-run_fit \
--tag nd-mdc-1-void \
--first-octant \
--run 1 \
--ntoymc 1 \
--param-values-option  prefit-errors \
--fitter minuit+with_simplex_fallback+maxcalls=1000000 \
--config analysis/sbn_numu_disapp_2020a1_03c1_08a2020_nd_1.xml \
--mesg-thresholds verbosity.xml \
--without-stat-fluct \
--normal \
--load-prior-from-file - \
--pot-override -999 \
--store-spectra \
--physics-hypothesis void \
--osc-prior-file priors/void_priors.xml \
--read-data

Note: This may be necessary if the sterile fit cannot attribute oscillations to the MC-data differences.

Plot SBND spectra

Get the prefit spectra

Near detector

valor-write_pdf \
--config analysis/sbn_numu_disapp_2020a1_03c1_08a2020_nd_1.xml \
--write-opt pdfts_rf+pdfcs_rf+SB \
--output-name nd_mc_data_pdf \
--write-data

Tweak the PDFs to get the error bands

Generate ntoys worth of different systematic throws for spectra. The square of the difference between each toy spectra and nominal (ntoy = -1 in code) is summed and then the total is divided by ntoys to give the 1 sigma error for each model/energy bin (is symmetric pm 1)

Near detector prefit

valor-tweak_pdf_N \
--config analysis/sbn_numu_disapp_2020a1_03c1_08a2020_nd_1.xml \
--output nd_tweak_pdf_output_prefit.root \
--ntoys 1000 \
--physics-hypothesis void

Near detector postfit

valor-tweak_pdf_N \
--config analysis/sbn_numu_disapp_2020a1_03c1_08a2020_nd_1.xml \
--load-prior-from-file <filename>.root \
--output nd_tweak_pdf_output_postfit.root \
--ntoys 1000 \
--physics-hypothesis void

<filename.root> is output from fit above

Plot the spectral overlays with error envelopes

$VALOR_SBN/macros/mock_data_challenge/draw_postfit_data_error_envelope.py \
-c analysis/sbn_numu_disapp_2020a1_03c1_08a2020_nd_1.xml \
-a nd_mc_data_pdf.root \
-b nd_tweak_pdf_output_postfit.root \
-p nd_tweak_pdf_output_prefit.root \
-f root \
-t nd-data-postfit-prefit \
-e 1 \
-m True \
--alabel 'SBND mock data' \
--blabel 'SBND postfit #pm 1 #sigma' \
--plabel 'SBND prefit #pm 1 #sigma' \
--ratio 0

Plot systematic parameter pulls

Easier argument list command:

valor -b -q $VALOR_EXPT/macros/mock_data_challenge/sbn_mdc_postfit_parameter_pulls.C'("'${config}'","'${fit}'","'${tag}'_'${t}'","nd",'${t}')'

Can set up as loop to plat various configs/fit points/orientation

• config: analysis/config.xml file
• fit: output file from fit above, or select a point around your contour to see detail for that point
• tag: tag on output name
• t: thesis option (from Rhiannon) 0 or 1 for landscape (0, presentations) or portrait (1, thesis) plot
• "nd": detector name

All systematics: Flux covariance, flux splines, proposal interaction splines, modern interaction splines, MEC

valor -l -b -q $VALOR_SBN/macros/mock_data_challenge/sbn_parameter_pulls.C'("../fits/nd-susav2-mdc-1-void_1.root","${VALOR_DATA}/sbn_numu_disappearance/prior_errors/03c1/","hadronic_flux_parameter_covariance.root","flux_spline_covariance.root","proposal_interaction_spline_covariance.root","modern_interaction_spline_covariance.root","cov","mec_100pct_covariance.root","","numuCCincl","output_tag")'

All plus detector

valor -l -b -q $VALOR_SBN/macros/mock_data_challenge/sbn_parameter_pulls.C'("../fits/nd-susav2-mdc-1-void_1.root","${VALOR_DATA}/sbn_numu_disappearance/prior_errors/03c1/","hadronic_flux_parameter_covariance.root","flux_spline_covariance.root","proposal_interaction_spline_covariance.root","modern_interaction_spline_covariance.root","cov","mec_100pct_covariance.root","detector_error_matrix_2020a1_np48_cor5.000pct_uncor0.500pct.root","numuCCincl","output_tag")'

Extrapolate to ICARUS and compare with corresponding ICARUS unoscillated mock data

ICARUS spectra

Far detector

valor-write_pdf \
--config analysis/sbn_numu_disapp_2020a1_03c1_08a2020_fd_1.xml \
--write-opt pdfts_rf+pdfcs_rf+SB \
--output-name fd_mc_data_pdf \
--write-data

Far detector prediction from near detector fit result

valor-write_pdf \
--config analysis/sbn_numu_disapp_2020a1_03c1_08a2020_fd_1.xml \
--write-opt pdfts_rf+pdfcs_rf+SB \
--output-name fd_pred_mc_data_pdf \
--write-data \
--generate-fd-pred \
--fit-output-file <filename>.root

Tweak the PDFs to get the error bands

Far detector prefit

valor-tweak_pdf_N \
--config analysis/sbn_numu_disapp_2020a1_03c1_08a2020_fd_1.xml \
--output fd_tweak_pdf_output_prefit.root \
--ntoys 1000 \
--physics-hypothesis void

Far detector prediction, postfit

valor-tweak_pdf_N \
--config analysis/sbn_numu_disapp_2020a1_03c1_08a2020_fd_1.xml \
--load-prior-from-file <filename>.root \
--output fd_tweak_pdf_output_postfit.root \
--ntoys 1000 \
--physics-hypothesis void

Plot the spectral overlays with error envelopes

$VALOR_SBN/macros/mock_data_challenge/draw_postfit_data_error_envelope.py \
-c analysis/sbn_numu_disapp_2020a1_03c1_08a2020_fd_1.xml \
-a fd_mc_data_pdf.root \
-b fd_tweak_pdf_output_postfit.root \
-p fd_tweak_pdf_output_prefit.root \
-f root \
-t fd-data-postfit-prefit \
-e 3 \
-m True \
--alabel 'ICARUS mock data' \
--blabel 'ICARUS prediction #pm 1 #sigma' \
--plabel 'ICARUS prefit #pm 1 #sigma' \
--ratio 0

Joint analysis

Run a full-SBN mock data fit