Produce oscillation sensitivity plots

New commands using jobsub_lite

Exclusion Contour

Stat only example, for including systematics increase jobs (~200-400) and memory, change config/tag/tarball (stat only runs very quickly if use tarball that only includes dsts)

jobsub_submit \
  --group sbnd \
  --role=Analysis --memory=8000MB --expected-lifetime=48h  --disk=50GB \
  --onsite-only \
  --singularity-image /cvmfs/singularity.opensciencegrid.org/fermilab/fnal-wn-sl7:latest \
  -N 40 \
  --generate-email-summary \
  -f /pnfs/sbnd/resilient/users/bslater/valor_prism_3or8_numu_stat.tar \
  file:////sbnd/app/users/bslater/valor/VALOR-SBN/prod/jobsub_submit_param_scan_multifits_v2_tarball.sh \
  --tag sbnd_prism3_numu_stat --param_nullhyp1 0 --param_nullhyp2 0 \
  --physics-hypothesis eff_sterile_osc \
  --config-file /pnfs/sbnd/persistent/users/bslater/valor/config/sbn_numu_disapp_prism3_2020a1_sbndonly_stat.xml \
  --param1 sinsq2thMuMu --min-value1 1E-3 --max-value1 1 --log-param1 1 --npoints1 40 \
  --param2 dm41sq --min-value2 1E-2 --max-value2 1E2 --log-param2 1 --npoints2 40 \
  --param3 sinsq2thMuE --param_nullhyp3 0 --fixed_at_val3 0 --fix_val3 0 \
  --param4 sinsq2thEE --param_nullhyp4 0 --fixed_at_val4 0 --fix_val4 1 \
  --discovery 1 --hierarchy normal --store-spectra 1 \
  --fit-option minuit+with_simplex_fallback \
  --load-prior-from-file none --pot-override -999 \
  --mesg-thresholds verbosity.xml --param-values-option prefit-errors \
  --jobdir /pnfs/sbnd/scratch/users/bslater2/valor/prism3/numu_stat \
  --tarball valor_prism_3or8_numu_stat.tar \
  --njobs 40 \
  --osc-prior-file /pnfs/sbnd/persistent/users/bslater/valor/priors/void.xml \

Tarball fix for jobsub_lite having size limit

-f is full tarball path

--tarball is name of tarball

file:/// changes from old to accommodate accessing tarball by jobsub copying it to the worker node from dCache, and making it available at the same path on the worker node

Arguments

Jobsub options you may want to change: --memory, --expected-lifetime, --disk, number of jobs (both -N and --njobs) (use your intuition)

--fix_val3 and --fix_val4 should have at least 1 floating (set to 0) for stat-only fits but for stat+syst fits both should eb fixed (set to 1)

VALOR options you may want to change: --config-file, --paramX (param1 is the mixing angle of interest, param3/4 are the other two mixing angles which we tend to fix to 0), --min-value1 and max-value1 (tend to use [1E-3,1] for numu and [1E-5,1] for nue)

Notes

Config file, prior file, jobdir, and tarball need to be on /pnfs. Obviously point these to one of your own directories (tarball on resilient, config/prior on persistent).

--jobdir should point to /pnfs/sbnd/scratch/users/usename/ as the token authentication path that users have access to depends on the username

Check job status at https://fifemon.fnal.gov/monitor/d/000000116/user-batch-details?orgId=1

Allowed regions

Same as above but with injected osc params.

i.e. --param_nullhyp has non-zero values.

Produce sensitivity contours

Put all the fit root files in one directory.

$VALOR_EXPT/macros/contours/fit_result_to_graphs.py '/path/to/root/files/*.root' -p ANALYSIS_VARIANT -e

where ANALYSIS_VARIANT ∈ {MuMu, MuE, EE} and -e tells the script we're producing an exclusion contour (omit if doing inclusion).

External limits for comparison are located in VALOR-SDK/data/contours/sterile_nu/.

Draw chisq surface with contours

valor VALOR-SBN/macros/contrib/tom/chisq_contours.cc'("output_root_file_from_fit_result_to_graphs.py","ANALYSIS_VARIANT")'

where ANALYSIS_VARIANT ∈ {mumu, mue, ee} <---> numu disappearance, nue appearance and nue disappearance.

Old (jobsub_client)

Example commands to produce many fits for sensitivity contours (at FNAL)

Need to setup the Jobsub client.

source /cvmfs/sbnd.opensciencegrid.org/products/sbnd/setup_sbnd.sh
setup jobsub_client

Exclusion contour

jobsub_submit \
  --group sbnd \
  --role=Analysis --memory=6500MB --expected-lifetime=48h  --disk=50GB \
  --append_condor_requirements='((TARGET.HAS_SINGULARITY=?=true)&&(TARGET.HAS_CVMFS_sbnd_opensciencegrid_org==true))' \
  --resource-provides=usage_model=DEDICATED,OPPORTUNISTIC,OFFSITE \
  -l '+SingularityImage=\"/cvmfs/singularity.opensciencegrid.org/fermilab/fnal-wn-sl7:latest\"' \                                                                                     
  -N 200 \
  --generate-email-summary \
  file:///sbnd/app/users/tham/VALOR_master/VALOR-SBN/prod/jobsub_submit_param_scan_multifits_v2.sh \
  --tag sbn_modern_numu_disapp --param_nullhyp1 0 --param_nullhyp2 0 \
  --physics-hypothesis eff_sterile_osc \
  --config-file /pnfs/sbnd/persistent/users/tham/valor/config_files/sbn_2020/MY_CONFIG.XML \
  --param1 sinsq2thMuMu --min-value1 1E-3 --max-value1 1 --log-param1 1 --npoints1 40 \
  --param2 dm41sq --min-value2 1E-2 --max-value2 1E2 --log-param2 1 --npoints2 40 \
  --param3 sinsq2thMuE --param_nullhyp3 0 --fixed_at_val3 0 --fix_val3 1 \
  --param4 sinsq2thEE --param_nullhyp4 0 --fixed_at_val4 0 --fix_val4 1 \
  --discovery 1 --hierarchy normal --store-spectra 1 \
  --fit-option minuit+with_simplex_fallback \
  --load-prior-from-file none --pot-override -999 \
  --mesg-thresholds verbosity.xml --param-values-option prefit-errors \
  --tarball /pnfs/sbnd/resilient/users/tham/valor/MY_TARBALL.tar \
  --jobdir /pnfs/sbnd/persistent/users/tham/valor/MY_DIRECTORY \
  --njobs 200 \
  --osc-prior-file /pnfs/sbnd/persistent/users/tham/valor/priors/void.xml \

Jobsub options you may want to change: --memory, --expected-lifetime, --disk, number of jobs (both -N and --njobs) (use your intuition)

VALOR options you may want to change: --config-file, --paramX (param1 is the mixing angle of interest, param3/4 are the other two mixing angles which we tend to fix to 0), --min-value1 and max-value1 (tend to use [1E-3,1] for numu and [1E-5,1] for nue)

Allowed regions

Same as above but with injected osc params.

i.e. --param_nullhyp has non-zero values.

Notes

Config file, prior file, tarball and jodir need to be on /pnfs. Obviously point these to one of your own directories.

VALOR_tarball.tar is a tarball of the VALOR-SBN and VALOR-SDK. Can create with tar -cvf valor.tar VALOR-SBN/ VALOR-SDK/ (use -hcvf if data files are located elsewhere and are connected via symlink).

Avoid making your tarball too large, ~2GB or so is fine (otherwise you may get shouted at by grid people..).

Check job status at https://fifemon.fnal.gov/monitor/d/000000116/user-batch-details?orgId=1

Produce sensitivity contours

Put all the fit root files in one directory.

$VALOR_EXPT/macros/contours/fit_result_to_graphs.py '/path/to/root/files/*.root' -p ANALYSIS_VARIANT -e

where ANALYSIS_VARIANT ∈ {MuMu, MuE, EE} and -e tells the script we're producing an exclusion contour (omit if doing inclusion).

External limits for comparison are located in VALOR-SDK/data/contours/sterile_nu/.

Draw chisq surface with contours

valor VALOR-SBN/macros/contrib/tom/chisq_contours.cc'("output_root_file_from_fit_result_to_graphs.py","ANALYSIS_VARIANT")'

where ANALYSIS_VARIANT ∈ {mumu, mue, ee} <---> numu disappearance, nue appearance and nue disappearance.