Walk Through Tutorial 2025 : Minimizing PROfit
This tutorial assumes that you are relatively familiar with the v1.0.x style XML configuration files. If not please read up on the basics at XML tutorial before starting to work though this document.
The purpose of this walk though is to show the commands and output for a variety of different combinations and studies currently possible in PROfit.
Note: This walk-through has been developed and tested on v1.0.3 PROfit release. We will update accordingly.
The standard PROfit executable command is of the form
PROfit -x my_xml_name.xml -t my_tag SUBCOMMAND
or longform
PROfit --xml my_xml_name.xml --tag my_tag SUBCOMMAND
Where the SUBCOMMAND is further instructions as to what to actually perform. The allowed subcommands can be found with PROfit --help along with other useful information:
PROfit: a PROfessional, PROductive fitting and oscillation framework. Together let's minimize PROfit!
Usage: PROfit [OPTIONS] [SUBCOMMAND]
Options:
-h,--help Print this help message and exit
-x,--xml TEXT REQUIRED Input PROfit XML configuration file.
-v,--verbosity ENUM [3] Verbosity Level [1-4]->[Error,Warning,Info,Debug].
-t,--tag TEXT [PROfit] Analysis Tag used for output identification.
-o,--output TEXT [v1] Additional output filename quantifier
-n,--nthread UINT [1] Number of threads to parallelize over.
-m,--max UINT Max number of events to run over.
-c,--chi2 TEXT [PROchi] Which chi2 function to use. Options are PROchi or PROCNP
-d,--data TEXT Load from a seperate data xml/data file instead of signal injection. Only used with plot subcommand.
-i,--inject FLOAT ... Physics parameters to inject as true signal.
-s,--seed INT [-1] A global seed for PROseed rng. Default to -1 for hardware rng seed.
--inject-systs [TEXT,FLOAT] ...
Systematic shifts to inject. Map of name and shift value in sigmas. Only spline systs are supported right now.
--poisson-throw Do a Poisson stats throw of fake data.
--scale TEXT ... Scale detector POT by a given value.
--syst-list TEXT ... Excludes: --exclude-systs
Override list of systematics to use (note: all systs must be in the xml).
--exclude-systs TEXT ... Excludes: --syst-list
List of systematics to exclude.
--fit-options [TEXT,FLOAT] ...
Parameters for single, detailed global best fit LBFGSB.
--scan-fit-options [TEXT,FLOAT] ...
Parameters for simpier, multiple best fits in PROfile/surface LBFGSB.
-p,--preset TEXT Preset fitting params. Available `fast`, `good` and `overkill` .
-f,--rwfile TEXT File containing histograms for reweighting
-r,--mockrw TEXT ... Vector of reweights to use for mock data
--log TEXT File to save log to. Warning: Will overwrite this file.
--scale-by-width Scale histgrams by 1/(bin width).
--event-by-event Do you want to weight event-by-event?
--statonly Run a stats only surface instead of fitting systematics
--force Force loading binary data even if hash is incorrect (Be Careful!)
--no-xrootd Do not use XRootD, which is enabled by default
--shapeonly Excludes: --rateonly
Run a shape only analysis
--rateonly Excludes: --shapeonly
Run a rate only analysis
Subcommands:
process PROcess the MC and systematics in root files into binary data for future rapid loading.
surface Make a 2D surface scan of two physics parameters, profiling over all others.
profile Make a 1D profiled chi2 for each physics and nuisence parameter.
plot Make plots of CV, or injected point with error bars and covariance.
fc Run Feldman-Cousins for this injected signal
protest Testing ground for rapid quick tests.
I will now go though the SUBCOMMANDS one by one outlining their usage and outputs.
process has no additional options or flags.
process is the subcommand used to process all input MC and data root files as defined in the xml, and serialize into a small lightweight binary files. This is true for all CV montecarlo as well as systematics needed to form all covariance and splines for the given binning as defined in the xml.
Once process has been run once, and binary files produced, they are used instead of directly loading the root files in all future subcommands that share the same tag . This is indeed the point of a --tag, to indicate a single processing of the root files into serialized binary files.
process is unique in that it will run automatically if the binary files are not automatically found. So manually running process is not needed. When the binary files are created a hash of the XML is saved internally. If you then change the XML (e.g binning or MC root files) and attempt to load the incorrect binaries (e.g you keep the same tag) PROfit will complain and in this case you can manually reprocess with process command. Its good practice with PROfie to use --log mylog.log or pipe output to a logfile manually. The command is as follows
PROfit -x PROfit_WalkThough_June2025.xml -t MYTAG --log mylog.log process
For this exact test tutorial file, this takes about 40 seconds, but only has to be ran once. the expected output is two binary files
MYTAG_prop.bin MYTAG_syst.bin
If you modify the xml and try to run PROfit again with reprocessing you will get the following error:
3 main || Loading PROpeller and PROsysts from precalc binary input: MyTutorial_prop.bin
3 load || Serialization load of PROpeller from file MyTutorial_prop.bin took 0.0179727 seconds
3 main || Done loading. Config hash (1789188867) and binary loaded PROpeller (4220862266) are here.
1 main || ERROR config hash (1789188867) and binary loaded PROpeller (4220862266) not compatable!
If you get this error you can simply re-run process (or change tag, as this will create a new file also).
Subcommand plot has all global arguments and options above as well as some plot specific ones: PROfit plot --help :
Make plots of CV, or injected point with error bars and covariance.
Usage: PROfit plot [OPTIONS]
Options:
-h,--help Print this help message and exit
--with-splines Include graphs of splines in output.
--area-norm Make area normalized histograms.
First lets run a very simple plot of the CV
PROfit -x PROfit_WalkThough_June2025.xml -t MYTAG plot
Outputs are as follows
- MYTAG_v1_PROplot_CV.pdf (CV spectra for main reco variable)
- MYTAG_v1_other_0_PROplot_CV.pdf (CV spectra for all OTHER variables)
- MYTAG_v1_PROplot_ErrorBand.pdf (CV spectra + Error band for main reco variable)
- MYTAG_v1_PROplot_other_0_ErrorBand.pdf (CV spectra + Error band for all OTHER variables)
- MYTAG_v1_PROplot_Covar.pdf (All covariance Variables)
- MYTAG_v1_PROplot_CV.pdf (CV spectra for main reco variable)
- MYTAG_v1_PROplot.root (Root file containing above into)
The following is the output of MYTAG_v1_PROplot_Errorband.pdf
You will note there is "data", which here is just the expected CV (or asimov data) as we have not specified otherwise. Lets change that and inject a signal. We do this using --inject followed by the physics parameters values you want. E.g a 10eV^2 sterile with a sin^2thetamumu of 0.3. The physics parameters are defined in your model (e.g in PROmodel.h)
PROfit -x PROfit_WalkThough_June2025.xml -t MYTAG --inject 10. 0.3 plot
Now we get (a) an additional file MYTAG_v1_PROplot_Osc.pdf and also out ErrorBar plot now has "data" corresponding to the fake-data injected point.
You will note you overwrote the last PDF. This is not always wanted behavior. You could change the tag but that would mean remaking the binaries. This is where -o or --output is useful. This allows you to have a secondary tag used for printing out only and allows you to generate many different plots without overwriting.
You can also inject systematic shifts. E.g in this example we shift the RPA_CCQE up by 1 sigma and NormCCMEC down by 2 sigma. Note we use -o RPA1CCMEC2 to not overwrite the last plot.
PROfit -x PROfit_WalkThough_June2025.xml -t MYTAG -o RPA1CCMEC2 --inject 10. 0.3 --inject-systs RPA_CCQE 1.0 NormCCMEC -2.0 plot
This fake data now includes both a systematic shift as well as a injected signal:
We can also add the following for additional tweaks. This applies for both subcommand plot and those to follow:
-
--poisson-throw(Add a random Poisson Throw after injection and syst shifts) -
--scale(e.g--scale SBND 0.2Scale channels that match SBND by 0.2) -
--shapeonly(Shape only analysis) -
--rateonly(Rate only analysis, i.e 1 Bin norm)
Ok, so we can make spectra and inject signals. Lets perform a fit.
Perform a full global fit, as well as profiled results for all physics params and splines
Usage: PROfit profile [OPTIONS]
Options:
-h,--help Print this help message and exit
--syst-only Profile over nuisance parameters only
--mcmc-prefit Use MCMC to sample the systematic priors for the pre-fit error band.
Lets take that last injected signal as an example. We will use Poisson throws at the end, so to make it reproducible lets fix the random number seed with --seed 404. Lets also introduce the idea of multi-threading, as we are now going to be performing a full fit. Most fitting code is already threaded in PROfit, and you can use -n or --nthread INT to speed up your studies. Lets do that now, I will use 16 threads but tailor to your situation:
PROfit -x PROfit_WalkThough_June2025.xml -t MYTAG -o RPA1CCMEC2_rand --seed 404 -n 8 --inject 10. 0.3 --poisson-throw --inject-systs RPA_CCQE 1.0 NormCCMEC -2.0 --log fit.log profile
Take a look at all your outputs. First lets look at (a subcset) the MYTAG_RPA1CCMEC2_rand_PROfile.pdf
In all of these we show 1 parameter at a time (either physics or nuisance) and profile over remaining. The top row shows the physics parameters and delta chi^2 after profiling over all others. First 2 (blue) are over whole range, and next 2 (purple) are zoomed in to approx 1 sigma. For this example we injected a strong signal so we easily see the signal injected. Note the x-axis here is log scale.
The remaining show each nuisance/spline parameters. The dashed red is the prior error (by definition 1 sigma gaussian) and the black shows the postfit profiled delta chi^2. You can see the +1 sigma RPA_CCQE injection has a bit effect, but we fit well to the new injected value. You will see that injected -2.0 sigma NormCCMEC was partly covered but out BF here is closer to -1, but the SBNDNorm has also been shifted lower. This shows they are very correlated, and its easier to achieve the same good fit result by lowering both a little compared to NormCCMEC by 2 sigma.
Often we only care about the new +/- 1 sigma error bars and not the full deltachi^2 surface this can be shown in MYTAG_RPA1CCMEC2_rand_PROfile_1sigma.pdf
Here red star is injected point. Black is best fit