DEPRECATED: Merging oscNext high stats analysis latest changes

[ts4051]

This is a merge of a number of changes implemented in recent months to support the oscNext high stats analyses.

Summary of changes:

1. Fix to bug in analysis fits where updates to params with multiple selections defined (e.g. mass orderings) are lost. This stopped sensitivity scans working for the numu disappearance analysis. The fix mostly involved removing taking copies of param objects, such as during the octant check.

2. Before scipy minimization begins, there is now a check to see if the initial params are already perfect (e.g. return metric == 0). If so, there is no need to minimize. This saves time, and avoids issues observed with the L-BFGS-B algorithm which can sometimes fail if the initial param values are exactly correct.

3. Added more metadata for failed fits.

4. Added smarter down-sampling of events, allowing breaking the events into arbitrary sized sub-populations, and selecting the desired sub-population. Used to create statistically independent templates with arbitrary statistics.

5. Added support for setting the tex member of the Param class via config files.

6. The grid.py stage (which creates fake input data on a grid, useful for making e.g. oscillograms) didn't work in PISA 4. This is now fixed.

7. Neutrino decoherence re-integrated into the nusquids.py stage )this has been broken by recent updates as part of the sterile analysis)

8. Some general performance improvements to the hypersurface code. No functional changes however, just better memory, file size, load times, error handling, etc. Also added additional data provenance.

9. Some ranges added to nufitv40.cfg.

10. A few general added/improved comments, etc.

[atrettin]

It looks like there are a lot of old commits in this branch that already should exist in the master branch... could this be rebased, please?

[philippeller]

@ts4051 what's the status of this? Do you think we can wrap-up this PR soon?

[atrettin]

I would like to review a few things first... and it would be nice if this could be rebased. I will see to it that I check it out tomorrow and have a look.

[ts4051]

Yes the rebase what is holding me up, will try and get to this tomorrow

…

On Mon, Nov 22, 2021 at 5:51 PM Alexander Trettin ***@***.***> wrote: I would like to review a few things first... and it would be nice if this could be rebased. I will see to it that I check it out tomorrow and have a look. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#684 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABMVI64F35II7G3EMZTIRKLUNJYKFANCNFSM5HSFTCYQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.