Integrating systematics handling into nanoevents

[lgray]

Sketching out ideas for rich/extensible/efficient systematics handling in nanoevents.

note / warning: This is not meant to suggest a buy-in framework for systematics, but rather to engineer a naming convention for extensions by which systematics can be added to an analysis. This naming convention is then assumed by statistics tools to create variations and ensembles expecting common data. What is here right now is attempting to get to the point where a schema can be defined (doing this beforehand is self defeating). So while it may look like I'm writing a deeply opt-in framework, what I am opting-in to is agreeing to an interface that we should put systematics in a field in an physics object or event called "systematics". I am not trying to extend nanoevents such that everyone has to use it forever to calculate systematics. I am trying to alter nanoevents to suit the naming convention we are solving for.

New methods class: Systematic. If an object has systematics associated to it then it should inherit from this.

Also adds systematics subdir to nanoaod.methods, there is a simple "Up/Down" systematics class there right now.

Right now NanoEvents, and then the NanoAOD object classes.

It may make more sense to tie it directly to NanoCollection so that everything gets it.

There is one special keyword so far weight which tells the systematics to append weight_SystematicName to the class instead of altering the value of a feature in the object. It's clunky right now but it gets the job done correctly.

I'll try to keep systematics.ipynb working while I hack on this.

Guiding ideas:

• This is to index and evaluate systematics in a numpythonic way
• "fancy indexing for systematics"
• specification for an array form that you can build any systematics evaluation / ensemble building machinery on top of.

[lgray]

To finish and merge:

• improve speed of eventwise weights (right now it needs to read a lot for ... reasons?)
• documentation

[lgray]

@nsmith- can you take a look?

[lgray]

@nsmith- ping?

[lgray]

@andrzejnovak too

[andrzejnovak]

Neat! Two UX questions. Should there be a specific one way syst option rather than filling up or down with nominal for example? Could add_systematics also take directly 2XN array or is it only function such that it can be lazy?

[lgray]

@andrzejnovak

1. So UpDownSystematic is meant to be an example and it's possible to implement any set of variations as a systematic when starting from the Systematic base class. I guess the real answer to this is I should include more example classes!
2. You could accomplish that by passing a lambda that returns the arrays, but yeah it's all built around lazy functions as it is implemented right now.

[lgray]

@nsmith- ping

[nsmith-]

I think it looks good, modulo users creating reference cycles in the user-supplied variation function.
I'm a bit confused how the UI looks w.r.t. looping over the weight systematic variations. I expected some function that multiplies all the nominal weights together, but it looks like here we only track up/down.

[nsmith-]

I'm worried it will be very hard for user-provided functions to not reference the array this virtual array is being attached to, and hence memory leaks.

[lgray]

Even though it's not a complete awkward schema, should we put the systematics instead in a dictionary or functor that's returned to the user instead?

Usually the pattern as it is right now almost always asks that you're referencing/altering an array that's in the object you're adding a systematic to.
The corrections or variations themselves are likely not a part of the record array though.

the what specification in add_systematic is usually some value in the record array. The input variations are not.

[nsmith-]

should we put the systematics instead in a dictionary or functor that's returned to the user instead

Can you sketch what this would look like for a user? I didn't quite follow