Structure of the code

[henrikjacobsenfys]

This is a loose stream-of-consciousnss description of the structure of the code.

Analyis

Responsible for fitting etc. Will take a SampleModel, InstrumentModel as input.
Will have multiple fit methods: fit sequential (fit all or a subset of all data to the model in the list of ComponentCollections as discussed in SampleModel, fit simultaneous across Q (probably automatically chosen if there's a DiffusionModel in the SampleModel, since in this case, we fit the diffusion model parameters, which define a number of Lorentzians and other components with dependent Parameters, or fit the widths of Lorentzians to a DiffusionModel That will probably have a different name.
I'm unsure how to handle it if there are two Lorentzians in the model and they should be fitted with two different DiffusionModels, but it's probably somewhat straightforward to choose from a list via indexing.
It will use a Calculator (name pending review) to actually do the math.

Calculator

Will calculate the total scattering from the sample, roughly using

convoluter = Convolution(sample_model,instrument_model,energy)
y = convoluter.convolution()  + instrument_model.calculate_background()

We will make a Calculator for each Q and temperature in the list (see below)

SampleModel

The current SampleModel will be renamed to ComponentCollection or something similar. It will take a list (perhaps sc.array) of Q among other things.

The SampleModel will allow the user to append DiffusionModels to a list of DiffusionModels and append ModelComponents to a ComponentCollection.

There will also be a list of ComponentCollections, where each is a copy of the ComponentCollection that the user supplied. The user will also be allowed to work directly with this list. The list is the same length as Q; each ComponentCollection corresponds to a single Q.

Behind the scenes, it will have a list of ComponentCollection, which contains all the user supplied ComponentCollections.

The DiffusionModel will also be able to generate components. It may be best to keep them in a separate list of ComponentCollections, just to make sure they don't accidentally get overwritten or changed by the user. It should be possible to append a DiffusionModel without actually generating the components it contains., Fitting entire diffusion models is very difficult until you have a very good understanding of your data, and can take very long - in preliminary tests, fitting sequentially took about 3 seconds, and a DiffusionModel from an ok, but not great, starting point took 15 minutes.

Perhaps it will have an explicit generate_diffusion_model_components or some such.

It should have a calculate and plot method to plot the model of the scattering of the sample before convolution. I suppose that they could take two lists of ComponentCollections and make a single ComponentCollection out of them?

It will have an optional Temperature, which when not None will include detailed balance calculations.

It will eventually also have to support taking a list of temperatures and allow models to vary as function of temperature - not entirely sure how that will work. Perhaps we will actually have a list of list of ComponentCollection, where one is over temperature, and the other is over Q. Let's deal with that when we get there.

SampleModel will inherit from SampleModelBase or something similar.

SampleModelBase

Base class for sample models - contains a list of ComponentCollection as function of Q and probably not much else

InstrumentModel

Will inherit from SampleModelBase but will not have temperature. Also contains an energy_offset. The ComponentCollection it contains will be called something like background_model, and it will have a method that tentatively is called calculate_background.
The data underpinning the instrument resolution description should probably live here, but I'm not quite sure about that. Thoughts are welcome.

ComponentCollection

This is essentially what the current SampleModel is, and as the name implies, just a list of components. It will probably be nice to add a __add__ method to easily add/append another ComponentCollection, e.g. if the user has supplied a DiffusionModel and a couple of other components for their fit.

DiffusionModel

Diffusion models generally can generate a number of (typically) DeltaFunctions and Lorentzians as function of Q. They get put into a ComponentCollection.
Some more advanced models may also generate a number of functions as function of temperature or perhaps even of sample, but we'll cross that bridge when we get there.

Convolution

Takes a SampleModel and InstrumentModel and computes their convolution as function of energy. The temperature for detailed balancing will live in SampleModel, and the offset in energy will live in InstrumentModel.
I may also want to allow to take ModelComponent for the model and ModelComponent or SampleModel for the InstrumentModel, since this is nice if users want to make use of convolutions.

Underneath, it has AnalyticalConvolution for analytical pairs of ModelComponents, NumericalConvolution for non-analytical pairs (or if temperature is not None), and perhaps even a full class for DeltaConvolution, although the latter feels like overkill since delta functions act as an identity in convolutions and are thus very easy to implement. There's also a NumericalConvolutionBase which inherits from ConvolutionBase. AnalyticalConvolution also inherits from ConvolutionBase.

Experiment

This will contain the data and not much else, I think. I'm actually not entirely sure how to share the responsibilities between Experiment (or Measurement?) and Instrument - thoughts are welcome.

Job

This, I think, will hold several Analysis objects as well as potentially several Experiments. I'm not exactly sure how to structure this yet. I think some chatting with a cup of tea next to a whiteboard could help clear this up.

Basically, the typical workflow people will want is this:

• Load in all their data. We'll probably want to store it as a single scipp.DataArray or similar, which will then contain intensity as function of energy, Q, and temperature. We may even want to have it as function of sample.
• Inspect their data - probably using the Plopp slicer. Potentially remove outliers and select which region of the data is actually interesting (Region of Interest is not really a known phrase in this field, so I hesitate to use it).
• Fit the resolution to some model and inspect the fit
• Fit a single data set to the convolution of the resolution and a model
• Fit all, or a larger selection of the data, to this model, by making a ton of independent fits.
• Plot the fit parameters as function of Q, temperature, sample etc. Again, probably using the Plopp slicer.
• Fit the fit parameters to various diffusion models to determine the correct one. Many people will stop here, but really good analysis continues to:
• Generate a new SampleModel which contains the diffusion model determined above, plus other components deemed necessary
• Make a simultaneous fit of the data at all Q at a single temperature to this model and inspect the fit
• Fit the data at all temperatures, possibly using the output of the previous temperature as input
• Plot the diffusion parameters as function of temperature. Even more people will stop here, but the following has been requested by several people
• Fit the diffusion parameters to a user-defined model of its temperature dependence
• Fit the entire data set to the temperature- and Q-dependent diffusion model plus local parameters
• Repeat this for other samples
• Inspect the fit, plot the various fit parameters as function of sample etc.

A question to still be answered: What should Q be? Right now, it's a np.array, which I think is nice for the users. But it will probably make some things easier if it's internally stored as a scipp.array, since we don't have to worry too much about units. Some dependency expressions use Q, and in those cases, we'll convert the (single) Q value to a DescriptorNumber.