Normalisation by monitor histogram introduces artefacts from low monitor counts

[jl-wynen]

We first found this as part of #145 where we started reducing all banks. For the forward endcap, we see a noisy histogram with extraneous peaks:

However, we do not see this when we normalise in 2D by way of IofDspacingTwoTheta:

Likewise, the histogram looks clean when we normalise by integrated monitor or by proton charge. It is only a problem when normalising by monitor histogram in 1D.

To investigate, here is the monitor:

It has very low counts for $\lambda < 0.6Å$. Looking at the forward endcap, we see that its peaks have significant contributions from low wavlengths (left). So when it is divided by the monitor (right), those peaks and any noise therein gets boosted:

For comparison, here is the same kind of plot for the mantle (left before, right after monitor normalisation). The peaks on the mantle are predominantly at larger wavelengths. So the influence from the low monitor counts is diminished (but not absent).

To summarise, peaks that are visible at low wavelength (esp. in the forward endcap due to low $2\theta$) are sensitive to noise which gets boosted by the monitor.

How do we solve this?

There are a couple of options here:

• Properly handle monitor uncertainties. This might assign large uncertainties to the boosted noise so that it can be properly handled in analysis.
• Remove low wavelength regions. We would lose valuable data this way. So maybe not a good idea.
• Don't normalise by monitor histogram. This might lead to worse data for large wavelength. But normalisation by proton charge seems to be common practice.

I prefer the first option. But it may be complicated if we want to uphold our correlation guarantees.

[jl-wynen]

There may be one or more mistakes in the code:

ToF ranges

The monitor's ToF range is not the same as the detector's. In particular, it seems to be cut off at lower wavelength than the detector. Here is the detector:

This seems to be a result of the tof workflow. We use that workflow on the monitor even though the monitor is already in ToF in the file. We need to fix this! (By converting to time offset after loading McStas data.)

Incorrect position?

It is surprising that the low-wavelength bins receive such a large weight under normalisation. Are we sure that the wavelengths are correct? A simple mistake might be that some positions are wrong. Note that the monitor position is hard coded in the notebook.

I checked, and it seems to match the value in the file given as # position: 0.03478 0 72.33.

[nvaytet]

How do we solve this?

Isn't there also the option of always going to 2d (two_theta, dspacing) before reducing to a 1d histogram?

[nvaytet]

Also, does this depend on the binning chosen for the monitor?
If you make very coarse bins, we should be getting closer to the "monitor integrated" case?
How was the binning chosen in the first place? Arbitrary?

[jl-wynen]

Isn't there also the option of always going to 2d (two_theta, dspacing) before reducing to a 1d histogram?

Yes. As long as you pick good bins. We saw that it was possible to get bad data with 2d bins, too.

If you make very coarse bins, we should be getting closer to the "monitor integrated" case?

I would think so. We could do that.

How was the binning chosen in the first place? Arbitrary?

It is what comes out of the time of flight workflow.

[nvaytet]

It is what comes out of the time of flight workflow.

Ok, so it's making bins that have a width equal to the mean bin width of the original histogrammed monitor.

[nvaytet]

We saw that it was possible to get bad data with 2d bins, too.

Were you able to reproduce that again?

[jokasimr]

We use that workflow on the monitor even though the monitor is already in ToF in the file. We need to fix this! (By converting to time offset after loading McStas data.)

If the McStas monitor registers "time of flight" and not "time of arrival" or "time offset" then converting to "time offset" after loading will be tricky.

To make this "exact" we would need to create an "inverse" lookup table that computes event_time_offset as a function of tof. If we want something approximate we can just mod the tof values to get approximately the event_time_offset.
The difference between the two is displayed in this graph:

Would it be simpler to make TofMonitorData the return type of load_mcstas_monitor, and not apply the tof workflow to it at all?

[jokasimr]

If the McStas monitor registers "time of flight" and not "time of arrival"

We should double check this with Peter or Celine.

Update: I asked Peter and what's called Time Of Flight in the McStas files is always time of arrival, that goes for detectors and monitors alike. So we don't have to worry about what I wrote above. Converting the time of arrival in the files to event_time_offset should just be a mod with the pulse period.

[jokasimr]

Update: Converting McStas "tof" to event_time_offset directly after loading the monitor makes the monitor wavelength distribution look a lot better at high wavelength.

The monitor still has low intensity in the $\lambda < 0.6 A$ region compared to the detector data.
But the problem seems to be the wavelengths in the detector data.

In the figure below I'm comparing the monitor wavelength histogram to the detector wavelength histogram and the true detector wavelength histogram (where the wavelength coordinate is loaded directly from the McStas/Gean4 output.

The monitor and the true detector wavelength distribution matches quite well, but the wavelength distribution from the tof workflow is offset to the left. (Update: the "true" wavelengths in the file are wrong)

[jokasimr]

Where can I look at the code that generates the lookup table? There might be something wrong there. The lookup table and the event_time_offset to tof relationship we see in the detector don't look the same. Figure displays slice of the lookup table at a fixed distance, and compares that to the tof / event_time_offset relationship at the same distance.

Could there be a mismatch between the chopper configuration used to create the lookup table, and the chopper configuration used in the McStas simulation?

Alternatively the wavelength values in the gean4 csv files might be incorrect, does that make sense?

[jl-wynen]

This is currently in flux. See here: #160