Energy calibration bias shift #411
Conversation
|
I understand that you want to merge to v1 feature branch because these are all breaking changes? |
Yes, they reqire configuration changes |
Pull Request Test Coverage Report for Build 9553353548Details
💛 - Coveralls |
tutorial/2_conversion_pipeline_for_example_time-resolved_ARPES_data.ipynb
Outdated
Show resolved
Hide resolved
|
This code here unfortunately performs rather poorly. The generation of the per-file data columns alone make the computation ~20% or so slower, adding the energy offset per electron multiplies the binning time by a factor of almost 4! |
Pull Request Test Coverage Report for Build 9587172660Details
💛 - Coveralls |
Pull Request Test Coverage Report for Build 9587913234Details
💛 - Coveralls |
rettigl
force-pushed
the
energy_calibration_bias_shift
branch
from
5f336a8
to
98c4e8c
Compare
Pull Request Test Coverage Report for Build 9588080384Details
💛 - Coveralls |
Pull Request Test Coverage Report for Build 9628853930Details
💛 - Coveralls |
Pull Request Test Coverage Report for Build 9628937920Details
💛 - Coveralls |
rettigl
force-pushed
the
energy_calibration_bias_shift
branch
from
0004113
to
c347bad
Compare
Pull Request Test Coverage Report for Build 9635909706Details
💛 - Coveralls |
Pull Request Test Coverage Report for Build 9635950194Details
💛 - Coveralls |
Pull Request Test Coverage Report for Build 9636169386Details
💛 - Coveralls |
Pull Request Test Coverage Report for Build 9636247350Details
💛 - Coveralls |
rettigl
force-pushed
the
energy_calibration_bias_shift
branch
from
2aaf8b5
to
54f9673
Compare
Pull Request Test Coverage Report for Build 9652315474Details
💛 - Coveralls |
rettigl
force-pushed
the
energy_calibration_bias_shift
branch
from
54f9673
to
ac1aa18
Compare
Pull Request Test Coverage Report for Build 9652396827Details
💛 - Coveralls |
…itt from offset dict to avoid applying twice.
Pull Request Test Coverage Report for Build 9659018433Details
💛 - Coveralls |
…alibration_bias_shift
…alibration_bias_shift
Pull Request Test Coverage Report for Build 9733951548Details
💛 - Coveralls |
Pull Request Test Coverage Report for Build 9757007395Details
💛 - Coveralls |
Pull Request Test Coverage Report for Build 9759204395Details
💛 - Coveralls |
Pull Request Test Coverage Report for Build 9847465611Details
💛 - Coveralls |
There was a problem hiding this comment.
I have tried to review but currently, I don't understand the internals of the calibration methods enough to review this properly. I have tried because I wanted to adapt these for the side band correction but failed to understand.
Honestly, It'd be great if a small guide on the internals can be made. We can then also put it in user documentation.
If that's too much, if you could explain why you made these changes, I should be able to review based on that.
The only thing I noticed currently was that the tutorials still have the ref_id (in methods find_bias_peaks etc.) whereas the code only uses the ref_energy now.
tutorial/2_conversion_pipeline_for_example_time-resolved_ARPES_data.ipynb
Show resolved
Hide resolved
Thank you for your effort. I will for now comment on the changes to explain them better. If I find time, I can also make a more general documentation item out of this. |
| ref_id: int = 0, | ||
| ref_energy: float = 0, |
There was a problem hiding this comment.
Generally, the calibration estimates the functional form Energy(TOF), based on the TOF values for a given feature at different energies (bias voltages). This is being done either by nonlinear least-squares fitting, or an analytic solution of a polynomial approximation.
As the energy values provided are Bias voltages, and not directly related to the energy scale to be calibrated, this leaves a constant energy offset free to be determined. The old method allowed to define a specific "reference trace", i.e. a specific bias voltage, and the energy of the reference feature at that particular bias voltage. Ideally, this should have been the bias voltage of the actual measurement, or a corresponding additional shift needed to be applied.
The new approach is to shift the calibration by the bias voltage of the calibration (here, any one can be used, per default the first. Hence no need for ref_id anymore.), i.e. to calculate the energy scale at 0V bias voltage. When applying the energy axis, now the actual bias voltage of the measurement is being added, to obtain the correct energy scale.
| xaxis + sign * (self.biases[itr] - self.biases[self.calibration["refid"]]), | ||
| xaxis + sign * (self.biases[itr]), |
There was a problem hiding this comment.
The view function used to offset by the reference bias. This is now not necessary anymore (as adding each bias to the corresponding trace again produces the correct energy scale).
| @@ -774,6 +772,7 @@ def append_energy_axis( | |||
| tof_column: str = None, | |||
| energy_column: str = None, | |||
| calibration: dict = None, | |||
| bias_voltage: float = None, | |||
There was a problem hiding this comment.
The append function now need the bias voltage. This is either taken from the dataframe if available, or provided as parameter. If not available, incorrect energies will be given (you need to know the bias in order to assign the energy correctly).
| if "energy_scale" not in calibration: | ||
| calibration["energy_scale"] = "kinetic" |
There was a problem hiding this comment.
This was missing before, a bugfix.
| if bias_voltage is not None: | ||
| df[energy_column] = df[energy_column] + scale_sign * bias_voltage | ||
| elif self._config["dataframe"]["bias_column"] in df.columns: | ||
| df = dfops.offset_by_other_columns( | ||
| df=df, | ||
| target_column=energy_column, | ||
| offset_columns=self._config["dataframe"]["bias_column"], | ||
| weights=scale_sign, | ||
| ) |
There was a problem hiding this comment.
Either shift by constant value or by dataframe column
| energy_offset = pfunc(-1 * ref_energy, pos[0]) | ||
| ecalibdict["E0"] = -(energy_offset - vals[0]) |
There was a problem hiding this comment.
We build a partial function with offset and TOF as free parameters, and evaluate this at the TOF value (pos[0]) of the reference feature in the first trace, with given energy value (-ref_energy). This returns the difference of the calibration method to the expected value + the first bias voltage. This value minus the respective bias voltage is the negative offset to apply to obtain the correct energy value.
| ecalibdict["E0"] = -energy_offset | ||
| ecalibdict["refid"] = ref_id | ||
| if t is not None: | ||
| ecalibdict["axis"] = pfunc(ecalibdict["E0"], t) |
There was a problem hiding this comment.
Here we can now calculate the energy axis (for the first bias value).
| @@ -2312,12 +2313,10 @@ def poly_energy_calibration( | |||
| ecalibdict["Tmat"] = t_mat | |||
| ecalibdict["bvec"] = bvec | |||
| ecalibdict["energy_scale"] = energy_scale | |||
| ecalibdict["E0"] = -(pfunc(-1 * ref_energy, pos[0]) + vals[0]) | |||
There was a problem hiding this comment.
Similar considerations here.
| for col, red in zip(offset_columns, reductions): | ||
| if red == "mean": | ||
| df[target_column] = df[target_column] + signs_dict[col] * df[col].mean() |
There was a problem hiding this comment.
The direct dataframe operations are substantially faster than map_partition according to my tests. Some of the map_partition calls also fail with newer dask versions.
There was a problem hiding this comment.
Changes here are to change the config style, and to allow per-file channels, e.g. for the bias voltage to be added to the dataframe.
|
For using the calibration function for features from sidebands, do the following:
|
Shift energy calibration by bias voltages, and apply shift of dataset bias upon adding energy column.
Unify dataframe channels config
closes #399
closes #138