Refine UB from 4D MDE #119

rosswhitfield · 2023-11-17T05:01:43Z

To test:
From the MDE menu select "Refine sample parameters"

Enter binning parameters then hit histogram. That should get you to the Refine UB tab.

You should be able to manually add peaks with the "Populate Peaks button"
"Predict peaks" button will fill the table with peaks
Recenter with recenter the selected peaks
Selecting a peak should udpate the 3 plots
Refine should recalculate the UB and replot the histogram
Undo should revert to original

AndreiSavici · 2023-11-20T14:16:00Z

I'm testing it with the /EXAMPLES/HYS/IPTS-19996/shared/merged_mde/merged_mde_MnO_20meV_5K_Px_NSF.nxs. The (1,1,0) and (0,0,1) directions are in the horizontal plane, with very little extent in the vertical direction (-1,1,0). I am running from within workbench.

The following things are not right:

the workspace needs to be 3D, so add in MakeSlice something like

if slice_name == "UB_refinement_workspace":
            ProjectMD(InputWorkspace=slice_name, OutputWorkspace=slice_name,ProjectDirection='K')

When I click Populate Peaks it opens the peaks actions panel. It is not intuitive that I need to click on AddPeaks button, and the panel is containing the same information as the list of the peaks on the right, so it is useless (and it takes valuable real estate. It should be hidden, and AddPeaks automatically selected
I select some peaks to recenter, click on the button, and the centroid peaks algorithm is running, but I see no figure update. The red cross should move to the new position
It does not seem to calculate the correct U matrix
If I try to click again on refine, I get an invalidated workspace error

Error occurred in handler:
Traceback (most recent call last):
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/models/adsobsever.py", line 22, in wrapper
    func(*args, **kwargs)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/models/adsobsever.py", line 71, in replaceHandle
    self.on_replace_workspace(ws_name, workspace)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/utils/qt/qappthreadcall.py", line 69, in __call__
    return self._callable(*args)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/presenters/presenter.py", line 385, in replace_workspace
    if self.model.check_for_removed_original_workspace():
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/models/model.py", line 519, in check_for_removed_original_workspace
    return self.num_original_workspaces_at_init != self.number_of_active_original_workspaces()
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/models/model.py", line 509, in number_of_active_original_workspaces
    for i in range(workspace.numOriginalWorkspaces()):
RuntimeError: Variable invalidated, data has been deleted.

I suggest you try to test with the above workspace, and set the UB slightly off SetUB(Workspace='merged_mde_MnO_20meV_5K_Px_NSF', a=4.44, b=4.44, c=4.44, u='0,0.2,1', v='1,1,0'), and see if you recover u along (0,0,1)

rosswhitfield · 2023-11-21T05:39:11Z

The following things are not right:

* the workspace needs to be 3D, so add in `MakeSlice` something like

if slice_name == "UB_refinement_workspace":
            ProjectMD(InputWorkspace=slice_name, OutputWorkspace=slice_name,ProjectDirection='K')

That dimension is already integrated, I can do this but it won't change anything?

* When I click `Populate Peaks` it opens the `peaks actions` panel. It is not intuitive that I need to click on AddPeaks button, and the panel is containing the same information as the list of the peaks on the right, so it is useless (and it takes valuable real estate. It should be hidden, and `AddPeaks` automatically selected

It now doesn't open the peaksviewer and just does add peaks.

* I select some peaks to recenter, click on the button, and the centroid peaks algorithm is running, but I see no figure update. The red cross should move to the new position

The plots were only updating when you selected a row, I have changed it to update after recentering.

* It does not seem to calculate the correct U matrix

Using your test file, the issue is not with what I have implemented here but with Mantid and the "Add Peaks" of the sliceviewer. It first tries to get the projection matrix from the basis vectors and if that fails then the W_MATRIX. So if you look it is not getting the correct HKL when you add a peak. MDNorm is setting the basis vectors but I think we need to use the W_MATRIX instead. This needs to be fixed in Mantid here.

If I change that method to just return self.projection_matrix_from_log(ws) then I am able to calculate the U matrix. Following your example and changing the UB with SetUB, after selecting 3 peaks [(1,1,-1), (1,1,1), (0,0,2)], recenter the peaks, then setting the HKL, after I do "Refine orientation only" I end up with

which looks good to me.

* If I try to click again on refine, I get an invalidated workspace error

I have now disabled the tab while things are running so that shouldn't happen.

rosswhitfield · 2023-11-21T23:21:34Z

I fixed the issue with the sliceviewer add peaks, making it always use W_MATRIX here and UB refinement should be correct now. 🤞

AndreiSavici · 2023-11-22T19:14:37Z

I was able to get the U Matrix right for the HYSPEC data. I need to test it with the full 3D refinement. There are some remaining issues:

When trying to redo the 2D histogram, I've accidentally left the name as UB_refinement_workspace. I get many errors

Error occurred in handler:
Traceback (most recent call last):
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/models/adsobsever.py", line 22, in wrapper
    func(*args, **kwargs)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/models/adsobsever.py", line 60, in deleteHandle
    self.on_delete_workspace(ws_name)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/utils/qt/qappthreadcall.py", line 69, in __call__
    return self._callable(*args)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/presenters/presenter.py", line 433, in delete_workspace
    self.view.emit_close()
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/observers/observing_view.py", line 41, in emit_close
    self.close_signal.emit()
RuntimeError: wrapped C/C++ object of type SliceViewerView has been deleted
CreatePeaksWorkspace started
Unsupported workspace type LeanElasticPeaksWorkspace for __shiver_peaks
Error occurred in handler:
Traceback (most recent call last):
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/observers/ads_observer.py", line 23, in wrapper
    func(*args, **kwargs)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/observers/ads_observer.py", line 70, in replaceHandle
    self.presenter.replace_workspace(name, workspace)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/workspacedisplay/table/presenter.py", line 140, in replace_workspace
    self.presenter.view.blockSignals(True)
RuntimeError: wrapped C/C++ object of type TableWorkspaceDisplayView has been deleted
Error occurred in handler:
Traceback (most recent call last):
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/models/adsobsever.py", line 22, in wrapper
    func(*args, **kwargs)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/models/adsobsever.py", line 71, in replaceHandle
    self.on_replace_workspace(ws_name, workspace)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/utils/qt/qappthreadcall.py", line 69, in __call__
    return self._callable(*args)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/presenters/presenter.py", line 385, in replace_workspace
    if self.model.check_for_removed_original_workspace():
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/models/model.py", line 519, in check_for_removed_original_workspace
    return self.num_original_workspaces_at_init != self.number_of_active_original_workspaces()
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/models/model.py", line 509, in number_of_active_original_workspaces
    for i in range(workspace.numOriginalWorkspaces()):
RuntimeError: Variable invalidated, data has been deleted.

If I try to redo the refinement, before I even press the histogram button, I get

Error occurred in handler:
Traceback (most recent call last):
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/models/adsobsever.py", line 22, in wrapper
    func(*args, **kwargs)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/models/adsobsever.py", line 71, in replaceHandle
    self.on_replace_workspace(ws_name, workspace)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/utils/qt/qappthreadcall.py", line 69, in __call__
    return self._callable(*args)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/presenters/presenter.py", line 385, in replace_workspace
    if self.model.check_for_removed_original_workspace():
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/models/model.py", line 519, in check_for_removed_original_workspace
    return self.num_original_workspaces_at_init != self.number_of_active_original_workspaces()
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/models/model.py", line 509, in number_of_active_original_workspaces
    for i in range(workspace.numOriginalWorkspaces()):
RuntimeError: Variable invalidated, data has been deleted.

Once I press the button for the case above, I get a new error message

Error occurred in handler:
Traceback (most recent call last):
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/observers/ads_observer.py", line 23, in wrapper
    func(*args, **kwargs)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/observers/ads_observer.py", line 70, in replaceHandle
    self.presenter.replace_workspace(name, workspace)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/workspacedisplay/table/presenter.py", line 140, in replace_workspace
    self.presenter.view.blockSignals(True)
RuntimeError: wrapped C/C++ object of type TableWorkspaceDisplayView has been deleted
Error occurred in handler:
Traceback (most recent call last):
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/models/adsobsever.py", line 22, in wrapper
    func(*args, **kwargs)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/models/adsobsever.py", line 71, in replaceHandle
    self.on_replace_workspace(ws_name, workspace)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/utils/qt/qappthreadcall.py", line 69, in __call__
    return self._callable(*args)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/presenters/presenter.py", line 385, in replace_workspace
    if self.model.check_for_removed_original_workspace():
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/models/model.py", line 519, in check_for_removed_original_workspace
    return self.num_original_workspaces_at_init != self.number_of_active_original_workspaces()
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/models/model.py", line 509, in number_of_active_original_workspaces
    for i in range(workspace.numOriginalWorkspaces()):
RuntimeError: Variable invalidated, data has been deleted.
Error occurred in handler:
Traceback (most recent call last):
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/observers/ads_observer.py", line 23, in wrapper
    func(*args, **kwargs)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/observers/ads_observer.py", line 70, in replaceHandle
    self.presenter.replace_workspace(name, workspace)
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/workspacedisplay/table/presenter.py", line 140, in replace_workspace
    self.presenter.view.blockSignals(True)
RuntimeError: wrapped C/C++ object of type TableWorkspaceDisplayView has been deleted

then the workbench crashes (I get the terminate-continue dialog box), with the following message:

Traceback (most recent call last):
  File "/home/3y9/Mantid/mantid/qt/python/mantidqt/mantidqt/widgets/sliceviewer/presenters/presenter.py", line 421, in refresh_view
    self.view.data_view.image_info_widget.setWorkspace(ws)
RuntimeError: wrapped C/C++ object of type ImageInfoWidget has been deleted

rosswhitfield · 2023-11-23T03:42:33Z

I hope I have fixed all those errors now.

AndreiSavici

Seems to be working as expected, at least for refining the orientation only. No crashes. I need to do more testing, for the cases with refining the full UB. I will tentatively approve it

rosswhitfield force-pushed the refine_ub branch from 7d874c3 to 92ad8bc Compare November 20, 2023 05:41

rosswhitfield requested a review from AndreiSavici November 20, 2023 05:50

rosswhitfield marked this pull request as ready for review November 21, 2023 05:39

rosswhitfield added 21 commits November 23, 2023 09:11

Create base UI

1ec9512

Connect populate peaks

0e8df09

Connect predict peaks button

938a6db

Add lattice UI elements

77cd356

Add Refine UB option to MDE menu

1db8655

Set lattice parameters

4644799

Implement UB refinement

3efd2e9

Remake slice after refinement

3bc54c9

Use lattice type and handle refinement errors

2e599d9

Implement Undo

75d9689

Add 3 perpendicular plots

a76c6ee

Fix a few mistakes

843bd4b

Add cross for peak position

8147294

Add docstrings

b4398b1

Add tests for Refine UB model

0193edb

Add tests for the Refine UB UI

d3973bd

Reindex peaks after centering and force plot update

4417385

Don't show peaksviewer, just do Add Peaks

40f8aad

Disable widget while things are running

0910591

Fix error on close

caf0fcc

Force the sliceviewer add_peaks to use W_MATRIX

e179eb1

rosswhitfield force-pushed the refine_ub branch from 98df012 to e179eb1 Compare November 22, 2023 22:43

Handle stale workspaces/widgets better

e626fdb

AndreiSavici approved these changes Nov 26, 2023

View reviewed changes

rosswhitfield merged commit 1cc7cbe into next Nov 26, 2023
5 checks passed

rosswhitfield deleted the refine_ub branch November 26, 2023 22:31

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refine UB from 4D MDE #119

Refine UB from 4D MDE #119

rosswhitfield commented Nov 17, 2023 •

edited

AndreiSavici commented Nov 20, 2023

rosswhitfield commented Nov 21, 2023

rosswhitfield commented Nov 21, 2023

AndreiSavici commented Nov 22, 2023

rosswhitfield commented Nov 23, 2023

AndreiSavici left a comment

Refine UB from 4D MDE #119

Refine UB from 4D MDE #119

Conversation

rosswhitfield commented Nov 17, 2023 • edited

AndreiSavici commented Nov 20, 2023

rosswhitfield commented Nov 21, 2023

rosswhitfield commented Nov 21, 2023

AndreiSavici commented Nov 22, 2023

rosswhitfield commented Nov 23, 2023

AndreiSavici left a comment

Choose a reason for hiding this comment

rosswhitfield commented Nov 17, 2023 •

edited