Abins: import force constants data using Euphonic

[ajjackson]

Description of work.

Builds on (and no longer blocked by) #30760

Use the new semi-optional Euphonic dependency to allow import from force-constant data in .castep_bin (CASTEP), .yaml (Phonopy) or .json (Euphonic native) formats.

This greatly expands the range of atomistic codes that could be used with Abins (via Phonopy) and lays the groundwork for implementing more sophisticated simulation schemes that properly account for energy-q relationships.

To test:
Manual testing

See #30760 testing guide for some suitable input files.

Open Abins Algorithm, select ForceConstants as input file type and run with .castep_bin, .yaml etc. files.
(For Euphonic .json try something from the Euphonic tests_and_analysis/test/data/force_constants directory.)
If Euphonic is not installed, the VibrationalOrPhononFile field should fail to validate, with a tooltip advising to install Euphonic with an installer from the ScriptRepository. (This should not be necessary for Conda-built Mantid.)

The q-point sampling mesh is determined by a variable in abins.parameters. This setup is used somewhat consistently in Abins to allow "advanced" options to be tweaked. The default value is on the high side to be safe, but the best-practice would be to check convergence. (This is not an option for other formats which use pre-computed phonon modes.) If Abins is re-run after changing abins.parameters.sampling['force_constants']['qpt_cutoff'], the Abins log should show that this invalidated the cache, and a new set of scattering spectra is calculated. The Monkhorst-Pack grid (i.e. regular divisions of the reciprocal unit cell) is related to the cutoff; the sampling is reported in the log and each direction should scale linearly with the cutoff and inversely with the crystalline unit cell size. (Values are rounded up so this behaviour may be less clear for large cells.)

Unit testing
At the moment there is no automatic unit test for this feature. This is because Euphonic is not yet available on the unit-testing CI. I propose to revisit this when the CI is running on a Conda build. But perhaps it would be an idea to include some kind of skippable test before then, which runs if Euphonic is available? It would be easy enough to hack something, but do we have a standard/maintainable way of doing that?

There is no associated issue.

---

Reviewer

Please comment on the following (full description):

Code Review

• Is the code of an acceptable quality?
• Does the code conform to the coding standards?
• Are the unit tests small and test the class in isolation?
• If there is GUI work does it follow the GUI standards?
• If there are changes in the release notes then do they describe the changes appropriately?

Functional Tests

• Do changes function as described? Add comments below that describe the tests performed?
• Do the changes handle unexpected situations, e.g. bad input?
• Has the relevant (user and developer) documentation been added/updated?

Does everything look good? Mark the review as Approve. A member of @mantidproject/gatekeepers will take care of it.

[ajjackson]

This is going very well: in this plot I was able to reproduce almost exactly the same curve by importing the .phonon file and .castep_bin file from the same phonon calculation. In the latter case, the data was "interpolated" to the same 2x2x2 mesh as the exact Dynamical Matrix points in the .phonon file. The .phonon file import uses the existing Abins importer, while the .castep_bin file contains force constants and was imported with the new Euphonic infrastructure.

Finally I included a line computed with more interpolated q-points: 7x7x7 grid. The difference is quite dramatic and illustrates just how sensitive this method is to q-point sampling when small unit cells are involved. Evidently these systems should be treated with care: Abins sums over all q-points, assuming the energy-q relationship is weak for a powder-averaged measurement. That assumption is instrument-dependent and likely to break down in such cases.

[github-actions]

👋 Hi, @ajjackson,

Conflicts have been detected against the base branch. Please rebase your branch against the base branch.

---

This message is automatically generated by prince-chrismc/label-merge-conflicts-action so don't hesitate to report issues/improvements there.

[github-actions]

👋 Hi, @ajjackson,

Conflicts have been detected against the base branch. Please rebase your branch against the base branch.

---

This message is automatically generated by prince-chrismc/label-merge-conflicts-action so don't hesitate to report issues/improvements there.

[github-actions]

👋 Hi, @ajjackson,

Conflicts have been detected against the base branch. Please rebase your branch against the base branch.

---

This message is automatically generated by prince-chrismc/label-merge-conflicts-action so don't hesitate to report issues/improvements there.

[AnthonyLim23]

Could you please send me some example data?

[ajjackson]

Could you please send me some example data?

Some sample data can be found in the Euphonic test suite: I suggest the quartz-666-grid.phonon and quartz.castep_bin files from https://github.com/pace-neutrons/Euphonic/tree/master/tests_and_analysis/test/data/castep_files/quartz and mp-7041-20180417.yaml file from https://github.com/pace-neutrons/Euphonic/tree/master/tests_and_analysis/test/data/phonopy_files/CaHgO2 . Also grab a CASTEP benzene example (i.e. to check existing functionality) from existing Mantid test data: Data/UnitTest/benzene_Abins.phonon (MD5 ec53aa8ff74b7cd257d7a82c3d6547fd).

[AnthonyLim23]

using ForceConstants and the .phonon file crashes python

[ajjackson]

using ForceConstants and the .phonon file crashes python

Full-on crash, not just an Exception? Interesting! The .phonon file is supposed to be used with the CASTEP option, so this is an incompatible format. I guess the loader needs to be made a touch more robust against incompatible file input.

[AnthonyLim23]

yes please. It was a full on crash of Python (not mantid)

[ajjackson]

Right, I see two things here.

• Other options enforce a file extension. (e.g. if you pick a .phonon file and GAUSSIAN, the inputs fail validation and it doesn't try to run.) I'm not a big fan of enforcing file extensions, but as it would be consistent with the existing behaviour and make it harder to do this accidentally, we could add it.

• The traceback I get is

********* UNHANDLED EXCEPTION *********
  what(): 'ValueError' object has no attribute 'message'
  at line 82 in '/mantid_src/Framework/PythonInterface/plugins/algorithms/Abins.py'

i.e. there's an error in the code which assumes Exceptions will have a .message attribute, but it turns out that ValueError doesn't. Easy to work around.

[ajjackson]

Ok, now the ValueError is read correctly and leads to meaningful feedback in the Algorithm GUI 🎉

[AnthonyLim23]

what option does the castep_bin file work with? I would expect castep, but it fails. I also cant get the yml file to work either.

[ajjackson]

what option does the castep_bin file work with? I would expect castep, but it fails. I also cant get the yml file to work either.

.castep_bin should work with the ForceConstants option, as we are reading force-constants data. The CASTEP option is used for .phonon files (i.e. frequencies/displacements at pre-specified q-points).

I agree this is infuriating and it would be nice to ditch the list of codes as it doesn't make a lot of sense when some codes produce multiple formats and some import methods support multiple codes. I think that could be a separate bit of work though, as it will need some tedious/robust automatic determination of file type...

What happens when the yaml file doesn't work? You should get a nice tooltip from the red asterisk telling you to install Euphonic.

[AnthonyLim23]

From what I can tell, this all works as expected