Add Feature: XAS Plugin #580
Conversation
Adds a working version of a plugin to calculate XANES spectra using the XspectraCrystalWorkChain of AiiDA-QE This commit requires a locally-situated copy of the core-hole pseudopotentials required for XSpectra to function, which is obtained from https://github.com/PNOGillespie/Core_Level_Spectra_Pseudos and downloaded and installed upon first activation of AiiDALab-QE
Refactors the system for importing core-hole pseudopotentials from using groups to using a `yaml` file to define which pseudos should be present in the ch pseudos archive. Greatly simplifies the logic for checking for the presence of pseudopotentials and populating lists of available pseudos. Yaml file `pseudo_toc` is obtained from the Core_Level_Spectra_Pseudos archive (https://github.com/PNOGillespie/Core_Level_Spectra_Pseudos) and must be updated on both the archive and the plugin when new pseudos are available on the CLS archive. Also adds a step in `workchain.py` to check for mismatches in which pseudos should be present and re-download the archive file if there is a mismatch. Added in order to check for inconsistencies and to automatically update the archive if `pseudo_toc.yaml` is updated with new pseudos.
Migrates the directory used for core-level spectra pseudos to `~/.local/lib/cls_pseudos` in order to allow integration tests to pass.
Adds a range of improvements and fixes to the plugin.
Fixes:
* Fixes an issue where the `Setting` panel of the plugin would
cause a crash due to a variable being re-defined during runtime.
* Adds a `.close()` statement to the end of the HTTP request when
accessing the pseudo archive file for download.
Additions:
* The app now retrieves all contributions to the final spectrum for the
selected element and plots each together. The spectra are aligned in terms of
energy in the same way as the `XspectraCrystalWorkChain` uses to generate the
final spectrum. Each contributing spectrum is scaled according to the ratio of
site multiplicity to total multiplicity. The provided broadening tools will apply
to all spectra on the widget simultaneously.
* Adds a button to the `Result` panel to download a CSV file of the spectra
presented on the Plotly widget. This will download a file titled
`{element}_XAS_Spectra.csv` which contains all spectra on the widget (total
spectrum and all site contributions). For sites with a multiplicity ratio
different from 1, the weighted and unweighted spectra are printed in separate
columns.
Codecov ReportAttention:
Additional details and impacted files@@ Coverage Diff @@
## main #580 +/- ##
==========================================
- Coverage 80.91% 77.55% -3.36%
==========================================
Files 49 54 +5
Lines 3426 3831 +405
==========================================
+ Hits 2772 2971 +199
- Misses 654 860 +206
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Really cool! Welcome XAS to join the QeApp family.
I give a broad look with two comments, @superstar54 is capable of giving it a detailed look.
| C: C.pbe-n-kjgipaw_psl.1.0.0.UPF | ||
| Cu: Cu.pbe-n-van_gipaw.UPF | ||
| F: F.pbe-gipaw_kj_no_hole.UPF | ||
| Li: Li.pbe-s-rrkjus-gipaw.UPF | ||
| O: O.pbe-n-kjpaw_gipaw.UPF | ||
| Si: Si.pbe-van_gipaw.UPF |
There was a problem hiding this comment.
I don't want to slow down the progress, is that possible to integrate this to aiida-pseudo?
There was a problem hiding this comment.
So, I will say for this that the solution being used here is not necessarily intended as a permanent solution to providing pseudopotentials for these types of calculations. In principle, integrating some sort of family in aiida-pseudo for core-hole pseudopotentials is the ideal long-term objective here - since then the procedure for assigning pseudos obviously follows that of other plugins.
The two issues I see with integrating the current library with aiida-pseudo are:
- Some sort of new
Classor other new feature will need to be made foraiida-pseudoin order to work with these types of pseudopotentials: for instance to ensure the correct core wavefunction data file is matched to the correct GIPAW pseudo, also that the core-hole and GIPAW pseudos are matched correctly. - The current library is a stub which covers only some elements, therefore potentially not enough to make it worth pursuing with just what's there at the moment.
I would be interested in your thoughts on this in any case @unkcpz.
| """Download button with dynamic content | ||
| The content is generated using a callback when the button is clicked. | ||
| Modified from responses to https://stackoverflow.com/questions/61708701/how-to-download-a-file-using-ipywidget-button#62641240 | ||
| """ |
There was a problem hiding this comment.
We have the widget to use in https://github.com/aiidalab/aiidalab-widgets-base/blob/master/aiidalab_widgets_base/export.py
There was a problem hiding this comment.
I can try it using the AWB version and see how that goes. In principle, it looks like it should be fine since the file for the output spectra is created outside of the widget, but I will let you know if there are any issues with using it here.
There was a problem hiding this comment.
Sorry, @PNOGillespie I think what you did are correct, the export widget is for exporting the aiida archive, we don't yet have the generic widget for export the file.
Can you try to it to aiidalab-qe/src/aiidalab_qe/common/, @AndresOrtegaGuerrero also implement a similar widget.
There was a problem hiding this comment.
Sure @unkcpz, I will work on using the export widget with the plugin, though I'm currently only a couple of days from vacation, so maybe in January at this rate 😆
I had intended it to show explicitly (for the user) that it would download a CSV file - also that the filename be related to the element selected to be displayed (e.g. O_K-edge_XAS.csv for spectra of oxygen). Does the widget allow for the filename or description text to be specified? @AndresOrtegaGuerrero
I don't mind working on changes to export in order to make this happen, if that's ok with everyone.
There was a problem hiding this comment.
@PNOGillespie i think @unkcpz refers to the BandsWidget PR #581 , you can adapt the export (code) to your needs, like the filename name
There was a problem hiding this comment.
Hi @unkcpz & @AndresOrtegaGuerrero, I've run some tests with the DownloadButton widget and compared it to the SpectrumDownloadButton widget I made for this plugin. I can get all the same functionality out of DownloadButton as I did for my own one, though the issue I notice is that the Plotly viewer runs noticeably slower when working with the DownloadButton.
The reason, I'm guessing, is that the DownloadButton needs the file's contents (payload) to be a bytes object, so each time the spectrum is re-drawn (e.g. if the spectrum broadening or element selection is changed) it needs to encode a new bytes object, which takes extra time (probably even more for spectra composed of multiple different absorbing atoms). The SpectrumDownloadButton doesn't have this issue because encoding is done during __on_click and so the input can just be a str object.
I do like the idea of a generalised DownloadButton widget (particularly as there's only really one way to do make this work anyway) but I think it needs some encoding methods implemented to enhance the performance.
There was a problem hiding this comment.
@PNOGillespie thanks for the work!
I did some test calculations. Here are some comments on the workchain part.
Enables the `supercell_min_parameter` setting for XAS calculations Also removes various extraneous lines of code which were commented out, had typos, or were requested for deletion.
for more information, see https://pre-commit.ci
There was a problem hiding this comment.
Hi @PNOGillespie, I see that you checked and downloaded the pseudo in the workchain.py part. I suggest moving them to the setting.py. Because the pseudo information is used in the setting panel. Of course, you can do this in another PR.
The PR will be good to me when you make the changes.
For the download button, let Andres finish his, and we can discuss which one to go with. You can update it in the future PR.
| self.on_click(self.__on_click) | ||
|
|
||
| def __on_click(self, b): | ||
| if self.contents() is None: |
There was a problem hiding this comment.
You pass Non to contents when creating the download_data button
| if self.contents() is None: | |
| if self.contents is None: |
| <h4>Element and Core-Hole Treatment Setting.</h4></div>""" | ||
| ) | ||
|
|
||
| # TODO: The element selection should lock the "Confirm" button if no elements have been |
There was a problem hiding this comment.
Could you open a issue? so that we can work on this in the future.
There was a problem hiding this comment.
I'll open one once this PR is concluded, no problem.
| } | ||
| return parameters | ||
|
|
||
| def set_panel_value(self, input_dict): |
There was a problem hiding this comment.
the element_and_ch_treatment_options is not set properly. I think you need to read the
core_hole_treatments, and elements_list from then input_dict, and set the widget values.
There was a problem hiding this comment.
Ok, this did confuse me a little on review, but element_and_ch_treatment_options is indeed not set, however it also doesn't do anything. The core_hole_treatments and elements_list however are set correctly - I tested it just now - but that's defined by the code in _update_element_selection_panel() (lines 166+).
The short answer to this is that we can just delete all of the code in lines 141-159. I can discuss this in more detail separately if you would like.
| ch_pseudos = self.core_hole_pseudos | ||
| structure = self.input_structure | ||
| available_elements = [k for k in ch_pseudos] | ||
| elements_to_select = sorted( | ||
| [ | ||
| kind.symbol | ||
| for kind in structure.kinds | ||
| if kind.symbol in available_elements | ||
| ] | ||
| ) | ||
|
|
||
| element_and_ch_treatment_options = {} | ||
| for element in elements_to_select: | ||
| if element in xch_elements: | ||
| element_and_ch_treatment_options[element] = "xch_smear" | ||
| else: | ||
| element_and_ch_treatment_options[element] = "full" | ||
|
|
||
| self.element_and_ch_treatment_options = element_and_ch_treatment_options |
There was a problem hiding this comment.
Hi @superstar54. Does this seem like a reasonable way to address your previous comment?
I suggest this because the logic to define the initial values for the element_and_ch_treatment panel has already been defined in _update_element_select_panel(). This works fine with my own testing and sets the right behaviour for set_panel_value().
| ch_pseudos = self.core_hole_pseudos | |
| structure = self.input_structure | |
| available_elements = [k for k in ch_pseudos] | |
| elements_to_select = sorted( | |
| [ | |
| kind.symbol | |
| for kind in structure.kinds | |
| if kind.symbol in available_elements | |
| ] | |
| ) | |
| element_and_ch_treatment_options = {} | |
| for element in elements_to_select: | |
| if element in xch_elements: | |
| element_and_ch_treatment_options[element] = "xch_smear" | |
| else: | |
| element_and_ch_treatment_options[element] = "full" | |
| self.element_and_ch_treatment_options = element_and_ch_treatment_options | |
| self._update_element_select_panel() |
Changes requested changes for PR aiidalab#580: * Moves the loading of pseudos and core wavefunction data to `setting.py`. `setting.py` now collects labels for pseudos and core_wfc_data, while `workchain.py` loads the nodes from the labels. * Replaces `set_panel_value()` content with call to `self._update_element_select_panel()`. * Fixes typos and removes redundant code noted in latest review.
|
The codecov decreased because there is no test on the workchain and its result. We will come to this when we resolve #533. |
Thanks for mentioning this. I'd suggest we iron the test first. |
superstar54
dismissed
unkcpz’s stale review
We leave the pseudopotential setup and the download button to the feature PR.
As we talked over the phone, please open issues for these isuses that not addressed. And we should try to do it as soon as possible, otherwise they will never happened. @superstar54 @PNOGillespie |
|
Also a mild reminder that try not do the hush merge next time. If you have DDL then we need to plan things ahead and make things happened under the plan. The integration test is failed so please try to fix it by this week @superstar54 @PNOGillespie |
|
In fact there is a crucial break in the PR, the integration test shows: |
|
You can reproduce it by start the container |
|
No worries, I add the integration test to the required test. It is quite robust and can give useful information, so we need to put an eye on it next time. |
Overview
This PR adds a plugin for calculating X-Ray Absorption (XAS) spectra using the
XspectraCrystalWorkChainof AiiDA-QE. Currently this is implemented for K-edge calculations of neutral systems. Calculations of systems with hubbard parameters and magnetic configurations can be made possible, though would require a backend change to AiiDA-QE (see aiidateam/aiida-quantumespresso#969 for relevant PR)Setting Panel
The
Settingpanel for the plugin allows users to simply select which element to calculate XAS for, as well as tuning the supercell size to be used in the calculation. This panel also provides the option to change the type of core-hole approximation between FCH (removing the excited electron with tot_charge = 1) and XCH (allowing the excited electron to occupy the conduction band. Since it is still an ongoing subject of debate in the literature, we allow the user to change this value - though recommended values for each element are provided, which will also occupy the default value in the dropdown box.Note that only elements for which appropriate pseudopotentials are available will be presented in the panel. See later section for notes about pseudopotentials.
Results Panel
Overview
The final XAS spectrum is displayed in the
Resultpanel. A dropdown box selects which element to display spectra for. The widget will display both the complete powder XAS for the input structure and the contributions from each symmetrically non-equivalent site. Note that all spectra are normalised with respect to area under the peak by default. Spectra from different sites are scaled in intensity according to the ratio of site multiplicity (number of symmetrically-equivalent sites) to total multiplicity (number of sites for the element).Broadening Tools
Broadening effects present in experimental XAS data greatly affect the appearance of the final spectrum (see Bunau & Calandra (2013)) and so post-procesing of spectra to include these effects (which cannot be computed ab-initio with DFT) is often used in published works using DFT-derived spectra. We therefore provide a set of broadening tools for the final spectrum. The tools apply one of two Lorentzian broadening schemes (toggled by a checkbox widget):
The 1st scheme uses just$\Gamma_{hole}$ (which refers to the natural linewidth of the core level) to define the Lorentzian parameter. The 2nd scheme uses each parameter to define an s-curve function with an inflection point at $E_{Center}$ . The broadening scheme is applied to all spectra on the plot simultaneously. Note that when the spectrum is downloaded, the plotted data on the Plotly widget (including broadening) is what is written to the CSV file - not the raw data. To view the raw data, the user can simply set $\Gamma_{hole}$ to 0.
Pseudopotentials
Calculations of XAS using QE require a pseudopotential to define the electron configuration of the final state of the excitation for the absorbing atom. In addition, a radial coordinate plot of the initial state (e.g. the 1s state of the absorbing atom in a K-edge calculation) is needed by XSpectra. To provide both of these, including a relevant pseudopotential for the ground-state of the absorbing element (and from which we generate the initial state wavefunction) a small pseudopotential library is provided with the plugin. This library is taken from another GitHub repository (https://github.com/PNOGillespie/Core_Level_Spectra_Pseudos) which is a small collection of pseudos which we have used already in XAS calculations. This repo has an archive file for each XC functional (currently only PBE) and a "Pseudo Table of Contents" (
pseudo_toc.yaml) file to record the filenames and directories for each pseudo and associated core wavefunction file. The plugin uses this YAML file to import files, check for missing files, load the correct nodes for calculation, and set the default core-hole approximation. The only requirement for maintainers is just to copy the YAML file fromCore_Level_Spectra_Pseudosinto the plugin directory in order to update the local pseudo archive (which is done automatically on startup if the archive file is missing or there are pseudos missing which should be there).