Move aiida-fleur xml_util #23

janssenhenning · 2021-03-02T11:28:32Z

This pull requests moves (and redesigns if necessary) most of the content of the tools/xml_util module in aiida_fleur (status of the masci_tools_parsers branch)

One fundamental change I made is that I introduced more splitting of responsibilities of these XML modifying functions. There are now 3 (or 4) categories of XML setter functions

Basic functions take a xpath and cannot create all the tags necessary if the xpath evaluation has no result (meaning xml_create_tag can only create a tag if all parent tags are already present)
Functions that take 2 xpaths. One of them is used for the evaluation (most of the time in one of the basic functions). The other one CANNOT include any complex syntax (like [@name='species-name']). This simple xpath is used in the schema dictionary to find out more information about the operation to perform. For example it can find out the enforced order of subtags or the types an attribute can have. These functions can create parent if they are not present
Functions that do not require an xpath to function (some of them still take an optional argument complex_xpath to customize behaviour). These are essentially all the user-friendly methods from aiida-fleur with some extensions
nmmpmat functions also do not need a xpath. However, they do not change the xmltree but the provided nmmp_mat file

In addition I implemented an equivalent for the Fleurinpmodifier in aiida-fleur, which operates on a concrete inp.xml (and n_mmp_mat if necessary) file. The usage is almost the same (except for initialisation and application to the file)

from masci_tools.io.fleurxmlmodifier import FleurXMLModifier

fm = FleurXMLModififier() #Note: we do not specify the inp.xml here. This is done to make evtl. integration of this class with Fleurinpmodifier less awkward
fm.set_inpchanges({'Kmax':9.0}) #Adding tasks is exactly the same (except these functions have more leniancy. i.e. automatic str conversion, case insensitivity for keys)
new_xmltree = fm.modify_xmlfile('/path/to/original/inp.xml')

Other small changes:

Added load_inpxml and load_outxml for easy access to an xmltree and version specific schema dict
Restructured module index in docs, since it was getting quite big

TODO:

Tests for xml_setters_names
Tests for xml_setters_nmmpmat
Tests for new functions in common_xml_util
Tests for load_inpxml, load_outxml and FleurXMLModifier

… be used in the old way `create_tag_xpath` or utilizing the schema_dictionary functionality `create_tag`

Split up into functions that can create missing tags on the fly and basic functions that rely on the tags being there and which do not have access to the schema_dict Added: - replace tag, delete tag/attribute - setting attributes/text (Thes functions were unified under one and only provided the first as a special alias)

Added: - set_simple_tag for single/multiple tags with only attributes - set_complex_tag Recursive handling for all kinds of inputs (ATTENTION: will delete all tags in several before rewriting) - set_species/set_inpchanges moved unchanged from aiida_fleur (masci_tools parser branch) TODO: - move conversion of attribute/text into strings into the xml_set_attrib_value and xml_set_text value - settle on a more consistent function name scheme to indicate the interface in the function name (schema_dict+name; schema_dict+base xpath; concrete xpaths with minimal requirements)

…ue/xml_set_text

- This is essentially FleurinpModifier without provenance Could be used as a superclass for FleurinpModifier. We just would need to call apply_modifications For ease of integration with FleurinpModifier the files to be modified are only provided once the modifications are supposed to happen `modify_xmlfile`

Notes: - change_atomgr_att -> set_atomgroup - change_atomgr_att_label -> set_atomgroup_label - add_num_to_att -> add_number_to_attrib (now also supports occurrences) - add_number_to_first_attrib (behaviour of add_num_to_att) - attribute/text setting from attributename with support for occurrences and complex xpaths - new function rotate_nmmpmat modifies exisiting density matrix block and rotates it according to the given angles - FleurXMLModifier.apply_modifications supports passing of extra functions for subclasses of this. This can be used for example to integrate functions using aiida nodes in to the same function without introducing aiida dependencies

… application to a xml file)

if a method starts with xml_ it REQUIRES atleast one xpath as argument

I don't know id I should use these functions in the parsers, since they are essentially the initial loading phase of these, but this could introduce some ugly dependencies

and add docstrings

All these functions will always return success since it does not make sense to make them obvious to use fro the outside (since inside they are always used without suc_return=False). There should always be a more high level function using the schema_dict, which will call these converters in the background

…und due to this

We might have to restructure the source code reference to be more structured since its becoming quite large

… organized and searchable/browseable

…nce these things should be tested in the xml_setters tests)

Some outschemas were missing and the '0.29' output schema still had '0.30' as version since it was copied from this schema

There were two errors in the outside_actions dict

…edge cases of enforced order without complicating the structure of the schema dict Essentially if a tag_order is encountered, which has no invalid/unknown tags and they are just out of order a new tag is built with the tags entered in the expected order

…r attributes and text (This way we can enter special cases in the schema_dict, where precision needs to be lower/higher)

…ida_fleur_xml_setters

janssenhenning added 30 commits February 25, 2021 19:56

Moved and modernized create_tag function from aiida_fleur. Can either…

c154096

… be used in the old way `create_tag_xpath` or utilizing the schema_dictionary functionality `create_tag`

Started adding tests for the basic xml setters

28310c0

All currently implemented basic functions have tests

218174b

pre-commit run

97fac85

Add automatic conversion to string on the level of xml_set_attrib_val…

7d8ae40

…ue/xml_set_text

Add more_itertools dependency

36bcc6f

pre-commit run

5133788

Only return nmmp_lines if there is something in there (For the direct…

0225846

… application to a xml file)

Small bugfix

93ff46e

More consistent naming of setter methods

7c67986

if a method starts with xml_ it REQUIRES atleast one xpath as argument

Add docstrings to xml_setters_basic/xml_setters_xpaths

3c22949

Added simple interface for loading input/output files

74eaba1

I don't know id I should use these functions in the parsers, since they are essentially the initial loading phase of these, but this could introduce some ugly dependencies

Add tests for xml_create_tag_schema_dict

0fd2505

pre-commit run

ea9890c

Add more tests

4eb2e25

Rename io_fleurxmlmodifier.py (io two times is kind of redundant)

1bd994d

and add docstrings

Added more tests. Nearly done with xml_setters_xpaths

f080eb7

Finished up tests of xml_setters_xpaths. Fixed some small problems fo…

7e591e1

…und due to this

Add docstrings for xml_setters_names and xml_setters_nmmpmat

7fbe94a

Add docstrings to all the facade methods in FleurXMLModifier

4c7cd09

Forgot docstrings on these methods

b64cac9

Add xml setters to docs

0511f75

We might have to restructure the source code reference to be more structured since its becoming quite large

Restructure the module index part of docs a bit to keep it a bit more…

674a1df

… organized and searchable/browseable

Update docs/index.rst

3414e97

Add fleurxmlmodifier user guide

36e75dd

janssenhenning added 9 commits March 10, 2021 13:31

Also add special alias for xcFunctional in set_attrib_value

b2eea99

Add tests for shift_value_species_label

335a37b

Add tests for shift_value

255bb3c

Added tests for set_inpchanges

a97ccbe

Add tests for the fleurxmlmodifier (no deep tests of functionality si…

ccd517c

…nce these things should be tested in the xml_setters tests)

Add tests for xml_setters_nmmpmat

7b54041

Fix bug in schema_dict tests

f8133a2

Some outschemas were missing and the '0.29' output schema still had '0.30' as version since it was copied from this schema

Add tests for io_fleurxml

6c0eeab

pre-commit run

4bee79b

janssenhenning marked this pull request as ready for review March 10, 2021 17:52

janssenhenning added 18 commits March 11, 2021 10:32

Add test for convert_to_fortran_bool

f691fdd

Add test for convert_attribute_to_xml

1e23067

Add test for convert_text_to_xml

7999412

Add tests for split_off_tag, split_off_attrib and check_complex_xpath

0fa8e98

Fix for fleurxmlmodifier

b8d941f

There were two errors in the outside_actions dict

Fix test of correcting misaligned order

9990d6e

Add missing import

fc53def

Another fix for misaligned order test

cee7934

Okay that was stupid I put the line into the wrong test :D

76cb47e

Try to fix order alignment

378ca9c

another fix for the aligning order test (now I'm sure it works :D)

a059a27

Make formatting of floats in convert_attribute_to_xml configurable fo…

02102c9

…r attributes and text (This way we can enter special cases in the schema_dict, where precision needs to be lower/higher)

Merge remote-tracking branch 'origin/aiida_fleur_xml_setters' into ai…

abf0f14

…ida_fleur_xml_setters

Merge branch 'develop' into aiida_fleur_xml_setters

06276d9

Fix merge

d8cbd88

Another fix for docs after merge

a530774

Fix heading in docs

81e4224

janssenhenning merged commit e56ca82 into develop Mar 22, 2021

janssenhenning deleted the aiida_fleur_xml_setters branch March 24, 2021 12:56

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Move aiida-fleur xml_util #23

Move aiida-fleur xml_util #23

janssenhenning commented Mar 2, 2021 •

edited

Move aiida-fleur xml_util #23

Move aiida-fleur xml_util #23

Conversation

janssenhenning commented Mar 2, 2021 • edited

janssenhenning commented Mar 2, 2021 •

edited