11IMASPy in the IMAS ecosystem
2- =================
2+ ============================
33
44.. image :: imaspy_ecosystem.png
55
@@ -9,8 +9,9 @@ read time, all files in the proper paths named IDSDef.zip are searched for data
99dictionary versions to support. Alternatively explicit paths to xml files can be
1010used.
1111
12+
1213IMASPy nested structure design
13- =================
14+ ==============================
1415
1516.. image :: imaspy_structure.png
1617
@@ -33,7 +34,7 @@ https://numpy.org/doc/stable/reference/generated/numpy.lib.mixins.NDArrayOperato
3334
3435
3536IMASPy usage
36- =================
37+ ============
3738
3839The IMASPy project defines a class :py:class: `IDSRoot ` which can be instantiated
3940with a DD version number or xml_path. It reads the XML file and recreates the
@@ -74,7 +75,8 @@ An example of instantiating this structure and opening an AL backend is:
7475
7576
7677Loading multiple DD versions in the same environment
77- =============
78+ ====================================================
79+
7880The main change necessary to enable loading multiple DD versions into different
7981:py:class: `IDSRoots ` is to enable the finding of the relevant
8082:py:class: `IDSDef.xml ` files. In the ‘classical’ IMAS approach a single
@@ -96,8 +98,10 @@ is useful for development.
9698Automated tests have been provided that check the loading of all of the DD
9799versions tagged in the data-dictionary git repository.
98100
101+
99102Extending the DD set
100- ---------
103+ --------------------
104+
101105A new command has been defined python setup.py build_DD which fetches new tags
102106from git and builds IDSDef.zip
103107
@@ -112,7 +116,8 @@ All paths are searched in order.
112116
113117
114118Conversion of IDSes between DD versions
115- ===============
119+ =======================================
120+
116121The conversion between DD versions hinges on the ability to read and write to a
117122backend data store in a different version than the current DD. To enable this, IMASPy
118123needs to read both the ‘main’ in-memory DD, as well as the ‘backend’ DD. This is
@@ -165,8 +170,10 @@ backend is opened, the DD version of the IDS is unknown. At the time of
165170`get() ` the DD version is found by `read_data_dictionary_version `, which
166171reads :py:class: `IDS_properties/version_put/data_dictionary `
167172
173+
168174Implicit conversions:
169- -----------
175+ ---------------------
176+
170177- Add field
171178 - No data can be converted
172179- Delete field
@@ -176,12 +183,14 @@ Implicit conversions:
176183- Move field
177184 - Handled by searching for change_nbc_previous_name on backend and current XML
178185 - This is complex, since we may have to search many elements to find the one
179- which was renamed. Changing depths makes this harder.
186+ which was renamed. Changing depths makes this harder.
180187 - Currently implemented up to a single depth change, though multiple are
181188 possible within this design
182189
190+
183191There are some limitations of the change_nbc paradigm:
184- ------------
192+ ------------------------------------------------------
193+
185194- Forward only
186195- May require reading an arbitrary number of intermediate versions
187196- Does not cover more complex migrations
@@ -194,6 +203,7 @@ occurs the conversion can easily be done in multiple steps.
194203
195204Time slicing
196205============
206+
197207The lowlevel API provides `ual_write_slice_data ` to write only a slice (in the
198208last dimension, time) to the backend, as well as `ual_begin_slice_action `. After
199209that normal `get() ` can be used. We have implemented time slicing support, with
@@ -233,8 +243,10 @@ verifying as a global array Reading slice by slice (single slice only) The tests
233243pass on the memory and MDSPlus backend (the ASCII backend does not support
234244slicing).
235245
246+
236247Resampling
237- ============
248+ ==========
249+
238250For resampling of data we stick close to the numpy and scipy APIs. The relevant
239251method signatures are reproduced here:
240252
@@ -318,8 +330,9 @@ Or as such (explicit in-memory copy + interpolation, producing a new data leaf/c
318330
319331 Implementation unit tests can be found in `test_latest_dd_resample.py `.
320332
333+
321334Alternative resampling methods
322- -------
335+ ------------------------------
323336
324337.. code-block :: python
325338
0 commit comments