From 29b214b63675c84a2f73316b8585e0175d84cce4 Mon Sep 17 00:00:00 2001 From: "Martin D. Weinberg" Date: Thu, 6 Nov 2025 20:34:04 -0500 Subject: [PATCH 01/15] Add some units documentation --- index.rst | 4 + topics/units.rst | 219 +++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 223 insertions(+) create mode 100644 topics/units.rst diff --git a/index.rst b/index.rst index 5f395b7..848fe8e 100644 --- a/index.rst +++ b/index.rst @@ -89,6 +89,7 @@ EXP concepts topics/bfetheory topics/codeintro topics/timesseries + topics/units topics/multistep topics/centering topics/yamlconfig @@ -106,6 +107,9 @@ mathematics used in EXP. :doc:`topics/timesseries` Descriptions for methods to analyze BFE time series +:doc:`topics/units` + Physical units in EXP + :doc:`topics/multistep` A quick review of EXP's ODE solver and multi time step ladder diff --git a/topics/units.rst b/topics/units.rst new file mode 100644 index 0000000..c5bb8d5 --- /dev/null +++ b/topics/units.rst @@ -0,0 +1,219 @@ +.. _units: + +Units in EXP and pyEXP +====================== + +Overview +-------- + +The EXP N-body code assumes the gravitational constant ``G=1`` and +accepts any mass, position and velocity units consistent with that +assumption. In other words, it does not enforce any particular unit +set. + +pyEXP will also accept any mass, position, time, and velocity units +provided by the imported simulation as well as an arbitrary value of +the gravitational constant. Explicit units types and the gravitational +constant `G` may be set at construction time as we will describe below +in :ref:`unit-schema`. pyEXP requires the user to set units +explicitly when coefficient sets are constructed. See +:ref:`intro-pyEXP-tutorial`. There is no default. + +All unit information is written into the EXP coefficient files as HDF5 +attributes. Also see :ref:`hdf5-unit-attributes` for details. + +.. _unit-scheme: + +The EXP unit schema +------------------- + +EXP requires one of the following two sequences of units: + +1. Mass, Length, Time, gravitational constant (G) + +2. Mass, Length, Velocity, gravitational constant (G) + + +Each unit is a _tuple_ which takes the form:: + + ('unit type', 'unit name', ) + +The type and name strings are checked against the allowed values as follows: + +- The ``unit type`` is one of 'length', 'mass', 'time', 'velocity' or + 'G'. The internal parser will accept a variety of aliases for these + such as 'Length', 'Len', 'len', 'l', 'L' for 'length'; 'Mass', 'm', + 'M' for 'mass'; 'Time', 't', 'T' for 'time', 'vel', + 'Vel', 'Velocity', 'v', 'V' for 'velocity'; and 'Grav', 'grav', + 'grav_constant', 'Grav_constant', 'gravitational_constant', + 'Graviational_constant' for 'G'. + +- The ``unit name`` is one of the usual unit names for each of the + ``unit type``. The allowed list is a subset of the standard + `astropy` strings. For example, 'pc' or 'kpc' for 'length'. There + is also a 'none' type which implies no assigned physical units. + +- The floating value defines the number of each unit for the type. + The value can be any valid floating-point number. + +For development reference, these allowed strings are defined in +`expui/UnitValidator.cc` in the EXP source tree. + +As an example, a Gadget user might define mass units as:: + + ('mass', 'Msun', 1.0e10) + +The full units specification is a list of tuples that includes one of +the four sequences. In C++, the list is a ``std::vector`` of tuples. + +As an example, all native EXP simulations have the following unit +lists:: + + [('mass', 'none', 1.0f), ('length', 'none', 1.0f), ('time', 'none', + 1.0f), ('G', 'none', 1.0f)] + + +Units in pyEXP +-------------- + +The ``pyEXP.basis`` classes will use the units specified by the user +in the ``pyEXP.coefs`` class created from simulation snapshots (see +:ref:`intro-pyEXP-tutorial`) or read by the coefficient factory from +an existing HDF5 file to produce accelerations using the value of the +gravitational constant and length units provided by the user. + + +.. _units-interface: + +The Units interface +------------------- + +The `pyEXP` user interface includes two member functions for explicity +setting and removing units as part of the `Coef` class. For setting +units, we have: + + .. code-block:: python + + setUnits(name, unit, value) + setUnits(list) + removeUnits(name) + +where ``name`` and ``unit` and strings and ``value`` is a float. The +list is a list of tuples of ``(name, unit, value)``. + +The C++ UI echos the functions above and adds functions to retrieve +units + + .. code-block:: C++ + + void setUnits(const std::string name, const std::string unit, float value); + void setUnits(std::vector> list); + void removeUnits(const std::string name); + std::vector> getUnits(); + +and to interact with HDF files that will only of interest to +developers creating new coefficient classes. + + +.. _hdf5-unit-attributes: + +The HDF5 specification +---------------------- + +.. note:: This section assumes basic familiarity with HDF5 and `h5py` + in particular. + +EXP HDF5 coefficient files contain a full set of metadata describing +their basis type, basis parameters, geometry, and units as attributes +of the root ("/") group. The ``snapshots`` group contains all of the +coefficient data and metadata (such as center and rotation +information) for each snapshot time in the coefficient set. + +The units information is stored in the root group as dataset named +"Units". The dataset is a sequence or list of 4 tuples. Each tuple +had three fields: two fixed length strings of sixteen (16) characters +and a float value. + +For an EXP run, the units specification appears as dataset in the root +group with the following hierarchy:: + + DATASET "Units" { + DATATYPE H5T_COMPOUND { + H5T_STRING { + STRSIZE 16; + STRPAD H5T_STR_NULLTERM; + CSET H5T_CSET_UTF8; + CTYPE H5T_C_S1; + } "name"; + H5T_STRING { + STRSIZE 16; + STRPAD H5T_STR_NULLTERM; + CSET H5T_CSET_UTF8; + CTYPE H5T_C_S1; + } "unit"; + H5T_IEEE_F32LE "value"; + } + DATASPACE SIMPLE { ( 4 ) / ( 4 ) } + DATA { + (0): { + "G", + "none", + 1 + }, + (1): { + "length", + "none", + 1 + }, + (2): { + "mass", + "none", + 1 + }, + (3): { + "time", + "none", + 1 + } + } + } + + +A units attribute list could be easily added to existing HDF5 files +using `h5py` using the schema described above. For example, assuming +that you already have an open HDF5 coefficient file named `f`, the +units described in the scheme above could be added to `f` with the +following code: + + .. code-block:: python + + import h5py + import numpy as np + + # Define the compound datatype with fixed-length UTF-8 strings and a float32 + dt = np.dtype([ + ('name', 'S16'), # Fixed-length ASCII string of 16 bytes + ('unit', 'S16'), # Fixed-length ASCII string of 16 bytes + ('value', ' Date: Thu, 6 Nov 2025 20:54:44 -0500 Subject: [PATCH 02/15] Update topics/units.rst Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- topics/units.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/topics/units.rst b/topics/units.rst index c5bb8d5..df07d3b 100644 --- a/topics/units.rst +++ b/topics/units.rst @@ -34,7 +34,7 @@ EXP requires one of the following two sequences of units: 2. Mass, Length, Velocity, gravitational constant (G) -Each unit is a _tuple_ which takes the form:: +Each unit is a ``tuple`` which takes the form:: ('unit type', 'unit name', ) From 6b9d2d539858fad9c1f6a96390b3f343c6a0a755 Mon Sep 17 00:00:00 2001 From: Martin Weinberg Date: Thu, 6 Nov 2025 20:54:56 -0500 Subject: [PATCH 03/15] Update topics/units.rst Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- topics/units.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/topics/units.rst b/topics/units.rst index df07d3b..3aee5b8 100644 --- a/topics/units.rst +++ b/topics/units.rst @@ -46,7 +46,7 @@ The type and name strings are checked against the allowed values as follows: 'M' for 'mass'; 'Time', 't', 'T' for 'time', 'vel', 'Vel', 'Velocity', 'v', 'V' for 'velocity'; and 'Grav', 'grav', 'grav_constant', 'Grav_constant', 'gravitational_constant', - 'Graviational_constant' for 'G'. + 'Gravitational_constant' for 'G'. - The ``unit name`` is one of the usual unit names for each of the ``unit type``. The allowed list is a subset of the standard From f755c2536bc18d11e0201bcda3467fbf84aec1b8 Mon Sep 17 00:00:00 2001 From: Martin Weinberg Date: Thu, 6 Nov 2025 20:55:15 -0500 Subject: [PATCH 04/15] Update topics/units.rst Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- topics/units.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/topics/units.rst b/topics/units.rst index 3aee5b8..66088eb 100644 --- a/topics/units.rst +++ b/topics/units.rst @@ -88,7 +88,7 @@ gravitational constant and length units provided by the user. The Units interface ------------------- -The `pyEXP` user interface includes two member functions for explicity +The `pyEXP` user interface includes two member functions for explicitly setting and removing units as part of the `Coef` class. For setting units, we have: From 9780658ee087a32d2e9da9e83a9f0daf286c8a40 Mon Sep 17 00:00:00 2001 From: Martin Weinberg Date: Thu, 6 Nov 2025 20:55:34 -0500 Subject: [PATCH 05/15] Update topics/units.rst Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- topics/units.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/topics/units.rst b/topics/units.rst index 66088eb..53ecc4f 100644 --- a/topics/units.rst +++ b/topics/units.rst @@ -98,7 +98,7 @@ units, we have: setUnits(list) removeUnits(name) -where ``name`` and ``unit` and strings and ``value`` is a float. The +where ``name`` and ``unit`` are strings and ``value`` is a float. The list is a list of tuples of ``(name, unit, value)``. The C++ UI echos the functions above and adds functions to retrieve From 7ae3a73b699b28ffbcc3ee4c85bf86f8cd8d1e93 Mon Sep 17 00:00:00 2001 From: Martin Weinberg Date: Thu, 6 Nov 2025 20:55:51 -0500 Subject: [PATCH 06/15] Update topics/units.rst Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- topics/units.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/topics/units.rst b/topics/units.rst index 53ecc4f..6f79181 100644 --- a/topics/units.rst +++ b/topics/units.rst @@ -111,7 +111,7 @@ units void removeUnits(const std::string name); std::vector> getUnits(); -and to interact with HDF files that will only of interest to +and to interact with HDF files that will only be of interest to developers creating new coefficient classes. From 5363eecb4c8d575191fd6f7219fd1073c379b0d6 Mon Sep 17 00:00:00 2001 From: Martin Weinberg Date: Thu, 6 Nov 2025 20:56:02 -0500 Subject: [PATCH 07/15] Update topics/units.rst Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- topics/units.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/topics/units.rst b/topics/units.rst index 6f79181..d57597c 100644 --- a/topics/units.rst +++ b/topics/units.rst @@ -131,7 +131,7 @@ information) for each snapshot time in the coefficient set. The units information is stored in the root group as dataset named "Units". The dataset is a sequence or list of 4 tuples. Each tuple -had three fields: two fixed length strings of sixteen (16) characters +has three fields: two fixed length strings of sixteen (16) characters and a float value. For an EXP run, the units specification appears as dataset in the root From c51ade39b5a6abe48acb5a89b96ed5c3428dbad9 Mon Sep 17 00:00:00 2001 From: Martin Weinberg Date: Thu, 6 Nov 2025 20:56:33 -0500 Subject: [PATCH 08/15] Update topics/units.rst Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- topics/units.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/topics/units.rst b/topics/units.rst index d57597c..20bee29 100644 --- a/topics/units.rst +++ b/topics/units.rst @@ -190,7 +190,7 @@ following code: import h5py import numpy as np - # Define the compound datatype with fixed-length UTF-8 strings and a float32 + # Define the compound datatype with fixed-length ASCII strings and a float32 dt = np.dtype([ ('name', 'S16'), # Fixed-length ASCII string of 16 bytes ('unit', 'S16'), # Fixed-length ASCII string of 16 bytes From eb91406736e54ebac265243f3df90d27885d1b25 Mon Sep 17 00:00:00 2001 From: "Martin D. Weinberg" Date: Fri, 7 Nov 2025 09:59:33 -0500 Subject: [PATCH 09/15] Some updates --- topics/units.rst | 71 ++++++++++++++++++++++++++++++++++-------------- 1 file changed, 50 insertions(+), 21 deletions(-) diff --git a/topics/units.rst b/topics/units.rst index c5bb8d5..2eeb236 100644 --- a/topics/units.rst +++ b/topics/units.rst @@ -6,18 +6,18 @@ Units in EXP and pyEXP Overview -------- -The EXP N-body code assumes the gravitational constant ``G=1`` and -accepts any mass, position and velocity units consistent with that -assumption. In other words, it does not enforce any particular unit -set. - -pyEXP will also accept any mass, position, time, and velocity units -provided by the imported simulation as well as an arbitrary value of -the gravitational constant. Explicit units types and the gravitational -constant `G` may be set at construction time as we will describe below -in :ref:`unit-schema`. pyEXP requires the user to set units -explicitly when coefficient sets are constructed. See -:ref:`intro-pyEXP-tutorial`. There is no default. +The EXP N-body code assumes the gravitational constant is unity, +``G=1``, and that mass, position and velocity units can have any +value defined by the user. In other words, EXP does not enforce any +particular unit set. + +pyEXP will also accept mass, position, time, and velocity with any +unit set defined by the imported simulation as well as an arbitrary +value of the gravitational constant. Explicit units types and the +gravitational constant ``G`` may be set at construction time as we +will describe below in :ref:`unit-schema`. pyEXP **requires** the +user to set units explicitly when coefficient sets are +constructed. See :ref:`intro-pyEXP-tutorial`. There is no default. All unit information is written into the EXP coefficient files as HDF5 attributes. Also see :ref:`hdf5-unit-attributes` for details. @@ -27,14 +27,14 @@ attributes. Also see :ref:`hdf5-unit-attributes` for details. The EXP unit schema ------------------- -EXP requires one of the following two sequences of units: +EXP requires one of the following two sequences of 4 unit types: 1. Mass, Length, Time, gravitational constant (G) 2. Mass, Length, Velocity, gravitational constant (G) -Each unit is a _tuple_ which takes the form:: +Each is defined by a _tuple_ which takes the form:: ('unit type', 'unit name', ) @@ -56,8 +56,11 @@ The type and name strings are checked against the allowed values as follows: - The floating value defines the number of each unit for the type. The value can be any valid floating-point number. -For development reference, these allowed strings are defined in -`expui/UnitValidator.cc` in the EXP source tree. +The allowed types and names may be queried interactively in pyEXP +using the ``getAllowedUnitNTypes()`` and +``getAllowedUnitNames(type)``, see :ref:`units-interface`. For +development reference, these allowed strings are defined in +``expui/UnitValidator.cc`` in the EXP source tree. As an example, a Gadget user might define mass units as:: @@ -94,22 +97,48 @@ units, we have: .. code-block:: python - setUnits(name, unit, value) + setUnits(type, unit, value) setUnits(list) - removeUnits(name) + removeUnits(type) + getAllowedUnitTypes() + getAllowedUnitName(type) -where ``name`` and ``unit` and strings and ``value`` is a float. The -list is a list of tuples of ``(name, unit, value)``. +where ``type`` and ``unit`` are strings and ``value`` is a float. The +list is a list of tuples of ``(name, unit, value)``. The last two +members return the list of unit types and their aliaes and the allowed +unit names for each unit type, respectively. + +For an example, suppose you are making a set of coefficients for a +simulation with default Gadget units. Say your coefficients instance +is called ``coefs``. The following command will register the unit set: + + .. code-block:: python + + coefs.setUnits([ ('mass', 'Msun', 1e10), ('length', 'kpc', 1.0), + ('velocity', 'km/s', 1.0), ('G', 'mixed', 43007.1) ]) + +These units will be in the HDF5 that you create using +``coefs.WriteH5Coefs('filename')``. A quick note: 'mixed' is an +allowed alias when dealing with gravitational constants that have +physical units. The C++ UI echos the functions above and adds functions to retrieve units .. code-block:: C++ + // Add or replace a unit by specifying its unit tuple void setUnits(const std::string name, const std::string unit, float value); + // Add or replace a unit(s) by specifying an array of unit tuple void setUnits(std::vector> list); - void removeUnits(const std::string name); + // Remove a unit tuple by type + void removeUnits(const std::string type); + // Retrieve the currently define unit set std::vector> getUnits(); + // Get a list of unit types and their aliases + std::vector getAllowedTypes(); + // Get a list of unit name and their aliases for a given unit type + std::vector getAllowedUnits(const std::string& type) and to interact with HDF files that will only of interest to developers creating new coefficient classes. From ecc64f8aa7a9c25eeb38f6943fe05854cdf15cb1 Mon Sep 17 00:00:00 2001 From: Martin Weinberg Date: Fri, 7 Nov 2025 10:50:31 -0500 Subject: [PATCH 10/15] Update topics/units.rst Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- topics/units.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/topics/units.rst b/topics/units.rst index f012590..8c8920d 100644 --- a/topics/units.rst +++ b/topics/units.rst @@ -57,7 +57,7 @@ The type and name strings are checked against the allowed values as follows: The value can be any valid floating-point number. The allowed types and names may be queried interactively in pyEXP -using the ``getAllowedUnitNTypes()`` and +using the ``getAllowedUnitTypes()`` and ``getAllowedUnitNames(type)``, see :ref:`units-interface`. For development reference, these allowed strings are defined in ``expui/UnitValidator.cc`` in the EXP source tree. From 69db69848b24c06ae43259009a59c3b45f91218f Mon Sep 17 00:00:00 2001 From: Martin Weinberg Date: Fri, 7 Nov 2025 10:50:49 -0500 Subject: [PATCH 11/15] Update topics/units.rst Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- topics/units.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/topics/units.rst b/topics/units.rst index 8c8920d..43b76bd 100644 --- a/topics/units.rst +++ b/topics/units.rst @@ -119,7 +119,7 @@ is called ``coefs``. The following command will register the unit set: ('velocity', 'km/s', 1.0), ('G', 'mixed', 43007.1) ]) These units will be in the HDF5 that you create using -``coefs.WriteH5Coeds('filename')``. You can query, for example, the +``coefs.WriteH5Coefs('filename')``. You can query, for example, the allowed 'mass' units with the call ``coefs.getAllowedUnitNames('mass')`` which returns: From f07b904bcc8d8fda7447f1b27e9969c98973663a Mon Sep 17 00:00:00 2001 From: Martin Weinberg Date: Fri, 7 Nov 2025 10:51:04 -0500 Subject: [PATCH 12/15] Update topics/units.rst Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- topics/units.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/topics/units.rst b/topics/units.rst index 43b76bd..ed7c923 100644 --- a/topics/units.rst +++ b/topics/units.rst @@ -154,7 +154,7 @@ units // Get a list of unit types and their aliases std::vector getAllowedTypes(); // Get a list aliases for each type - std::vector getAllowedTypeAliaes(const std::string& type); + std::vector getAllowedTypeAliases(const std::string& type); // Get a list of unit name and their aliases for a given unit type std::vector getAllowedUnits(const std::string& type) From 5eb58a724291f2202f2ec409d0d4d60d54445f66 Mon Sep 17 00:00:00 2001 From: Martin Weinberg Date: Fri, 7 Nov 2025 10:51:30 -0500 Subject: [PATCH 13/15] Update topics/units.rst Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- topics/units.rst | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/topics/units.rst b/topics/units.rst index ed7c923..6c0054f 100644 --- a/topics/units.rst +++ b/topics/units.rst @@ -239,28 +239,28 @@ following code: # Define the compound datatype with fixed-length ASCII strings and a float32 dt = np.dtype([ - ('name', 'S16'), # Fixed-length ASCII string of 16 bytes - ('unit', 'S16'), # Fixed-length ASCII string of 16 bytes - ('value', ' Date: Fri, 7 Nov 2025 10:53:05 -0500 Subject: [PATCH 14/15] Remove tabs in favor of spaces --- topics/units.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/topics/units.rst b/topics/units.rst index 6c0054f..f68386e 100644 --- a/topics/units.rst +++ b/topics/units.rst @@ -168,7 +168,7 @@ The HDF5 specification ---------------------- .. note:: This section assumes basic familiarity with HDF5 and `h5py` - in particular. + in particular. EXP HDF5 coefficient files contain a full set of metadata describing their basis type, basis parameters, geometry, and units as attributes From 0d28624ec30b751e7c19501cec39a68f0d80e30c Mon Sep 17 00:00:00 2001 From: "Martin D. Weinberg" Date: Fri, 7 Nov 2025 14:27:37 -0500 Subject: [PATCH 15/15] A few more minor changes for readability --- intro/overview.rst | 10 ++++++ topics/units.rst | 77 ++++++++++++++++++++++++++++++---------------- 2 files changed, 61 insertions(+), 26 deletions(-) diff --git a/intro/overview.rst b/intro/overview.rst index 688e5e5..1f48132 100644 --- a/intro/overview.rst +++ b/intro/overview.rst @@ -96,6 +96,11 @@ Here's the code for computing coefficients:: # coef = basis.createFromArray(data[:,0], data[:,1:4], time=3.0) + # Add your simulation units (we assume a unit-free simulation here) + # + coef.setUnits([('mass', 'none', 1.0), ('length', 'none', 1.0), + ('time', 'none', 1.0), ('G', 'none', 1.0)]) + # Make an HDF5 file # coefs = pyEXP.coefs.Coefs.makecoefs(coef) @@ -144,6 +149,11 @@ More detailed information on YAML and config parameters is available in the :ref:`What is YAML?` and :ref:`How to visualize the BFE bases used to make your coefficients` pages. +The unit system in EXP is described in more detail in :ref:`Units in +EXP and pyEXP`. For this example, we assume a unit-free +simulation but it's easy to add any unit system that is convenient for +you. + pyEXP is then ready to make the coefficients from your phase-space data. This example assumes that the mass and positions of your particles are in columns 1, 2, 3, 4 of the file and that the positions diff --git a/topics/units.rst b/topics/units.rst index f68386e..a610ddb 100644 --- a/topics/units.rst +++ b/topics/units.rst @@ -1,3 +1,7 @@ +.. role:: python(code) + :language: python + :class: highlight + .. _units: Units in EXP and pyEXP @@ -6,10 +10,10 @@ Units in EXP and pyEXP Overview -------- -The EXP N-body code assumes the gravitational constant is unity, -``G=1``, and that mass, position and velocity units can have any -value defined by the user. In other words, EXP does not enforce any -particular unit set. +The EXP N-body code assumes the gravitational constant is unity and +that mass, position and velocity units can have any value defined by +the user. In other words, the EXP N-body code does not enforce any +particular unit set consistent with ``G=1``. pyEXP will also accept mass, position, time, and velocity with any unit set defined by the imported simulation as well as an arbitrary @@ -17,12 +21,15 @@ value of the gravitational constant. Explicit units types and the gravitational constant ``G`` may be set at construction time as we will describe below in :ref:`unit-schema`. pyEXP **requires** the user to set units explicitly when coefficient sets are -constructed. See :ref:`intro-pyEXP-tutorial`. There is no default. +constructed. There is no default. See :ref:`intro-pyEXP-tutorial` for +an end-to-end example of coefficient computation. All unit information is written into the EXP coefficient files as HDF5 -attributes. Also see :ref:`hdf5-unit-attributes` for details. +attributes. Also see :ref:`hdf5-unit-attributes` for details. You +may add units to previously computed coefficient files using the +script described in :ref:`units_old`. -.. _unit-scheme: +.. _unit-schema: The EXP unit schema ------------------- @@ -34,7 +41,12 @@ EXP requires one of the following two sequences of 4 unit types: 2. Mass, Length, Velocity, gravitational constant (G) -Each unit is a ``tuple`` which takes the form:: +Other combinations are possible in principle, such as Mass, Length, +Velocity and Time. However, this would require an automatic deduction +of the gravitational constant. EXP does not current know about +physical unit conversion. This feature may be added in the future. + +Each separate unit is defined as a ``tuple`` which takes the form:: ('unit type', 'unit name', ) @@ -57,8 +69,8 @@ The type and name strings are checked against the allowed values as follows: The value can be any valid floating-point number. The allowed types and names may be queried interactively in pyEXP -using the ``getAllowedUnitTypes()`` and -``getAllowedUnitNames(type)``, see :ref:`units-interface`. For +using the :python:`getAllowedUnitTypes()` and +:python:`getAllowedUnitNames(type)`, see :ref:`units-interface`. For development reference, these allowed strings are defined in ``expui/UnitValidator.cc`` in the EXP source tree. @@ -67,13 +79,14 @@ As an example, a Gadget user might define mass units as:: ('mass', 'Msun', 1.0e10) The full units specification is a list of tuples that includes one of -the four sequences. In C++, the list is a ``std::vector`` of tuples. +the four sequences. In C++, the list is a :cpp:any:`std::vector<>` of +tuples. As an example, all native EXP simulations have the following unit -lists:: +lists: - [('mass', 'none', 1.0f), ('length', 'none', 1.0f), ('time', 'none', - 1.0f), ('G', 'none', 1.0f)] + .. code-block:: python + [('mass', 'none', 1.0f), ('length', 'none', 1.0f), ('time', 'none', 1.0f), ('G', 'none', 1.0f)] Units in pyEXP @@ -119,24 +132,28 @@ is called ``coefs``. The following command will register the unit set: ('velocity', 'km/s', 1.0), ('G', 'mixed', 43007.1) ]) These units will be in the HDF5 that you create using -``coefs.WriteH5Coefs('filename')``. You can query, for example, the -allowed 'mass' units with the call -``coefs.getAllowedUnitNames('mass')`` which returns: +:python:`coefs.WriteH5Coefs('filename')`. You can query, for example, +the allowed 'mass' units with the call +:python:`coefs.getAllowedUnitNames('mass')` which returns: .. code-block:: python ['gram', 'earth_mass', 'solar_mass', 'kilograms', 'kg', 'g', 'Mearth', 'Msun', 'None', 'none'] -A quick note: 'mixed' is an allowed alias when dealing with -gravitational constants that have physical units. You can see all -unit types with ``getAllowedUnitTypes()``; this returns ``['mass', -'length', 'time', 'velocity', 'G']``. You can see the recognized -aliases for each type using ``getAllowedTypeAliases(type)``. For -example, the recognized aliases for 'G' are: +The allowed mass units for 'G' are: .. code-block:: python - ['G', 'Grav', 'Grav_constant', 'Gravitational_constant', 'grav', 'grav_constant', 'gravitational_constant'] + ['gram', 'earth_mass', 'solar_mass', 'kilograms', 'kg', 'g', 'Mearth', 'Msun', 'None', 'none'] + + +A quick note: 'mixed' is an allowed alias when dealing with +gravitational constants that have physical units. You can see all +unit types with :python:`getAllowedUnitTypes()`; this returns +:python:`['mass', 'length', 'time', 'velocity', 'G']`. You can see +the recognized aliases for each type using +:python:`getAllowedTypeAliases(type)`. The gravitational constant is + a special case. The recognized unit names for 'G' are: :python:`['','mixed', 'none', 'unitless']``. The C++ UI echos the functions above and adds functions to retrieve units @@ -167,8 +184,8 @@ developers creating new coefficient classes. The HDF5 specification ---------------------- -.. note:: This section assumes basic familiarity with HDF5 and `h5py` - in particular. +.. note:: This and the following sections assumes basic familiarity + with HDF5 and `h5py` in particular. EXP HDF5 coefficient files contain a full set of metadata describing their basis type, basis parameters, geometry, and units as attributes @@ -226,6 +243,11 @@ group with the following hierarchy:: } +.. _units_old: + +Backward compatibility +---------------------- + A units attribute list could be easily added to existing HDF5 files using `h5py` using the schema described above. For example, assuming that you already have an open HDF5 coefficient file named `f`, the @@ -264,3 +286,6 @@ following code: # File is automatically closed on leaving the 'with' block, flush and # update saved data on disk + +You can update the :python:`data` array in the example above to match your +unit set.