Skip to content

Commit

Permalink
Add docs about database versions
Browse files Browse the repository at this point in the history
  • Loading branch information
youngmit committed Jan 16, 2020
1 parent ad87848 commit 360aed0
Show file tree
Hide file tree
Showing 2 changed files with 41 additions and 1 deletion.
30 changes: 29 additions & 1 deletion armi/bookkeeping/db/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,35 @@
The database can be visualized through various tools such as XTVIEW.
This module contains factories for selecting and building DB-related objects
This module contains factories for selecting and building DB-related objects.
Some notes on versions
----------------------
Persistent storage of ARMI models has seen many changes throughout the years.
Calculation results were originally stored on a SQL database (version 1), which has been
fully deprecated at this point.
Version 2 was the first to use HDF5 as the primary storage format. This was beneficial,
as it did not rely on any external infrastructure to operate, and benefited from the
suite of tools that help interact with HDF5 files. It was eventually replaced because
it did not store a complete model of the reactor, but rather a ghost of assembly, block,
and reactor parameters that could be applied to an existing reactor model (so long as
the dimensions were consistent!). This led to loading reactors being inconvenient and
error-prone, and also posed a limitation for representing more complex systems that have
non-core components.
Version 3 was created to make the schema more flexible and to permit storing the entire
reactor model within the HDF5 file. All objects in the ARMI Composite Model are written
to the database, and the model can be recovered in its entirety just from the HDF5 file.
Since it's inception, it has seen a number of tweaks to improve its functionality and
fix bugs.
Being a serialization format, the code associated with reading and writing database
files may not benefit from Don't Repeat Yourself (DRY) practices in the same way as
other code. Therefore, we do not share much code between different major versions of the
databases. Minor revisions (e.g. M.(N+1)) to the database structure should be simple
enough that specialized logic can be used to support all minor versions without posing a
maintenance burden. A detailed change log should be maintained of each minor revision.
"""
import os
from typing import Optional, List, Tuple
Expand Down
12 changes: 12 additions & 0 deletions armi/bookkeeping/db/database3.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,18 @@
for a given object or collection of object from the database file. When interacting with
the database file, the ``Layout`` class is used to help map the hierarchical Composite
Reactor Model to the flat representation in the database.
Minor revision changelog
------------------------
- 3.1: Improve the handling of reading/writing grids.
- 3.2: Change the strategy for storing large attributes from using an Object Reference
to an external dataset to using a special string starting with an "@" symbol (e.g.,
"@/c00n00/attrs/5_linkedDims"). This was done to support copying time node datasets
from one file to another without invalidating the references. Support is maintained
for reading previous versions, and for performing a ``mergeHistory()`` and converting
to the new reference strategy, but the old version cannot be written.
"""
import collections
import copy
Expand Down

0 comments on commit 360aed0

Please sign in to comment.