Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[TEP002] Final changes to the various to_hdf methods #601

Merged
merged 5 commits into from Jun 24, 2016

Conversation

ftsamis
Copy link
Member

@ftsamis ftsamis commented Jun 23, 2016

Note: When this is merged it will mean that the upstream to_hdf branch is ready to be merged with master.

This basically introduces a io.util.to_hdf method, as a uniform way to store various properties to HDF.

Currently the rules by which properties are stored are the following:

  1. Scalars for each module (plasma,model,etc) are stored in a Series under <path>/scalars
  2. 1D arrays are stored under <path>/property_name as distinct Series
  3. 2D arrays are stored under <path>/property_name as distinct DataFrames
  4. astropy Units are stored as their SI value
  5. As an exception to 2 and 3, the runner, when stored in an HDF file, will have an arrays value which will contain the 1D arrays, instead of having a separate Series for each one of them. Don't ask me why, I don't know.
  6. There is a special case for model.plasma.levels which is of type pandas.MultiIndex which in turn if passed to a pandas.Series for storage throws a NotImplementedError. Therefore I store it as a pandas.DataFrame which in turn produces a PerformanceWarning when calling to_hdf on it.

@ftsamis ftsamis changed the title Final changes to the various to_hdf methods [TEP002] Final changes to the various to_hdf methods Jun 23, 2016
@ftsamis ftsamis force-pushed the to_hdf-cleanup branch 2 times, most recently from ba31a58 to af516c4 Compare June 23, 2016 14:04
@wkerzendorf wkerzendorf merged commit 31fa4cd into tardis-sn:to_hdf Jun 24, 2016
@ftsamis ftsamis deleted the to_hdf-cleanup branch December 20, 2016 15:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants