Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Created a smart HDF Reader which can re-produce simulation object #2052

Merged
merged 24 commits into from Jun 27, 2022

Conversation

satwik-kambham
Copy link
Contributor

📝 Description

Type: 🚀 feature

Feature to allow storing and loading simulation objects from HDF files.

  • Adds interfaces for storing and loading simulation, runner, plasma and model objects.

closes #1251

🚦 Testing

How did you test these changes?

  • Testing pipeline
  • Other method (describe)
  • My changes can't be tested (explain why)

☑️ Checklist

  • I requested two reviewers for this pull request
  • I updated the documentation according to my changes
  • I built the documentation by applying the build_docs label

@tardis-bot
Copy link
Contributor

*beep* *bop*

Hi, human.

I'm the @tardis-bot and couldn't find your records in my database. I think we don't know each other, or you changed your credentials recently.

Please add your name and email to .mailmap in your current branch and push the changes to this pull request.

In case you need to map an existing alias, follow this example.

@codecov
Copy link

codecov bot commented Jun 13, 2022

Codecov Report

Merging #2052 (be085af) into master (20cb998) will decrease coverage by 0.31%.
The diff coverage is 43.39%.

❗ Current head be085af differs from pull request most recent head f95959d. Consider uploading reports for the commit f95959d to get more accurate results

@@            Coverage Diff             @@
##           master    #2052      +/-   ##
==========================================
- Coverage   58.34%   58.03%   -0.32%     
==========================================
  Files          76       76              
  Lines        8593     8741     +148     
==========================================
+ Hits         5014     5073      +59     
- Misses       3579     3668      +89     
Impacted Files Coverage Δ
tardis/transport/frame_transformations.py 49.01% <ø> (ø)
tardis/io/model_reader.py 67.49% <40.26%> (-30.29%) ⬇️
tardis/transport/r_packet_transport.py 20.00% <50.00%> (ø)
...lo_numba/continuum/r_packet_transport_continuum.py 15.15% <100.00%> (ø)
tardis/montecarlo/montecarlo_numba/estimators.py 41.37% <100.00%> (ø)
tardis/montecarlo/montecarlo_numba/interaction.py 28.00% <100.00%> (ø)
tardis/montecarlo/montecarlo_numba/r_packet.py 50.00% <100.00%> (ø)
.../montecarlo/montecarlo_numba/single_packet_loop.py 26.86% <100.00%> (ø)
tardis/montecarlo/montecarlo_numba/vpacket.py 21.78% <100.00%> (ø)

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

@Rodot-
Copy link
Contributor

Rodot- commented Jun 14, 2022

Just as a reminder from our meeting today and so others see it. In addition to making the model to and from hdf functions, there should be a set of tests that make sure these functions work. The test should create a simulation, run for a single iteration, export the runner and model (two separate tests) to hdf then delete them from the simulation. Load up the model and runner from the hdfs, add them back to the simulation, and run for another iteration. Verify the results are the same as running for two iterations (This might be tricky since there is some information about the random number generator state that needs to be preserved).

Additionally, make sure to add your name and email to the mailmap file!

@tardis-bot
Copy link
Contributor

*beep* *bop*

Hi, human.

I'm the @tardis-bot and couldn't find your records in my database. I think we don't know each other, or you changed your credentials recently.

Please add your name and email to .mailmap in your current branch and push the changes to this pull request.

In case you need to map an existing alias, follow this example.

@satwik-kambham satwik-kambham marked this pull request as ready for review June 21, 2022 12:02
tardis/io/model_reader.py Outdated Show resolved Hide resolved
tardis/io/model_reader.py Outdated Show resolved Hide resolved
@andrewfullard
Copy link
Contributor

Very good work! codecov thinks you need to add more tests, but I'm not sure if that will change much.

Copy link
Member

@AlexHls AlexHls left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

tardis/io/model_reader.py Show resolved Hide resolved
tardis/io/model_reader.py Outdated Show resolved Hide resolved
@Rodot- Rodot- self-requested a review June 27, 2022 14:13
@Rodot- Rodot- merged commit 136e539 into tardis-sn:master Jun 27, 2022
jayantbhakar pushed a commit to jayantbhakar/tardis that referenced this pull request Jul 29, 2022
…rdis-sn#2052)

* Added functions to store and load a runner object

* Updated hardcode file name in runner_from_hdf

* Updated store_runner_to_hdf to append to file

* Added name and email to .mailmap

* runner hdf functions convert to cgs

* store_runner_to_hdf clears group before storing

* Storing units for astropy quantities

* Formatted code with black

* Added functions to store and load model objects

* Fixed typo and decoded string from bytes

* Fixed typo

* Added test for model_to_dict

* Formatted previous test code

* Added test for store_model_to_hdf

* Fixed formatting of model_to_hdf

* Fixed issue with test for model_to_dict

* Added test for runner_to_dict

* Fixed issue with single packet seed storage

* Updated test null value of single packet seed

* Added test for store_runner_to_hdf

* Updated to use isinstance and hasattr

* runner_to_dict returns dicts instead of iterators

* Reordered imports to match pep8 style
andrewfullard pushed a commit that referenced this pull request Aug 3, 2022
* rpacket_tracker returns dataframe

* added black formatting

* replaced variable i

* added rpacket_tracker_df as separate variable

* added preallocation step

* added non-preallocated-df generator function

* tests for df generator function

* renamed variables

* assert statements corrected

* added seed column in df

* added documentation for rpacket_tracker_df

* added doc str

* initial commit

* updated documention

* moved function to r_packet.py

* added block indexing to df

Co-authored-by: Wolfgang Kerzendorf <wkerzendorf@gmail.com>

* updated the df function

* added multiindexing

* shifted the df function call

* initial commit

* interaction type added

* removed unused imports

* added tests

* initial commit

* interaction type added

* Created a smart HDF Reader which can re-produce simulation object (#2052)

* Added functions to store and load a runner object

* Updated hardcode file name in runner_from_hdf

* Updated store_runner_to_hdf to append to file

* Added name and email to .mailmap

* runner hdf functions convert to cgs

* store_runner_to_hdf clears group before storing

* Storing units for astropy quantities

* Formatted code with black

* Added functions to store and load model objects

* Fixed typo and decoded string from bytes

* Fixed typo

* Added test for model_to_dict

* Formatted previous test code

* Added test for store_model_to_hdf

* Fixed formatting of model_to_hdf

* Fixed issue with test for model_to_dict

* Added test for runner_to_dict

* Fixed issue with single packet seed storage

* Updated test null value of single packet seed

* Added test for store_runner_to_hdf

* Updated to use isinstance and hasattr

* runner_to_dict returns dicts instead of iterators

* Reordered imports to match pep8 style

* Remove yaml_load_config_file (#2062)

* Remove yaml_load_config_file

* updated mailmap

* Updated __init__.py

* Modified Function Name in Config validator according to PEP8 (#2076)

Fix function name according pep8

* Added documetation on how to compare environment before update (#2082)

* Moved `trace_packet` to tardis tansport (#2083)

* Moved the trace_packet function to the transport module, renamed the function to remove continuum reference as it is generic

* Formatted to PEP 8

* Reformatted single_packet_loop for PEP 8

* removed old continuum trace packet files

* removed references to old files

* Added some docstrings

* Missed an old function call

* Fix broken links in workflow files and goverance md (#2084)

* Adding physics introduction to documentation (#2026)

* creating files and sections

* adding images

* writing intro to tardis physics and adding it to sidebar

* changing spectra page structure

* mostly complete draft of light and matter page

* spectrum page

* fixing up physics stuff

* fixing docstring

* Revert "fixing docstring"

This reverts commit e24cadd.

* andrew's changes

* a few more edits

* adding note about angstroms

* Improved arepo parser (#1941)

* Removed line profile as option

* Removed mapping to Cartesian grid

* Removed automatic plotting after profile creation

* Fixed typo in arepo plot

* Updated documentation

* [build docs]

* [build docs] Removed outputs from doc-notebook

* Updated tests with new models

* Removed deprecated keyword in snapshot loader

* Replaced wget with requests for testfile download

* Updated file retrieval for testing

* Fixed fixture request

* Added arepo data to refdata helper

* Updated refdata location in docs

* Updated paths in tests

* Update download_reference_data.sh

* Testing areop_parser_tests

* Fixed deleted reference file

* Arepo docs fix (#2088)

* [build-docs] Fixed link in arepo docs

* [build-docs] Changed filename in arepo docs

* [build-docs]

* [build-docs]

* [build-docs]

* [build docs]

* [build-docs] Changed refdata url

* [build_docs]

* Updated mailmap

* [build_docs] Updated url for refdata download

* Creating new docs building instructions (#2097)

* creating new docs building instructions

* typos, grammar, and text in hyperlinks

* added tests

* added documentation, black formatting

Co-authored-by: Wolfgang Kerzendorf <wkerzendorf@gmail.com>
Co-authored-by: Satwik Kambham <satwik.kambham@gmail.com>
Co-authored-by: aman kumar <ak584584@gmail.com>
Co-authored-by: Rohith Varma Buddaraju <rohith.varma.buddaraju@gmail.com>
Co-authored-by: Jaladh Singhal <jaladhsinghal@gmail.com>
Co-authored-by: Jack O'Brien <jobrien585@gmail.com>
Co-authored-by: Isaac Smith <71480393+smithis7@users.noreply.github.com>
Co-authored-by: AlexHls <70367168+AlexHls@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

Create a smart HDF Reader which can re-produce simulation object
5 participants