Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Switch to MessagePack #1045

Merged
merged 10 commits into from Apr 28, 2022
Merged

Switch to MessagePack #1045

merged 10 commits into from Apr 28, 2022

Conversation

shorowit
Copy link
Contributor

@shorowit shorowit commented Apr 21, 2022

Pull Request Description

Closes #903. Replaces use of EnergyPlus SQL output with MessagePack for performance & file size reasons.

Here are results for various simulations. Runtime refers to the entire end-to-end workflow (creation of IDF, simulation, and output processing). Filesize refers to just the sql/msgpack output files.

  Prev Runtime [s] New Runtime [s] Runtime Reduction Prev Filesize [MB] New Filesize [MB] Filesize Reduction
Simulation w/ 60-min timestep, no timeseries output 5.0 5.0 0% 0.5 0.1 80%
Simulation w/ 60-min timestep, 60-min outputs (end uses only) 6.0 5.5 8% 8.2 1.6 80%
Simulation w/ 60-min timestep, 60-min outputs (ALL outputs) 10.7 7.5 30% 28.1 5.0 82%
Simulation w/ 10-min timestep, no timeseries output 15.2 15.1 1% 0.5 0.1 80%
Simulation w/ 10-min timestep, 10-min outputs (end uses only) 21.1 17.9 15% 47.5 8.4 82%
Simulation w/ 10-min timestep, 10-min outputs (ALL outputs) 47.6 29.8 37% 171.7 29.2 83%

This PR also now allows OS-HPXML output files (e.g., results_timeseries.*) to be in MessagePack format (instead of CSV/JSON) if chosen.

Checklist

PR Author: Check these when they're done. Not all may apply. strikethrough and check any that do not apply.

PR Reviewer: Verify each has been completed.

  • Schematron validator (EPvalidator.xml) has been updated
  • Sample files have been added/updated (via tasks.rb)
  • Unit tests have been added/updated (e.g., HPXMLtoOpenStudio/tests)
  • Documentation has been updated
  • Changelog has been updated
  • openstudio tasks.rb update_measures has been run
  • Checked the code coverage report on CI
  • No unexpected changes to simulation results of sample files

@shorowit shorowit self-assigned this Apr 21, 2022
@shorowit shorowit marked this pull request as ready for review April 21, 2022 13:58
…JSON outputs and it's not much extra effort to support.
@shorowit shorowit mentioned this pull request Apr 21, 2022
5 tasks
…o msgpack

# Conflicts:
#	BuildResidentialHPXML/measure.xml
#	HPXMLtoOpenStudio/measure.xml
#	ReportUtilityBills/measure.xml
@shorowit
Copy link
Contributor Author

This provides a nice speed improvement but we're unable to proceed because it's not currently working on Windows.

…o msgpack

# Conflicts:
#	Changelog.md
#	HPXMLtoOpenStudio/measure.rb
#	HPXMLtoOpenStudio/measure.xml
#	ReportUtilityBills/measure.xml
@shorowit shorowit merged commit d62d1ae into master Apr 28, 2022
@shorowit shorowit deleted the msgpack branch April 28, 2022 18:21
@shorowit shorowit mentioned this pull request Feb 6, 2023
7 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Replace SQLite w/ msgpack
1 participant