Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TSV Speed Improvements #228

Merged
merged 1 commit into from
Apr 12, 2019
Merged

TSV Speed Improvements #228

merged 1 commit into from
Apr 12, 2019

Conversation

shorowit
Copy link
Contributor

This PR substantially speeds up integrity checks for TSVs with large numbers of rows (and has the side benefit of speeding up sampling) by using caching.

Results for the multifamily project:

Scenario Before [sec] After [sec] Savings
Rake integrity check 1133 110 90%
10k sampling 454 363 20%

I verified that the sampling results are unchanged and that the integrity checks still catch the variety of errors that could be in the TSVs.

@joseph-robertson any chance you could do a quick test run to make sure simulations don't somehow break?

@shorowit shorowit changed the title Speed improvements TSV Speed Improvements Apr 12, 2019
@codecov-io
Copy link

Codecov Report

Merging #228 into master will increase coverage by <.01%.
The diff coverage is 79.16%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #228      +/-   ##
==========================================
+ Coverage   91.18%   91.18%   +<.01%     
==========================================
  Files         166      166              
  Lines       48183    48177       -6     
==========================================
- Hits        43934    43929       -5     
+ Misses       4249     4248       -1
Impacted Files Coverage Δ
resources/buildstock.rb 65.59% <79.06%> (+0.14%) ⬆️
resources/run_sampling.rb 86.59% <80%> (-0.3%) ⬇️
project_resstock_national/tests/test.rb
project_resstock_testing/tests/test.rb 83.33% <0%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 78591ff...85cc629. Read the comment docs.

Copy link
Contributor

@joseph-robertson joseph-robertson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Testing project seems to run ok.

@shorowit shorowit merged commit 469e01c into master Apr 12, 2019
@shorowit shorowit deleted the speed_improvements branch April 12, 2019 19:30
@joseph-robertson joseph-robertson added this to the ResStock v2.1.0 milestone Jan 30, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants