Bug fixes for the preprocessing steps #142
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR fixes two bugs that were preventing the preprocessing steps from running (i.e.
python baus.py --mode preprocessing
).There were a couple of places where the
settings
injectable needed to be updated tomappings
, to reflect a change in yaml files from a few months ago.Due to changes in the base data, I had to loosen a requirement in the
allocate_jobs()
helper function that's used in thepreproc_jobs
step.The
baseyear_taz_controls.csv
file lists job counts for each TAZ, which are allocated to specific buildings using this code. If there are jobs but no buildings for a particular TAZ,allocate_jobs()
raises an error and crashes.This is now happening for a couple of TAZ's. I modified the code to log the problem and move on, which lets the remainder of the preprocessing complete. The mismatches will need to be resolved on the data side (see separate discussion in Slack).
Testing
With these changes, preprocessing runs successfully on my machine (Mac, Python 3.6) using a clean copy of the repo and base data. Output in the log looks reasonable, aside from the problem allocating jobs that's noted above.