Skip to content

feat: support ResStock and multifamily building handling#32

Open
nllong wants to merge 2 commits into
feat/support-measure-package-comparisonfrom
feat/support-resstock
Open

feat: support ResStock and multifamily building handling#32
nllong wants to merge 2 commits into
feat/support-measure-package-comparisonfrom
feat/support-resstock

Conversation

@nllong

@nllong nllong commented Jul 23, 2026

Copy link
Copy Markdown
Member

Summary

Adds support for NREL's residential building stock dataset, ResStock, alongside the existing ComStock support, and handles ResStock's representation of multifamily buildings.

Stacked on #31 (measure/package comparison) — this targets that branch, not main, so it will show that PR's diff too until it merges.

Background: where ResStock lives, and multifamily handling

ResStock is hosted on the same OEDI data lake as ComStock (nrel-pds-building-stock/end-use-load-profiles-for-us-building-stock/{year}/resstock_*), but differs in a few important ways I confirmed by inspecting the bucket directly:

  • Metadata partitioning: only by state (state=DE/DE_upgrade0.parquet), not state+county like ComStock.
  • Building types: residential housing categories, not commercial (Single-Family Detached, Multi-Family with 5+ Units, etc.), stored in in.geometry_building_type_recs.
  • County format: in.county_name has no state prefix ("Kent County", not "DE, Kent County").
  • Measure crosswalk: an Excel file (measure_name_crosswalk_res_{year}_{release}.xlsx), not csv.
  • Releases: only resstock_amy2018_release_1 has been remastered into the current, consistent 2025 layout so far — the newer resstock_amy2018_release_2 still lives at its original (differently-structured) 2024 path, so it isn't supported yet.

Multifamily buildings: ResStock simulates individual dwelling units, not whole buildings — a "Multi-Family with 5+ Units" metadata row is one apartment unit, not the building. There's no shared "building id" tying units back to a real building; each is an independently sampled, weighted record. Relevant columns: in.geometry_building_number_units_mf (units in that unit's building), in.geometry_building_horizontal_location_mf/in.geometry_building_level_mf (the unit's position, affecting heat transfer with neighbors), and weight/in.units_represented (population scaling).

What changed

  • Extracted BuildingStockProcessor (building_stock_processor.py): a shared base class with the S3 listing/downloading, list_upgrades(), and process_building_time_series() logic that's identical between ComStock and ResStock (same bucket, same timeseries layout, same upgrades_lookup.json convention). ComStockProcessor now subclasses it, unchanged behaviorally — all existing tests pass without modification.
  • Added ResStockProcessor (resstock_processor.py):
    • SUPPORTED_RELEASES = {"release_1": ...} (only the one confirmed-working layout, for now).
    • RESSTOCK_BUILDING_TYPES — the 5 valid in.geometry_building_type_recs values, validated in the constructor.
    • State-only partitioned process_metadata()/process_metadata_for_upgrades() (simpler than ComStock's, no county sub-listing needed).
    • county_name filtering without the state prefix, get_measure_crosswalk()/find_upgrade_id() adapted to the xlsx format.
  • Added openpyxl dependency (to read the xlsx crosswalk).
  • Tests: tests/test_resstock_processor.py mirrors the ComStock test suite — release/building-type validation, mocked default-upgrades behavior, and integration tests for metadata download/filtering (including both multifamily categories and county-name-without-prefix), caching, list_upgrades, get_measure_crosswalk, find_upgrade_id, process_metadata_for_upgrades, and time series downloads.
  • README: new "ResStockProcessor Class" section covering the differences from ComStock and a "Handling Multifamily Buildings" walkthrough.
from pathlib import Path
from resstock_processor import ResStockProcessor

processor = ResStockProcessor(
    state="DE", county_name="All", building_type="Multi-Family with 5+ Units", upgrade="0",
    base_dir=Path("./datasets/resstock"),
)
multifamily_units_df = processor.process_metadata(save_dir=processor.base_dir)
multifamily_units_df["weight"].sum()  # weighted total real housing units this sample represents

Validation

  • uv run pytest tests/ -m unit ✅ (16 passed — release/building-type validation, mocked behaviors, for both ComStock and ResStock)
  • uv run pytest tests/ -m integration ✅ (27 passed, ~1 min — full ComStock suite unchanged + new ResStock metadata/multifamily/crosswalk/timeseries tests for Delaware)
  • uv run mypy ✅ (6 source files)
  • uv run pre-commit run --all-files
  • Manual smoke tests confirming metadata download, multifamily filtering, find_upgrade_id, and time series downloads all work end-to-end against the live OEDI bucket

Nicholas Long and others added 2 commits July 23, 2026 06:50
Adds support for NREL's residential building stock dataset, ResStock, alongside
the existing ComStock support, and documents/handles ResStock's per-dwelling-unit
representation of multifamily buildings.

Background: ResStock is hosted on the same OEDI data lake as ComStock
(nrel-pds-building-stock/end-use-load-profiles-for-us-building-stock/{year}/resstock_*),
but differs in several ways: metadata is partitioned only by state (not
state+county), building types are residential housing categories (not commercial),
county names don't include a state prefix, and the measure crosswalk is an Excel
file, not csv. Multifamily buildings are represented at the *dwelling unit* level:
one metadata row is one simulated unit (not a whole building), with
weight/in.units_represented for population scaling and
in.geometry_building_number_units_mf/in.geometry_building_horizontal_location_mf/
in.geometry_building_level_mf describing the unit's context within its building.

- Extract a shared BuildingStockProcessor base class (building_stock_processor.py)
  containing the S3 listing/downloading, upgrade package lookup
  (list_upgrades), and time series download logic that is identical between
  ComStock and ResStock. ComStockProcessor now subclasses it, keeping its own
  state+county partitioned metadata logic and csv-based measure crosswalk.
  BuildingStockRelease replaces the former ComStock-only ComStockRelease
  dataclass (same shape, shared name); public symbols used elsewhere
  (SUPPORTED_RELEASES, DEFAULT_RELEASE, ComStockProcessor) are unaffected.
- Add ResStockProcessor (resstock_processor.py), a new subclass with its own
  SUPPORTED_RELEASES (release_1 -> resstock_amy2018_release_1; ResStock hasn't
  been remastered across multiple releases into one consistent layout yet, unlike
  ComStock, so only this one is supported for now), state-only partitioned
  metadata download, RESSTOCK_BUILDING_TYPES validation (residential housing
  categories including the two multifamily categories), county_name filtering
  without a state prefix, and an xlsx-based get_measure_crosswalk/find_upgrade_id.
  process_metadata_for_upgrades works the same as ComStockProcessor's, for
  comparing a dwelling unit across measure packages.
- Add openpyxl as a dependency (needed to read ResStock's .xlsx measure
  crosswalk).
- Add tests/test_resstock_processor.py mirroring the ComStock test suite:
  unit tests for release/building-type validation and mocked
  default-to-every-upgrade behavior, and integration tests for metadata
  download/filtering/caching (including both multifamily categories and the
  county_name-without-state-prefix format), list_upgrades, get_measure_crosswalk,
  find_upgrade_id, process_metadata_for_upgrades, and time series downloading.
  All existing ComStockProcessor tests still pass unchanged after the refactor.
- Document ResStockProcessor, its differences from ComStockProcessor, and how to
  handle multifamily buildings in README.md.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant