Skip to content
This repository has been archived by the owner on Jun 30, 2023. It is now read-only.
SPTKL edited this page Apr 30, 2021 · 8 revisions

Developments Database (DevDB)

GitHub release (latest SemVer) Test - github hosted database

The goal of the developments database is to provide a detailed view of both residential and non-residential development that has occured since the 2010 census. The core of the developments data base is a compilation of NYC DOB job and permit information. This data captures changes in units resulting from new buildings, major alterations, and demolitions. Rough timelines of development are captured in the DOB job status dates. To more reliably capture development completion, the database also includes dates associated with certificates of occupancy. Where possible, the database also tracks Housing New York affordable units created as part of each development.

We provide context for each development record by geocoding, then merging the data with various political, geographic, and administrative boundaries. Lot-level zoning and land-use information comes from PLUTO.

Source data

The majority of the data contained in the developments database comes from NYC DOB. These include permit issuance data, job application records, and new certificates of occupancy. Housing New York affordable housing unit information comes from HPD. Administrative and geographic boundary information primarily comes from DCP. Other contextual data comes from PLUTO

Workflow Diagram

built with mermaid

Mermaid Markdown
graph TD
    dob_jobapplications[dob_jobapplications]
    _INIT_devdb(_INIT_devdb)
    INIT_devdb{INIT_devdb}
    _GEO_devdb(_GEO_devdb)
    GEO_devdb[GEO_devdb]
    _SPATIAL_devdb(_SPATIAL_devdb)
    SPATIAL_devdb[SPATIAL_devdb]
    _OCC_devdb(_OCC_devdb)
    OCC_devdb[OCC_devdb]
    _UNITS_devdb(_UNITS_devdb)
    UNITS_devdb[UNITS_devdb]
    occ_lookup[OCC_lookup]
    status_lookup[STATUS_lookup]
    CO_devdb[CO_devdb]
    STATUS_Q_devdb[STATUS_Q_devdb]
    dob_permitissuance[dob_permitissuance]
    dob_cofos[dob_cofos]
    _MID_devdb(_MID_devdb)
    MID_devdb{MID_devdb}
    STATUS_devdb[STATUS_devdb]
    HNY_devdb[HNY_devdb]
    dob_jobapplications --> _INIT_devdb
    _INIT_devdb -->|geocoding| _GEO_devdb
    occ_lookup --> _OCC_devdb
    status_lookup --> _OCC_devdb
    _INIT_devdb --> INIT_devdb
    INIT_devdb --> |recode with occ_lookup| _OCC_devdb
    INIT_devdb --> _UNITS_devdb
    _GEO_devdb --> GEO_devdb
    GEO_devdb --> |spatial joins| _SPATIAL_devdb
    _SPATIAL_devdb --> SPATIAL_devdb
    SPATIAL_devdb --> INIT_devdb
    _OCC_devdb --> |corrections + occ_category| OCC_devdb
    OCC_devdb --> _UNITS_devdb
    _UNITS_devdb --> |corrections + units_net| UNITS_devdb
    INIT_devdb --> CO_devdb
    dob_cofos --> CO_devdb
    INIT_devdb --> STATUS_Q_devdb
    dob_permitissuance --> STATUS_Q_devdb

    INIT_devdb --> _MID_devdb
    STATUS_Q_devdb --> _MID_devdb
    CO_devdb --> _MID_devdb
    UNITS_devdb --> _MID_devdb
    OCC_devdb --> _MID_devdb

    _MID_devdb --> STATUS_devdb
    STATUS_devdb --> MID_devdb
    _MID_devdb --> MID_devdb
    MID_devdb --> | find hny matches| HNY_matches
    hpd_hny_units_by_building --> | find dob matches| HNY_matches
    HNY_matches -->  | hny_id + all_hny_units + affordable_units|HNY_devdb
    MID_devdb -->  HNY_devdb