KPDB for City of Yes for Housing Opportunity

[damonmcc]

Project description

DCP has proposed a set of zoning text amendments called City of Yes for Housing Opportunity (CYHO). The Housing team must develop a new analytical model to do environmental review of CYHO. To do that, they need a version of KPDB where the dates of source data stop at or around 6/30/23.

Requirements

• KPDB source data versions:
  • ZAP data from mid-June or mid-July
  • Housing DB 23Q2
• Thorough QA by the Housing and Capital Planning teams
• Fixes to any data issues flagged by QA

Timing

• KPDB aggregations have duplicates and odd unit counts #547
• the Housing team needs a version of KPDB where the dates of source data stops at or around 6/30/23, specifically:
  • ZAP data from mid-June or mid-July
  • Housing DB 23Q2
• GIS shares the map with Housing team during the week of 3/11
• update columns in KPDB aggregation tables #665
• review KPDB wiki for accuracy
• https://github.com/NYCPlanning/edm-overview/issues/1009

---

Notes

New KPDB aggregation table Schema Doc from GIS

[damonmcc]

cc @AmandaDoyle

had issues with missing columns in ZAP data archived from June through July. used version 20230905 and confirmed with the Housing team that the date and date_type columns can be used to filter out unwanted records

generated outputs via this PR test run and will share with Housing Team when they choose which Sharepoint folder they'd like it in

[damonmcc]

built and shared with Housing team for QA on 10/3 via their Sharepoint folder

[damonmcc]

per closing of #252, must cherry pick or independently merge to main the fixes from #275

then using those fixes in this branch, must build and share with Housing team

[damonmcc]

next steps from 11/30

1. Damon sends HPD-related source files to Sam for revisions
2. Damon sends ZAP record ID correction file to Sam for revisions
   • these revisions will be based on confirmations that certain old rezonings that are in ZAP shouldn't be in KPDB
3. Damon runs KPDB with:
   • these and other updated corrections files
   • changes to the code to move records with no geometry to their own table
   • changes to the code to undo the duplicate records fix (since they may be caused by duplicate DOB records that the QATeam will correct)
4. QA Team reviews that new build with a focus on:
   • corrected rezonings
   • DOB records that have been assigned to multiple projects

[damonmcc]

build from 12/12 has issues with some corrections not being applied

[damonmcc]

build from 12/13 is in QA

[damonmcc]

corrections based on the 12/13/23 build were give to DE on 1/5/24 and used in build runs here

latest build is in QA

[damonmcc]

most urgent priority is to fix issues flagged by GIS in aggregation outputs

[damonmcc]

notes from Housing Map demo on 2/23

• nice map
• "future units without phasing" are pretty much completed units. by design, we haven't been dropping records with 100% completed units from KPDB so that people can see all units related to a project (past and future)
• Housing Team would like to investigate records with net units and no phasing

[damonmcc]

awaiting new Data Dictionary from GIS. will use it to revise columns in KPDB aggregation tables (with a focus on the NTA and CDTA tables)