Database of NYC Housing Data
Clone or download

README.md

nyc-db

Let's research the landlord! New York City is in a housing crisis. Some landlords leave their buildings in despair and let their tenants suffer without heat in winter. Others evict their tenants, legally or illegally, in order to flip buildings and profit off of gentrification. Affordable housing is a scarce resource.

Residents, lawyers, tenants, and organizers who want to use data in their struggle turn to proprietary databases and resources, like PropertyShark, designed for real estate or contend with CSV and printouts from city websites. NYC-DB aims to give technologists and researchers who want to volunteer their time helping community groups who are defending the city against the real estate industry a leg up by providing a ready-to-use database filled with housing data.

NYC-DB builds a postgresql database containing the following datasets:

  • Department of City Planning's Pluto: versions 16v2, 17v1, and 18v1
  • DOB Job Filings
  • DOB Complaints
  • HPD Violations
  • HPD Registrations
  • HPD Complaints
  • Department of Finance Rolling Sales
  • Tax bills - Rent Stabilization Unit Counts (John Krauss's data)
  • ACRIS
  • 2017 Marshal Evictions

NYC-DB is a python3 command line program that downloads and loads datasets into postgres.

Get a copy

Just want a copy of the database?

Here are the latest versions available to download from S3:

License: CC BY-NC-SA 4.0

It's ~2.5gb compressed and ~14gb decompressed.

If you have aws cli installed, you can download it easily this way: aws s3 cp s3://nyc-db/nyc-db-2018-09-28.sql.bz2 ./

To decompress: bunzip2 nyc-db-2018-09-28.sql.bz2

Load the db: psql -d database-name -f nyc-db-2018-09-28.sql

Build it yourself!

nycdb cli

To manage and create copies of the database yourself, you can see the nycdb command line tool available on pypi: pip3 install nycdb

see src/README.rst for more information on using the command line tool.

Using the Makefile to build the database

As a convenience you can create the database in one go using this command:

make nyc-db DB_HOST=localhost DB_DATABASE=nycdb DB_USER=databaseuser DB_PASSWORD=mypassword

Setup the database and API on a cloud server

See the folder /ansible for ansible playbooks to setup the database on a sever.

Acknowledgments

Future datasets to add:

  • census data

LICENSE: AGPLv3

NYC-DB - Postgres database of NYC housing data
Copyright (C) 2016-2018 Ziggy Mintz

This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.

You should have received a copy of the GNU Affero General Public License
along with this program.  If not, see <https://www.gnu.org/licenses/>.

The database files provided on this page are licensed CC BY-NC-SA 4.0.