Skip to content
No description, website, or topics provided.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
config
etl
model DSSG Summer 2016 Work Oct 6, 2016
.gitignore
.travis.yml
LICENSE
README.md
datafiles.yaml
requirements.txt

README.md

Syracuse

Build Status

About

Syracuse is a city located in Onondaga County in Central New York. It has a population of about 150,000 and a metro area population of approximately 665,000. The Syracuse Innovation Team is a Bloomberg Philanthropies funded office created in 2015. It was set up with a specific focus on solving infrastructure problems. The city has a rich history of innovation, but at this point, the government does not do a lot of work that relies on data to help make decisions. Having heard about the work that the Center for Data Science and Public Policy did with Cincinnati on proactive blight reduction during the 2015 Data Science for the Social Good Fellowship, the Syracuse Innovation Team reached out to DSaPP about participating in the 2016 Fellowship. Infrastructure, particularly the state of water mains in the city, is especially important. Based on review of prior DSSG projects, the city believes that a partnership could be beneficial in pushing data-led initiatives forward, ultimately benefiting the infrastructure as a whole, as well as the residents. More information can be found here

Project Overview

This project entails designing and implementing a data-driven process to proacively address water main breaks and leaks. The ulimate goal is to predict areas where water mains are most at risk of breaking, and which features are the best for predicting a water main break (e.g, year laid, materials, soil composition).

Installation

Get the code

git clone https://github.com/dssg/syracuse
cd syracuse

Python Dependencies

cd syracuse
pip install -r requirements.txt

Database Configuration

Database Type: PostGreSQL 9.4 with PostGIS extension

syracuse=> select PostGIS_full_version();
                                                                                postgis_full_version
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 POSTGIS="2.1.8 r13780" GEOS="3.5.0-CAPI-1.9.0 r4084" PROJ="Rel. 4.9.2, 08 September 2015" GDAL="GDAL 1.11.4, released 2016/01/25" LIBXML="2.9.1" LIBJSON="UNKNOWN" TOPOLOGY RASTER

See database credential files for details.

Example:

PGPORT: 5432
PGHOST: "postgres.123fake.com"
PGDATABASE: "123fake"
PGPASSWORD: "123fake"

Load data into postgres

See the etl directory for details

bash ./etl/do_etl.sh

Create features from the data

See model/features directory for details

bash ./model/features/do_features.sh

Run the modeling pipeline

See model/README.md for details

Directory Structure

.
├── config
├── descriptive_stats
│   ├── mains_streets_stats
│   └── water_work_orders
├── etl
│   ├── bin
│   ├── geology
│   ├── road_ratings
│   ├── soil
│   ├── street_line_data
│   ├── tax_data
│   ├── updated_main_data
│   ├── waterorders
│   └── water_system
├── model
│   ├── config
│   ├── features
│   └── log
├── models_evaluation
└── results
    └── figures

Low hanging fruit TODO

  • Implement a logger instead of print statements
  • Make sure package is python 3 compatible
  • Make more unit tests that test whole pipeline
You can’t perform that action at this time.