Skip to content
master
Switch branches/tags
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
csv
 
 
raw
 
 
txt
 
 
 
 
 
 
 
 
 
 
 
 
 
 

README.md

budgetScrape

Node scraper for NYC Budget PDFs.

Uses NYC "supporting schedules" budget documents, outputs a csv for lowest-level budgeted amounts with associated hierarchy. (OMB publishes pdfs here, this scraper grabs data from the 2000-ish page long "Supporting Schedules", which you can see on each financial plan page such as this one.

The hierarchy looks like this:

Agency > Unit of Appropriation > Responsibility Center > Budget Code > Object Class > Object and Dollar Amount

For example:

002 Mayoralty > 020 Office of the Mayor > 0006 Counsel to the Mayor > 0230 Mayor's Judiciary Committee > 01 F/T Salaried > 001 Full Year Positions $202,774

##How to Use

Clone this repo, install dependencies npm install

First, scrape raw text out of PDFs using node scrape.js. This script is looking for pdf files in the /raw directory, and will output a text file of the same name in /txt.

Next run node parsetxt.js {filepath} replacing {filepath} with the path to the .txt file you want to parse. This will create a csv of the same name in /csv.

The FY 2017 Executive Budget is provided as an example in this repo, you may need to modify the script to get different columns of text depending on which budget docs you are parsing.

I make no guarantee that this thing is working properly, please check your work against the source document. Pull Requests Welcomed!

##Just give me the damn data

Executive Budget 2017 - CSV

About

Node scraper for NYC Budget PDFs

Resources

Releases

No releases published

Packages

No packages published