In the wake of George Floyd's death at the hands of officers of the Minneapolis Police Department, New York State repealed Section 50-a of its civil service law, permitting law enforcement agencies to shield personnel records from public disclosure.
On August 20, 2020, the New York Civil Liberties Union released a dataset of 323,911 misconduct complaints filed against officers of the City of New York Police Department (NYPD) with the Civilian Complaint Review Board (CCRB). The dataset includes complaints with incident dates ranging from August 7, 1978 to May 24, 2020. In addition to incident dates, the data include officer identifiers, the commands to which officers were assigned, and CCRB dispositions.
This repository contains the data and code used in the Public Safety Lab's analyis of these misconduct data.
Note: Many of the raw and temporary files referenced are stored in a public S3 bucket (e.g., the complete output data is stored at https://psl-ccrb.s3.amazonaws.com/out/data.csv).
The process.py script reads in the raw CCRB data and merges the following additional data sources:
- 2010 U.S. Census data, mapped to 2020 NYPD precincts by John Keefe
- Annual counts of the numbers of NYPD employees, officers, arrests, offenses, and cleared offenses, compiled by Jacob Kaplan
- The NYPD stop-and-frisk dataset
- The NYPD crime complaints dataset from NYC Open Data
- The NYPD arrests dataset from NYC Open Data
The final data is batched into 6 files in the out directory (data_chunk_0 thru data_chunk_5).
The visualize.py script reads in the processed CCRB data, flattens per precinct-year and per precinct counts for various features, and creates visualizations corresponding to the figures in the NYPD Officer Misconduct Analysis report. All visualizations are accessable via our public S3 bucket with URLs following the sample pattern https://psl-ccrb.s3.amazonaws.com/viz/fig-1.html.
Please contact Public Safety Lab Director Anna Harvey or Lead Data Scientist Orion Taylor (WHO WE ARE) with questions, comments and feedback.