Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update to use new data #10

Open
ghing opened this issue Jun 30, 2017 · 5 comments
Open

Update to use new data #10

ghing opened this issue Jun 30, 2017 · 5 comments

Comments

@ghing
Copy link
Contributor

@ghing ghing commented Jun 30, 2017

Hannah Cushman at datamade pointed out that the state released new IUCR codes. We should update this library to reflect the new codes.

@hancush

This comment has been minimized.

Copy link

@hancush hancush commented Jun 30, 2017

Hi!

This (.xlsx) looks like the best and most recent resource via the Illinois State Police, dated December 29, 2016.

I did some quick and dirty munging, looks like there are about 75 codes in the update that aren't in the lib's data (.csv). (Probably a good idea to cq this. 😅)

The statutes may take more time to reconcile because I'm not sure we can assume they're in the same format.

@ghing

This comment has been minimized.

Copy link
Contributor Author

@ghing ghing commented Jun 30, 2017

@hancush Thanks! Do we have to worry about IUCRs mapping to different ILCS codes? Or is this something that we should verify?

@hancush

This comment has been minimized.

Copy link

@hancush hancush commented Jul 5, 2017

Yes, in some cases the ILCS code may have been updated or repealed. Luckily old IUCR codes are "retained for historical purposes" in the data.

Here's the key provided by ISP (emphasis added):

The statutes associated with many offense codes have been repealed over the past six years. These offense codes have been ‘retired’ and retained for historical purposes only. ‘Retired’ offense codes will never be reassigned to a different offense. ‘Retired’ offense codes are denoted with the phrase ‘retained for historical purposes’ in the ILCS reference field.

Some offense codes had very minor changes to either the name of the offense, or the ILCS reference. These changes are denoted in bold black font. The entire ILCS reference was placed in bold though the change may have been as minor as adding or removing a suffix.

In only a few instances an offense code was both re-named and the ILCS reference updated. This only occurred when the revised offense mirrored the previous offense.
Example:
4410 previous Destruction of Draft Card
4410 revised Mutilation of Draft Card

Additional offense codes have been created and are represented in blue font.

Not every ILCS statute is represented in the revised code table. It will continue to be the practice of the I-UCR Program to refrain from creating an offense code for an offense that would never be reported in any of the I-UCR program reporting elements. Agencies are encouraged to create in-house codes, using the 6000 series or higher, for offenses outside the realm of UCR reporting.

Agencies are encouraged to begin use of the revised offense codes immediately.

(The blue/bolding is in the source .xlsx I linked above.)

@hancush

This comment has been minimized.

Copy link

@hancush hancush commented Aug 16, 2017

Did a full diff of the existing and updated codes. This output is all codes where the ILCS statute is different in the update than it is in the existing data. Some just get more specific (including (a)/(b), etc.) and some are entirely different statutes. I can sort thru by hand.

(Makefile and comparison script plus source data also in that gist.)

@hancush

This comment has been minimized.

Copy link

@hancush hancush commented Aug 23, 2017

Reading back over the notes, I had some thoughts.

If the primary objective of this lib is to convert ILCS statutes to IUCR codes, preserving historical code/statute pairs as we update the data is in our best interest.

I propose adding two columns to the underlying data:

  • year_published – Year the source data was published.
  • year_retired – If a code is retired, year the data indicating its retirement was published, i.e. these codes would have a year_retired value of 2016. Null if code is active as of the most recent update.

We could also update the API, taking a leaf from census' book by returning results from the most recent update by default but allowing a year to be passed for older data (h/t @fgregg).

Does this seem reasonable, @ghing?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.