Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2019 session planning #2681

Open
jamesturk opened this issue Nov 15, 2018 · 37 comments

Comments

@jamesturk
Copy link
Member

@jamesturk jamesturk commented Nov 15, 2018

In past years we've turned off all the scrapers until we could quickly verify that things are looking normal on state sites. We've seen enough weirdness where we were getting 2017 bills despite hitting the API with the 2016 slug or vice versa that this sort of became unofficial protocol.

I know a few states are in special session & a few have prefiles, but we can turn those back on really quick, or potentially even vet them before I make the big push to task-definitions to turn them off.

When turning back on we also want to give each state a quick once-over to make sure that there weren't any hard-coded year/session values (I think we got all of these, but they creep in from time to time)

I was thinking I'd flip this switch to turn things off next week, and we can use this ticket as a master list to get things turned back on after vetting.

How to help: If you've run a states bill/votes scraper and verified it is getting 2019 data (or is in a special session) and taken a look at the code to look for hard-coded session/year values then leave a comment and we'll update the list below. Next week I'll make a PR to turn off all non-checked states and then as states are checked we'll turn them back on. (If a state needs work first, we'll discuss that in a separate ticket, like #2554)

  • Alaska
  • Louisiana
  • Alabama
  • Arizona
  • Arkansas
  • California
  • Colorado
  • Connecticut
  • Delaware
  • District of Columbia
  • Florida
  • Georgia
  • Hawaii
  • Idaho
  • Illinois
  • Indiana
  • Iowa
  • Kansas
  • Kentucky
  • Maine
  • Maryland
  • Massachusetts
  • Michigan
  • Minnesota
  • Mississippi
  • Missouri
  • Montana
  • Nebraska
  • Nevada
  • New Hampshire
  • New Jersey
  • New Mexico
  • New York
  • North Carolina
  • North Dakota
  • Ohio
  • Oklahoma
  • Oregon
  • Pennsylvania
  • Puerto Rico
  • Rhode Island
  • South Carolina
  • South Dakota
  • Tennessee
  • Texas
  • Utah
  • Vermont
  • Virginia
  • Washington
  • West Virginia
  • Wisconsin
  • Wyoming

cc @jmcarp @showerst

@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Nov 15, 2018

Sounds good to me. We've got a few states already running the 2019 prefiles that look fine; i've checked these off above.

I've also got a patch incoming with a few states that have added empty 2019 sessions, so I added commented out session entries and added 'em to the ignore list, just to avoid bobsled errors. Sounds like i can hold off on doing any more of those if we're going to turn them off, so i'll push that commit later today.

@jamesturk once you turn stuff off; i'll start the vetting process from the bottom of the alphabet, and you can start from the top? @jmcarp if you have time to jump in maybe you can at like Maryland?

jamesturk added a commit that referenced this issue Nov 25, 2018
@jamesturk

This comment has been minimized.

Copy link
Member Author

@jamesturk jamesturk commented Dec 18, 2018

All caught up adding these to task-definitions, I should be able to start adding more myself soon.

P.S. when checking a box mind posting a comment here so I can know which ones are new? I realize GitHub doesn't make it easy to tell

@jamesturk jamesturk referenced this issue Dec 18, 2018
52 of 52 tasks complete
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Dec 18, 2018

@jamesturk no problem.

@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Dec 19, 2018

  • Added the KY 2018 Special, which you only need to scrape as a one-off (it's over.)
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Dec 19, 2018

  • Added UT 2019.
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Dec 19, 2018

  • TX is verified working
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Dec 20, 2018

  • Added WY 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Dec 21, 2018

  • Added CA 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Dec 26, 2018

  • Added ND 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 2, 2019

JMCarp Added

  • PA 2019
  • RI 2019

RI has no bills yet on my run.

@jamesturk jamesturk pinned this issue Jan 2, 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 2, 2019

  • Added NY 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 3, 2019

  • Added MA 2019
@csnardi

This comment has been minimized.

Copy link
Contributor

@csnardi csnardi commented Jan 3, 2019

I believe IN 2019 was added in #2733.

@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 3, 2019

@csnardi thanks, forgot to comment that one.

@jamesturk :

  • Added IN 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 3, 2019

  • Added DC 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 7, 2019

  • Added ID 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 7, 2019

  • Added CO 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 7, 2019

  • Added KS 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 8, 2019

  • Added OH 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 8, 2019

  • Added MS 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 9, 2019

  • Added VT 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 9, 2019

  • WI 2019 is live
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 9, 2019

  • Added MI 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 9, 2019

  • Added CT 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 9, 2019

  • Added IL 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 10, 2019

  • Added MN 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 10, 2019

  • Restored NM 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 11, 2019

  • Added OR 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 11, 2019

  • Added AL 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 11, 2019

  • Added NE 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 11, 2019

  • Added WV 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 11, 2019

Unchecked NC, code is in init but they'll need a new scraper.

@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 14, 2019

  • Added NC 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 15, 2019

  • Added IA 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 16, 2019

  • Added HI 2019
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Jan 16, 2019

  • PR is working with no changes (continuation of an older session)
@showerst

This comment has been minimized.

Copy link
Contributor

@showerst showerst commented Feb 12, 2019

@jamesturk jamesturk unpinned this issue Feb 13, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.